JDK-8246036 : [macos] Mongolian characters and Yi Syllables are displayed as rectangles
  • Type: Bug
  • Component: client-libs
  • Sub-Component: 2d
  • Affected Version: 8u162,11,15
  • Priority: P3
  • Status: Closed
  • Resolution: Won't Fix
  • OS: os_x
  • Submitted: 2020-05-28
  • Updated: 2020-10-27
  • Resolved: 2020-10-07
Related Reports
Relates :  
Relates :  
Relates :  
Description
ADDITIONAL OS VERSION INFORMATION :
macOS 10.15.4, macOS 10.14.6

A DESCRIPTION OF THE PROBLEM :
If characters from Mongolian and Yi Syllables Unicode code ranges are entered to a text component, they are displayed correctly until a character from Tibetan or Arabic Unicode code range is entered after them. After that moment Tibetan or Arabic character is displayed properly, while all preceding Mongolian characters or Yi Syllables become displayed as rectangles.

STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
1. Go to "System Preferences -> Keyboard -> Input Sources" and add "Unicode Hex Input".
2. Compile and run the test case, whose code is provided below.
3. Press and hold Option (Alt) key and type "1821a008", then:
- for the test scenario #1 type "0f11" and release Option key. "1821" - Tibetan character.
- for the test scenario #2 type "062e" and release Option key. "062e" - Arabic character.

The bug is reproduced, if first two characters "1821a008" are displayed as rectangles, for example as it is on the attached screenshot "IncorrectCharactersWithJDK8u261b12OnMacOS10.15.6.png", where JDK 8u261 b12 was used on macOS 10.15.6. The bug is not reproduced, if all three characters are displayed correctly, for example as it is on the attached screenshot "CorrectCharactersWithJDK8u161b12OnMacOS10.15.6.png" on which JDK 8u161 b12 was used.

---------- BEGIN SOURCE ----------
import java.awt.BorderLayout;
import java.awt.Container;
import javax.swing.JFrame;
import javax.swing.JTextField;
import javax.swing.SwingUtilities;

public class TestCase {
    public static void main(String[] args) {
        SwingUtilities.invokeLater(new Runnable() {
            @Override
            public void run() {
                JFrame f = new JFrame("TestCase for JDK-8246036");
                Container c = f.getContentPane();
                c.setLayout(new BorderLayout());

                JTextField textField = new JTextField(15);
                c.add(textField, BorderLayout.CENTER);

                f.pack();
                f.setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
                f.setVisible(true);
            }
        });
    }
}
---------- END SOURCE ----------
Comments
Bug closed WNF - removing BPR label
27-10-2020

CONCLUSION: Mongolian script has never been supported by JDK officially. There is no Mongolian script in the section "Supported Writing Systems" on the next web page. https://www.oracle.com/java/technologies/javase/jdk8-jre8-suported-locales.html The test scenario in this issue is not officially supported use case for currently under development JDK 16 and any older JDK release families, hence removing the label "regression_*". Java 2D Graphics Development team confirmed that Mongolian is not supported by JDK and stated that such issue should be treated as an RFE. For example: https://bugs.openjdk.java.net/browse/JDK-8252091 Closing the bug with "Won't Fix" resolution, and leaving opportunity to file it again as a new RFE.
07-10-2020

REASON OF THE BUG: Debugging JDK 16 code showed that Mongolian character "U+1821" (" ��� ") and Yi Syllable "U+a008" (" ��� ") do not have corresponding glyphs in the used by JDK font "Lucida Grande". The following glyph values are returned by "sun.font.CCharToGlyphMapper.charToGlyph(char unicode)" for these two characters in runtime with the test case: 1. "U+1821" (" ��� ") --> CCharToGlyphMapper.charToGlyph(char) --> -6177 (Negative integer) 2. "U+a008" (" ��� ") --> CCharToGlyphMapper.charToGlyph(char) --> -40968 (Negative integer) No glyphs are found for these two characters in both cases, when they are followed by a complex character: Tibetan "U+0f11" (" ��� ") or Arabic "U+062e" (" �� "), and when they are not followed by any other character so they are displayed properly. However, when Mongolian character and Yi Syllable are not followed by any other character, they are displayed properly, because their glyphs are drawn by macOS through "sun.font.CStrike.getGlyphImagePtrsNative(long nativeStrikePtr, long[] glyphInfos, int[] uniCodes, int len)". On higher level the situation is: - when Mongolian and Yi characters are drawn correctly the whole string in the text field is successfully drawn through "sun.java2d.SunGraphics2D.drawString(String str, float x, float y)" method. - when Mongolian and Yi characters are drawn as rectangles the text field draws the string through "sun.java2d.SunGraphics2D.drawGlyphVector(GlyphVector gv, float x, float y)". POTENTIAL WORKAROUND: To set to the text field a font which has glyphs for characters from all involved in the test case scripts: Mongolian, Yi, Tibetan, Arabic. There are fonts from Google which provide glyphs for Mongolian and Yi characters, they are "Noto Sans Mongolian", "Noto Sans Yi". For JDK 8 I tried to set up JDK through the font configuration file to fallback to these two Google fonts but unfortunately font fallback mechanism is not supported by JDK 8 on macOS in fact and by documentation (https://docs.oracle.com/javase/8/docs/technotes/guides/intl/font.html#logical).
06-10-2020

THE FACTS DEFINED BY CURRENT MOMENT: It is practically proven that in JDK 16 this bug is tightly connected with the fact that in runtime the method "sun.font.FontUtilities.isComplexCharCode(int)" from the file "src/java.desktop/share/classes/sun/font/FontUtilities.java" returns "true" for Tibetan (Unicode range: 0F00 ��� 0FFF) and Arabic (Unicode range: 0600 ��� 06FF) characters. If this method is rewritten to return "false" for Tibetan and Arabic code ranges the involved in the test case characters from Mongolian and Yi Syllables Unicode ranges are displayed correctly, but text layout logic for Arabic becomes switched off and Arabic starts being entered from left to right in such JDK. Also through dependency on "sun.font.FontUtilities.isComplexCharCode(int)" method this bug is connected with the document property "javax.swing.text.AbstractDocument.I18NProperty" in the method "javax.swing.text.AbstractDocument.handleInsertString(int, String, AttributeSet)" from the file "src/java.desktop/share/classes/javax/swing/text/AbstractDocument.java". But the real reason of the bug should be that in scenario, when Mongolian characters or Yi Syllables are followed by a complex character (Tibetan or Arabic character) then glyphs for those first characters (Mongolian or Yi Syllables) are not found, since they are displayed as rectangles. Trying to understand, where from the correct glyphs are taken, when these characters are displayed successfully and why the glyphs are not found, when the characters are followed by a complex character.
30-09-2020

The next two files were attached to the bug record: 1. "CorrectCharactersWithJDK8u161b12OnMacOS10.15.6.png" - Correct and expected displaying of all three characters involved in the test case executed with JDK 8u161 b12. 2. "IncorrectCharactersWithJDK8u261b12OnMacOS10.15.6.png" - The bug reproduced with JDK 8u261 b12.
30-09-2020

Verified that the bug appeared exactly in JDK 8u162 b01 using the user's test case on macOS 10.13.6. Verified that the bug is caused by the fix for the bug JDK-7162125 by means of a custom build of JDK 8 compiled from the source code of JDK 8u162 b01 from which the fix for JDK-7162125 was manually reverted. Verified that presence or absence of the fix for the bug JDK-8147002 in JDK code does not influence this bug. Started to work on identification of the root cause of the bug.
10-09-2020