Bug ID: JDK-8233385 Align some one-way conversion in MS950 charset with Windows

Type: CSR
Component: core-libs
Sub-Component: java.nio.charsets

Priority: P3
Status: Closed
Resolution: Approved
Fix Versions: 15

Submitted: 2019-11-01
Updated: 2020-06-23
Resolved: 2020-02-25

Summary
-------

MS950 charset encoder behaves differently as defined in the Traditional Chinese Windows specification

Problem
-------

Windows code page 950 has some n:1 byte-to-char mappings for certain code points. In JDK's MS950 charset, there are 4 char-to-byte mappings differ from Traditional Chinese Windows.<br>
(Actual issue was in https://bugs.openjdk.java.net/browse/JDK-8232161)

Solution
--------

I recommend that following 4 char-to-byte mappings need to change.<br>
<br>
Before:
<pre>
\u2550 -> \xA2\xA4
\u255E -> \xA2\xA5
\u2561 -> \xA2\xA7
\u256A -> \xA2\xA7
</pre>
After:
<pre>
\u2550 -> \xF9\xF9
\u255E -> \xF9\xE9
\u2561 -> \xF9\xEB
\u256A -> \xF9\xEA
</pre>
<br>
Definition:<br>
Traditional Chinese Windows conversion table is:<br>
https://unicode.org/Public/MAPPINGS/VENDORS/MICSFT/WINDOWS/CP950.TXT<br>
Newer MS950 definition is:<br>
https://unicode.org/Public/MAPPINGS/VENDORS/MICSFT/WindowsBestFit/bestfit950.txt<br>
<br>
\u2550, \u255E, \u2561 and \u256A are in BOX DRAWINGS Unicode range.<br>
(See attached 4Chras.png for font glyphs)<br>

Specification
-------------
N/A

After some additional information from [~naoto], moving to Approved. Please consider a release not for this change.
25-02-2020
Expected result: Java's working behavior should be the same as Windows' one. Exact change: I'd like to change one-way trip conversion definitions by changing make/data/charsetmapping/MS950.nr.<BR> No logic change is included. Working behavior: Customer's case is as follows: * He uses Traditional Chinese Windows with Version Control System (VCS). Windows implementation: * He opens the file which has \xF9\xF9 via Windows application, like Notepad. * He saved the file without any change. * \xF9\xF9 is stored as \xF9\xF9 ==> VCS does not detect the change. * He opens the file which has \xA2\xA4 via Windows application * He saved the file without any change. * \xA2\xA4 is stored as \xF9\xF9 ==> VCS can detect the change. Current implementation: * He opened the file which has \xF9\xF9 via Java application without any change * He saved the file without any change * \xF9\xF9 is stored as \xA2\xA4, then VCS can detect the changes ==> VCS can detect the change. * He opens the file which has \xA2\xA4 via Windows application * He saved the file without any change. * \xA2\xA4 is stored as \xA2\xA4 ==> VCS does not detect the change. New implementation: * He opened the file which has \xF9\xF9 via Java application without any change * He saved the file without any change * \xF9\xF9 is stored as \xF9\xF9, then VCS can detect the changes. ==> VCS does not detect the change. * He opened the file which has \xA2\xA4 via Java application without any change * He saved the file without any change * \xA2\xA4 is stored as \xF9\xF9, then VCS can detect the changes. ==> VCS can detect the change. If the change is applied, Java's working behavior is the same as Windows' one.
20-02-2020
This request will stay pended until the requested information is provided.
19-02-2020
[~itakiguchi], please explain what exactly this CSR proposes to alter, the value with of which methods would differ, etc.
18-02-2020
Sorry, I'm late. > the difference before and after is which set of box characters get mapped to? Unicode side codes are not changed. It means font glyphs are not changed.<BR> For example, MS950 - Uncode mapping are not changed<BR> \xA2\xA4 -> \u2550<BR> \xF9\xF9 -> \u2550<BR> Before change:<BR> \u2550 -> \xA2\xA4<BR> After change:<BR> \u2550 -> \xF9\xF9<BR> > Is there any JDK or Java SE specification that needs to be updated? No. This CSR does not affect Java SE specification. It just follows the latest Microsoft's CP950 specification.
16-02-2020
Marking the request as pended until the questions above are answered.
14-02-2020
[~itakiguchi], so the difference before and after is which set of box characters get mapped to? Is there any JDK or Java SE specification that needs to be updated?
14-02-2020