JDK-8262108 : SimpleDateFormat formatting broken for sq_MK Locale
  • Type: Bug
  • Component: core-libs
  • Sub-Component: java.text
  • Affected Version: 16,17
  • Priority: P3
  • Status: Closed
  • Resolution: Fixed
  • Submitted: 2021-02-21
  • Updated: 2021-09-23
  • Resolved: 2021-04-16
The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.
JDK 17
17 b19Fixed
Related Reports
Relates :  
JDK-8247781 - Day periods support
Description
ADDITIONAL SYSTEM INFORMATION :
Microsoft Windows [Version 10.0.19041.746]

openjdk version "16" 2021-03-16
OpenJDK Runtime Environment (build 16+36-2231)
OpenJDK 64-Bit Server VM (build 16+36-2231, mixed mode, sharing)

Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
Maven home: C:\Java\apache-maven-3.6.3\bin\..
Java version: 16, vendor: Oracle Corporation, runtime: C:\Program Files\Java\openjdk\jdk-16
Default locale: en_US, platform encoding: Cp1252
OS name: "windows 10", version: "10.0", arch: "amd64", family: "windows"

A DESCRIPTION OF THE PROBLEM :
Trying to format and parse back a Date using SimpleDateFormat and the sq_MK Locale creates the wrong Date.


REGRESSION : Last worked in version 15.0.2

STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
git clone https://github.com/apache/commons-lang.git
mvn clean test -Dtest=FastDateParserTest -V


EXPECTED VERSUS ACTUAL BEHAVIOR :
EXPECTED -
Not a unit test failure.
ACTUAL -
[ERROR] Failures:
[ERROR]   FastDateParserTest.testParses:529->validateSdfFormatFdpParseEquality:696 Locale: 'sq_MK', parse: 'mb.K./2000/shkurt/0010/0012/0000/00/e paradites/e enjte', format: 'GGGG/yyyy/MMMM/dddd/hhhh/mmmm/ss/aaaa/EEEE', time zone: 'America/New_York' ==> expected: <Thu Feb 10 00:00:00 EST 2000> but was: <Sat Feb 12 12:00:00 EST 2000>


---------- BEGIN SOURCE ----------
Class: https://github.com/apache/commons-lang.git
https://github.com/apache/commons-lang/blob/master/src/test/java/org/apache/commons/lang3/time/FastDateParserTest.java
Git repo: https://github.com/apache/commons-lang.git
---------- END SOURCE ----------

CUSTOMER SUBMITTED WORKAROUND :
None.


FREQUENCY : always
Comments
No response from the submitter.
23-09-2021
Changeset: 64e21307 Author: Jaikiran Pai <jpai@openjdk.org> Date: 2021-04-16 06:45:39 +0000 URL: https://git.openjdk.java.net/jdk/commit/64e21307
16-04-2021
Reopening this since I now have additional details that helps reproduce this issue. I looked into this issue in the commons-lang project which still fails against the Java 16 GA version (as well as upstream OpenJDK master branch). I have now been able to narrow this down to a simple (jtreg) testcase which reproduces the problem. Based on what I have been able to understand, looking at the code in the commons-lang project (specifically the FastDateParser class), IMO it exposes a bug in the implementation of java.util.Calendar.getDisplayNames(...) in the JDK: public Map<String, Integer> getDisplayNames(int field, int style, Locale locale) The introduction of day period support in recent versions of Java as part of https://bugs.openjdk.java.net/browse/JDK-8247781 seems to have introduced this bug. The Calendar.getDisplayNames() when passed a field of the Calendar is expected to return a Map which contains the display name as the key and a (valid) field value corresponding to that field. However, what's happening is that this method is now returning invalid (integer) values for the Calendar.AM_PM field where day period data is present. The result of this is that, code of the following form in the commons-lang library, results in setting incorrect value in the calendar instance for the AM_PM field: // find display names for AM_PM field final Map<String, Integer> displayNames = calendar.getDisplayNames(Calendar.AM_PM, Calendar.ALL_STYLES, locale); .... String n = // choose one of the display names from that returned map, essentially when parsing a string to a date int fieldVal = displayNames.get(n); // set the AM_PM field value on the calendar calendar.set(Calendar.AM_PM, fieldVal); In the specific test in commons-lang where this is failing, the locale is "sq" and the display name is "e paradites". This should have translated into a field value == Calendar.AM == 0, instead the integer returned by the Calendar.getDisplayNames for this combination is 5 (which is an invalid value for the Calendar.AM_PM field; only 0 and 1 are valid values for this field). Once the calendar is set with this incorrect AM_PM field value, a subsequent Calendar.getTime() call in that library code ends up returning the incorrect/wrong/unexpected date which is what causes the test failure. A simple jtreg test which reproduces this issue with Calendar.getDisplayNames(...) for Calendar.AM_PM field is follows: import org.testng.Assert; import org.testng.annotations.Test; import java.util.Calendar; import java.util.Locale; import java.util.Map; /** * @test * @bug 8262108 * @summary Verify the results returned by Calendar.getDisplayNames() API * @comment Locale providers: COMPAT,SPI * @run testng/othervm -Djava.locale.providers=COMPAT,SPI CalendarDisplayNamesTest * @comment Locale providers: CLDR * @run testng/othervm -Djava.locale.providers=CLDR CalendarDisplayNamesTest */ public class CalendarDisplayNamesTest { @Test public void testAM_PMDisplayNameValues() { final int[] styles = new int[]{Calendar.ALL_STYLES, Calendar.SHORT_FORMAT, Calendar.LONG_FORMAT, Calendar.SHORT_STANDALONE, Calendar.LONG_STANDALONE, Calendar.SHORT, Calendar.LONG}; for (final Locale locale : Locale.getAvailableLocales()) { for (final int style : styles) { final Calendar cal = Calendar.getInstance(); cal.setLenient(false); final Map<String, Integer> names = cal.getDisplayNames(Calendar.AM_PM, style, locale); if (names == null) { continue; } for (final Integer fieldValue : names.values()) { Assert.assertTrue(fieldValue == Calendar.AM || fieldValue == Calendar.PM, "Invalid field value " + fieldValue + " for calendar field AM_PM, in locale " + locale + " with style " + style); } } } } } I've a potential fix for this issue which I've verified locally against the commons-lang project as well as some existing Calendar testcases in the OpenJDK upstream repo. I'll open a PR shortly with the necessary details.
13-04-2021
Requested the information for above question from the submitter.
23-02-2021
Looks like this test case is failing from JDK16 b25, in which the day periods are supported (JDK-8247781). I briefly looked at the test case, and the reason the test is failing is that the method FastDateParser.parse() seems to return an incorrect date. Since the method is in the user's library, asking for more info as to why parsing is failing.
22-02-2021
The observations on Windows 10: JDK 11: Passed. JDK 16ea+24: Passed. JDK 16ea+25: Failed, expected: <Thu Feb 10 13:00:00 CST 2000> but was: <Sun Feb 13 01:00:00 CST 2000> JDK 17ea+6: Failed.
22-02-2021