JDK-8226704 : DateTimeFormatter.parse throws exception after tzdata updates
  • Type: Bug
  • Component: core-libs
  • Sub-Component: java.time
  • Priority: P2
  • Status: Resolved
  • Resolution: Not an Issue
  • OS: generic
  • CPU: generic
  • Submitted: 2019-06-24
  • Updated: 2020-06-12
  • Resolved: 2020-06-11
Related Reports
Relates :  
JDK-8234347 - "Turkey" meta time zone does not generate composed localized names
Relates :  
JDK-8234347 - "Turkey" meta time zone does not generate composed localized names
Relates :  
JDK-8181157 - CLDR Timezone name fallback implementation
Description
The issue is reproducible with latest 11u builds.

Steps to reproduce:

1) update to tzdata2018i using a TimeZone updater tool (Note: not reproducible with tzdata2018g)
2) run the test (in attachments)
3) if the test throws exception, the issue is reproducible

Exception in thread "main" java.time.format.DateTimeParseException: Text '2019-06-18T20:11:40.196PST' could not be parsed: null
	at java.base/java.time.format.DateTimeFormatter.createError(DateTimeFormatter.java:2021)
	at java.base/java.time.format.DateTimeFormatter.parse(DateTimeFormatter.java:1882)
	at Test.main(Test.java:8)
Caused by: java.lang.NullPointerException
	at java.base/java.time.format.DateTimeFormatterBuilder$PrefixTree.prefixLength(DateTimeFormatterBuilder.java:4532)
	at java.base/java.time.format.DateTimeFormatterBuilder$PrefixTree.add0(DateTimeFormatterBuilder.java:4401)
	at java.base/java.time.format.DateTimeFormatterBuilder$PrefixTree.add(DateTimeFormatterBuilder.java:4396)
	at java.base/java.time.format.DateTimeFormatterBuilder$ZoneTextPrinterParser.getTree(DateTimeFormatterBuilder.java:4143)
	at java.base/java.time.format.DateTimeFormatterBuilder$ZoneIdPrinterParser.parse(DateTimeFormatterBuilder.java:4254)
	at java.base/java.time.format.DateTimeFormatterBuilder$CompositePrinterParser.parse(DateTimeFormatterBuilder.java:2374)
	at java.base/java.time.format.DateTimeFormatter.parseUnresolved0(DateTimeFormatter.java:2111)
	at java.base/java.time.format.DateTimeFormatter.parseResolved0(DateTimeFormatter.java:2040)
	at java.base/java.time.format.DateTimeFormatter.parse(DateTimeFormatter.java:1878)
	... 1 more
Comments
Thanks, Martin, Kiran. This is indeed a regression caused by the fix to JDK-8181157, where CLDR names which partially provides names for certain metazones. As a result, it allowed zones without any l10n names which resulted in failing generating names (in this case, newly added "Asia/Qostanay"). This has been recently fixed with JDK-8234347 where any missing l10n names are synthesized. So for JDK15 and after, this is/will not be an issue, but it needs to be backported to prior releases. Anyway, the issue can be reproduced by adding a dummy zone such as: --- Zone Dummy/Dummy +00:00 +0 --- To any of the make/data/tzdata/asia|europe|... file, and build/test.
11-06-2020
Kiran: agreed. You can recast this bug as "make it safer for users of stock binaries to run tzupdater in future". Or "make it safer for corporate jdk teams or linux distro teams to patch their jdk sources with a new tzdata release".
11-06-2020
I was able to reproduce the issue locally too. Thanks, Martin, It is indeed generic and will affect future tzdata updates that introduce a new timezone. Issue is seen in - DateTimeFormatterBuilder.java zoneStrings = TimeZoneNameUtility.getZoneStrings(locale); for (String[] names : zoneStrings) { String zid = names[0]; if (!regionIds.remove(zid)) { nonRegionIds.add(zid); continue; } zid = ZoneName.toZid(zid, locale); int i = textStyle == TextStyle.FULL ? 1 : 2; for (; i < names.length; i += 2) { tree.add(names[i], zid); } } names[i] is null for a new timezone unless it is defined under TimeZoneNames.java Looks to me like a regression introduced by an enhancement that went into jdk11u+. Linking the bug. A fallback mechanism is required for new timezones. [~naoto] Can I get your opinion on the bug too, please? Thanks, Kiran
10-06-2020
I changed the summary to be more generic and reassigned to Naoto - hope that's OK. I tried my little test JDK_8226704 with all recent openjdk versions and it runs successfully on all - no exceptions thrown. But the underlying problem likely remains - a future tzdata update that introduces a new timezone name may result in failures. JDK code should probably be more resilient.
09-06-2020
I raised priority to P2 because of the "time-bomb" nature of this problem. Updating tzdata from the IANA distribution, whether via tzupdater or patching the source files, should be a safe operation. But when tzdata adds a new timezone, additional metadata about that timezone is missing, and jdk sources need to be resilient to that. The failure in our own JDK was fixed by applying the diff to TimeZoneNames.java @@ -691,6 +691,9 @@ {"Asia/Pontianak", WIT}, {"Asia/Pyongyang", KST}, {"Asia/Qatar", ARAST}, + {"Asia/Qostanay", new String[] {"Kostanay Standard Time", "QOST", + "Kostanay Summer Time", "QOSST", + "Kostanay Time", "QOST"}}, {"Asia/Qyzylorda", new String[] {"Qyzylorda Time", "QYZT", "Qyzylorda Summer Time", "QYZST", "Qyzylorda Time", "QYZT"}}, (Also, there's no actual time zone with short form "QOST" known to the non-java timezone world - that data looks like it was invented, not imported) (Also, have you considered open-sourcing tzupdater?)
20-09-2019
We ran into the same problem without using the tzupdater tool. Instead, we patched jdk11's tzdata files with the tzdata2018i data, without changing any of the java source files. (This is probably equivalent to running tzupdater) This was the tzdata update that added a new tijmezone - Asia/Qostanay. Then this java program: import java.time.*; import java.time.format.*; public class JDK_8226704 { public static void main(String[] args) { DateTimeFormatter formatter = DateTimeFormatter.ofPattern("yyyy-MM-dd HH:mm:ss zzz"); ZonedDateTime zdt = ZonedDateTime.parse( "2017-01-01 01:00:00 PST", formatter); } } fails with Exception in thread "main" java.time.format.DateTimeParseException: Text '2017-01-01 01:00:00 PST' could not be parsed: null at java.base/java.time.format.DateTimeFormatter.createError(DateTimeFormatter.java:2017) at java.base/java.time.format.DateTimeFormatter.parse(DateTimeFormatter.java:1952) at java.base/java.time.ZonedDateTime.parse(ZonedDateTime.java:598) at JDK_8226704.main(JDK_8226704.java:8) Caused by: java.lang.NullPointerException at java.base/java.time.format.DateTimeFormatterBuilder$PrefixTree.prefixLength(DateTimeFormatterBuilder.java:4527) at java.base/java.time.format.DateTimeFormatterBuilder$PrefixTree.add0(DateTimeFormatterBuilder.java:4396) at java.base/java.time.format.DateTimeFormatterBuilder$PrefixTree.add(DateTimeFormatterBuilder.java:4391) at java.base/java.time.format.DateTimeFormatterBuilder$ZoneTextPrinterParser.getTree(DateTimeFormatterBuilder.java:4138) at java.base/java.time.format.DateTimeFormatterBuilder$ZoneIdPrinterParser.parse(DateTimeFormatterBuilder.java:4249) at java.base/java.time.format.DateTimeFormatterBuilder$CompositePrinterParser.parse(DateTimeFormatterBuilder.java:2370) at java.base/java.time.format.DateTimeFormatter.parseUnresolved0(DateTimeFormatter.java:2107) at java.base/java.time.format.DateTimeFormatter.parseResolved0(DateTimeFormatter.java:2036) at java.base/java.time.format.DateTimeFormatter.parse(DateTimeFormatter.java:1948)
20-09-2019