JDK-8195595 : Negative daylight saving time breaks tzdata handling
  • Type: Bug
  • Component: core-libs
  • Sub-Component: java.time
  • Priority: P3
  • Status: Resolved
  • Resolution: Duplicate
  • Submitted: 2018-01-17
  • Updated: 2019-06-24
  • Resolved: 2019-06-24
Related Reports
Duplicate :  
JDK-8212970 - TZ database in "vanguard" format support
Relates :  
JDK-8203233 - (tz) Upgrade time-zone data to tzdata2018e
Relates :  
JDK-8195837 - (tz) Upgrade time-zone data to tzdata2018c
Description
After updating in the obvious way to tzdata2018a we have a test failure as follows:

Very likely due to changes to Ireland's time zones:

A colleague reports:

"Ireland will observe standard time during the summer and have a negative DST offset during the winter, an arrangement that has not been used before."

STDOUT:
Compiling tz files!
testing!
OLD.getZoneInfoOld()[1]=1279
OLD.getZoneInfoOld()[2]=661
OLD.getZoneInfoOld()[3]=600
OLD.getAvailableIDs()=4793, total=627
OLD.getAliasTable()=1322, total=227
OLD.TotalTZ()=289 (ms)
NEW.getTimeZone()[1]=4728
NEW.getTimeZone()[2]=440
NEW.getTimeZone()[3]=310
NEW.getAvailableIDs()=33, total=627
NEW.getAliasTable()=85, total=227
NEW.TotalTZ()=114 (ms)
******************************
Eire : Eire
    offset=3600000,dstSavings=3600000,useDaylight=true,transitions=228,offsets=6,checksum=1280024022,gmtChanged=false
[NG]offset=3600000,dstSavings=0,useDaylight=true,transitions=228,offsets=6,checksum=1280024022,gmtChanged=false
    -------------
    SimpleTimeZone (NG)
       stz=java.util.SimpleTimeZone[id=Eire,offset=3600000,dstSavings=3600000,useDaylight=true,startYear=0,startMode=2,startMonth=2,startDay=-1,startDayOfWeek=1,startTime=3600000,startTimeMode=2,endMode=2,endMonth=9,endDay=-1,endDayOfWeek=1,endTime=3600000,endTimeMode=2]
      stz0=null
    -------------
STDERR:
javazic: warning: found last rules for Eire inconsistent.
java.lang.RuntimeException:   FAILED:  Eire
	at TestZoneInfo310.main(TestZoneInfo310.java:174)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.sun.javatest.regtest.agent.MainActionHelper$SameVMRunnable.run(MainActionHelper.java:229)
	at java.lang.Thread.run(Thread.java:745)
Comments
Closing this as duplicate, as this bug is now worked as part of the enhancement: JDK-8212970
24-06-2019
Updating openjdk tip to handle IANA tzdata vanguard format is clear progress, and may eventually lead to not having to worry about vanguard vs. rearguard, BUT: Until all consumers of tzdata are updated to accept vanguard data format and those updates are ubiquitous throughout the industry, my strong recommendation is to continue to consistently use rearguard format everywhere. If IANA does not provide a rearguard tarball download, then we can generate one using the makefile (we currently do this at Google). If IANA deletes the rearguard format generation script from the source distribution, then the plan is to fork it and maintain the script outside of IANA. IANA maintainers seem to believe that tzdata format changes can be adopted by the ecosystem within months, when reality is closer to a decade.
23-06-2019
(per TZ list) passing on the the warning given in https://mm.icann.org/pipermail/tz/2019-May/027978.html > The next release (2019b, which should be reasonably soon) may be a good > time to skip my publication of a rearguard-format tarball.
22-06-2019
Temporary workaround: From tzdata2018g onwards, IANA is providing rearguard format tar bundles along with the main release, as a workaround for this issue please use the rearguard tzdata bundle with -l option. e.g.: For tzdata2018g the command will look like this: java -jar tzupdater.jar -l https://web.cs.ucla.edu/~eggert/tz/release/2018g/tzdata2018g-rearguard.tar.gz The link for the rearguard tar can be obtained from the bottom of the tzdata release announce mail. As an example you can refer to this one- http://mm.icann.org/pipermail/tz-announce/2019-March/000055.html
03-05-2019
Hi Ramanand, I have a customer who has the following question/query: "tzupdater version 2.2.0-b01 (last available download) does not yet support negative DST offsets, We need to install temporary rearguard-format to be able to get the latest updates. We would like to know when tzupdater will be updated to support negative DST offsets." Can you please provide latest update on this issue. Thanks, Sergei
02-05-2019
Hi Martin, Your point is very valid, given the recent changes done by tzdata maintainers it is risky to depend on the rearguard version, which they can change/remove at any point of time in future. JDK sustaining team maintains the tzupdater tool, including integration of tzdata into different JDK release chains. And currently, I do most of the fixes and integration work related to tzdata. I am planning to start work on the fix soon.
24-05-2018
I definitely agree this is the best way to handle 2018e right now, but what about the future? tzdata maintainers obviously want us to migrate to the vanguard versions eventually, so being able to consume data in vanguard format seems something worth doing. Joda-time's strategy seems good, even though it's more work. I don't know who is maintaining the tzupdater tool, but work is probably required there. (TIL about /usr/bin/rename)
22-05-2018
Hi [~martin] and [~scolebourne], thanks for sharing your solutions. I too have used a solution similar to what is mentioned by Martin. I have just used the rearguard tar created from IANA tzdb and everything is working as expected. I have found that using the rearguard option is very simple. As per the tzdata2018e release notes: "The command 'make tarballs' now also builds the tarball tzdataVERSION-rearguard.tar.gz, which is like tzdataVERSION.tar.gz except that it uses rearguard format intended for trailing-edge data parsers. " You have to just get the latest tzdb from IANA(tzdb-latest.tar.lz), extract it and use "make tarballs". It produces the rearguard version of tzdata named correctly for that version. In the case of tzdata2018e, it produced tzdata2018e-rearguard.tar.gz. I have converted my steps into a simple script below, which will give the latest tzdata version into a tzdata-latest directory. wget https://www.iana.org/time-zones/repository/tzdb-latest.tar.lz lzip -d tzdb-latest.tar.lz mkdir tzdata-latest && tar xf tzdb-latest.tar -C tzdata-latest --strip-components 1 rm -f tzdb-latest.tar cd tzdata-latest && make tarballs shopt -s extglob rm -rf !(tzdata*rearguard.tar.gz) rename 's/(.*)-rearguard/$1/g' *
22-05-2018
It is possible to change the time-zone parser to read the negative save values and convert them back to positive ones. The process is a bit messy, but the whole file format is very messy anyway. This is the Joda-Time change: https://github.com/JodaOrg/joda-time/commit/c5c681c91c74a508e67911e0af4980846d676ba2
16-05-2018
So 2018e arrived with the negative DST offsets that we feared/expected. But tzdata maintainers also introduced a compatibility mode to get the traditional data format. Actually they moved to a model where you can ask for vanguard, main, or rearguard, but it's not obvious how to do so. At Google we chose to use the rearguard version of the 2018e tzdata everywhere, and we have locally updated openjdk source code. In part, this is to give everyone more time to handle the vanguard variant, but also we see no advantage for libraries or for users to switch. Here's a snippet of shell code we use to derive the rearguard versions of the tzdata files, that you might find useful: # Fetches the latest tz data from iana.org into $IANA_TEMP_DIR and # sets $IANA_TZDATA_VERSION # # We build the so-called "rearguard" version of tzdata because we don't # want to deal with negative zone offsets as introduced for Eire. # # Traditional tz tarballs are compressed with gzip, but the full source # distribution is compressed with lzip. fetch_iana_tzdata() { if [[ -z "${IANA_TZDATA_VERSION-}" ]]; then # This code reflects the iana.org website reorg of 2018-01 local -r iana_home="https://www.iana.org/time-zones" local -r iana_html="$(wget -qO- $iana_home)" [[ "$iana_html" =~ \ https://data.iana.org/time-zones/releases/tzdb-(20[0-9][0-9][a-z]+)\.tar\.lz ]] \ || die "Can't determine current IANA tzdata version" local -r iana_tzdata_tarball="${BASH_REMATCH[0]}" declare -rg IANA_TZDATA_VERSION="${BASH_REMATCH[1]}" wget -qO- "$iana_tzdata_tarball" | tar xf - --lzip -C "${IANA_TEMP_DIR}" local -r subdir="${IANA_TEMP_DIR}/tzdb-${IANA_TZDATA_VERSION}" # check the version file in downloaded sources local -r tarball_version="$(< "${subdir}/version")" [[ "$tarball_version" == "$IANA_TZDATA_VERSION" ]] \ || die "Unexpected IANA tzdata version mismatch: \"%s\" vs \"%s\"" \ "$tarball_version" "$IANA_TZDATA_VERSION" printf "IANA tzdata version: %s\n" "$IANA_TZDATA_VERSION" ( cd "$subdir" local -r rearguard_tarball="tzdata${IANA_TZDATA_VERSION}-rearguard.tar.gz" # See http://mm.icann.org/pipermail/tz/2018-May/026446.html make VERSION="$IANA_TZDATA_VERSION" "$rearguard_tarball" mkdir -p "$IANA_TZDATA_DIR" tar xzf "$rearguard_tarball" -C "$IANA_TZDATA_DIR" ) fi }
15-05-2018
Hi Martin [~martin] and Stephen [~scolebourne], Thank you very much for the heads up about this issue. A huge thank you to Stephen for taking a stand and explaining this issue to IANA tzdata maintainers. We should keep eye on the future IANA changes to tzdata. I have started working on the tzdata2018c integration into JDK 10/11. Till now, I have not found any TZ related test case failures after the integration. Also, all the related JCK tests (from util, time, text) are passed after this integration. I will be sending out a public review for this today and will be starting the integration into JDK8u soon after that.
29-01-2018
Although with 2018c everything is quiet for the moment, tzdata maintainers may still be planning to ship future tzdata updates with negative daylight saving time offsets. If they do that, there will be huge confusion in the timezone ecosystem. I suggest we carry a local patch to tzdata to revert any such future change, as the best of a poor choice. Else we would need to investigate and fix all supported java versions to work well with negative offsets, and that work needs to be done now.
29-01-2018
tzdata2018c was released with the troublesome changes to Irish time reverted and Google adopted it internally in the usual straightforward way without any trouble (so far!). Openjdk should also update to tzdata2018c as soon as possible. The future of Irish time handling in IANA tzdata is stiill uncertain. Discussion continues on http://mm.icann.org/pipermail/tz/2018-January. Thanks for Stephen Colebourne and others for representing the java ecosystem.
25-01-2018
Stephen, thank you very much for working with the tzdata maintainers. I'm hoping they will completely revert the changes to the Irish time zones, but right now we don't know what will happen, and we should not try to apply these tzdata changes to openjdk. Let's wait at least until 2018c.
19-01-2018
Based on my experience with ThreeTen-Backport, I think the correct approach is to undo this change when parsing the tzdb source. ie. to convert negative daylight savings back to positive ones. http://mm.icann.org/pipermail/tz/2018-January/025844.html It seems to me that very few users of Java will be happy with the method isDaylightSavings() returning true in winter (in Ireland). https://docs.oracle.com/javase/8/docs/api/java/time/zone/ZoneRules.html#getDaylightSavings-java.time.Instant- In particular, the methods on `TimeZone` are also affected where they take in a boolean flag to indicate if daylight text is required or not. https://docs.oracle.com/javase/8/docs/api/java/util/TimeZone.html#getDisplayName-boolean-int-java.util.Locale- While the meaning of all of these methods is not firmly set to state "daylight time is when the clocks have moved forward" I believe that there is a huge cultural understanding of that meaning. If we don't revert the change (by parsing tzdb differently) then I think we will just see bugs complaining that isDaylightSavings() is true in winter in Ireland. At the time of writing, v2018c will partially revert this change, but not fully.
19-01-2018