JDK-8246129 : ZIP entries created for DOS epoch include local timezone metadata
  • Type: Bug
  • Component: core-libs
  • Sub-Component: java.util.jar
  • Affected Version: 11,15
  • Priority: P4
  • Status: Closed
  • Resolution: Fixed
  • Submitted: 2020-05-28
  • Updated: 2020-07-02
  • Resolved: 2020-06-10
The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.
JDK 15 JDK 16
15 b27Fixed 16Fixed
Description
ADDITIONAL SYSTEM INFORMATION :
All OSes; the bug is platform-independent.

A DESCRIPTION OF THE PROBLEM :
(This is regression from JDK7, but the above form doesn't allow specifying that as an option.)

Build tools needing deterministic timestamps use January 1, 1980 (the MSDOS epoch) as a "zero" value.  Unfortunately, using midnight (00:00:00) of that day causes extra metadata (e.g. local timezone information) to be embedded so that the ZIP files are no longer generated hermetically.

The root cause is that java.util.zip.ZipEntry. DOSTIME_BEFORE_1980 is actually a timestamp in 1980 (exactly midnight).

The bug can be seen in this commit: http://hg.openjdk.java.net/jdk8u/jdk8u/jdk/rev/9a3a791cd28b (The commit message doesn't indicate where it was backported from.)

STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
1) Run MakeZips (source code attached) which creates four ZIP files.  This sets the local timezone to two different values, creating a pair of ZIP files for each.
2) Compare the sizes of the files
3) Compare the contents of pairs out1{a,b}.zip and out2{a,b}.zip

EXPECTED VERSUS ACTUAL BEHAVIOR :
EXPECTED -
1) all created ZIP files should have the same size (138 bytes)
2) out1a.zip should be identical to out1b.zip
3) out2a.zip should be identical to out2b.zip
ACTUAL -
1) files are not all the same size:
$ wc -c *.zip
     156 out1a.zip
     156 out1b.zip
     138 out2a.zip
     138 out2b.zip

2) out1{a,b} differ:
$ cmp out1{a,b}.zip
out1a.zip out1b.zip differ: char 45, line 1

3) (sanity check) out2{a,b} are identical:
$ cmp out2{a,b}.zip ; echo $?
0

---------- BEGIN SOURCE ----------
import java.io.File;
import java.io.FileOutputStream;
import java.util.Calendar;
import java.util.GregorianCalendar;
import java.util.TimeZone;
import java.util.zip.*;

public class MakeZips {
  public static void main(String[] args) throws Exception {
    TimeZone.setDefault(TimeZone.getTimeZone("America/Los_Angeles"));

    // java.util.zip.ZipEntry.DOSTIME_BEFORE_1980 is actually 1980-01-01 00:00:00,
    // meaning that the beginning of the DOS epoch causes extra metadata to be written
    makeZip(
        new File("out1a.zip"),
        new GregorianCalendar(1980, Calendar.JANUARY, 01, 0, 0, 0).getTimeInMillis());

    // ZIP files use 2-second precision, so the next possible timestamp is 1980-01-01 00:00:02;
    // this doesn't have extra metadata.
    makeZip(
        new File("out2a.zip"),
        new GregorianCalendar(1980, Calendar.JANUARY, 01, 0, 0, 2).getTimeInMillis());

    TimeZone.setDefault(TimeZone.getTimeZone("GMT"));

    // java.util.zip.ZipEntry.DOSTIME_BEFORE_1980 is actually 1980-01-01 00:00:00,
    // meaning that the beginning of the DOS epoch causes extra metadata to be written
    // --- this file will differ from out1a.zip
    makeZip(
        new File("out1b.zip"),
        new GregorianCalendar(1980, Calendar.JANUARY, 01, 0, 0, 0).getTimeInMillis());

    // ZIP files use 2-second precision, so the next possible timestamp is 1980-01-01 00:00:02;
    // this doesn't have extra metadata.
    // --- this file will be identical to out1a.zip
    makeZip(
        new File("out2b.zip"),
        new GregorianCalendar(1980, Calendar.JANUARY, 01, 0, 0, 2).getTimeInMillis());
  }

  private static void makeZip(File f, long time) throws Exception {
    try (ZipOutputStream out = new ZipOutputStream(new FileOutputStream(f))) {
      ZipEntry e = new ZipEntry("entry.txt");
      e.setTime(time);
      out.putNextEntry(e);
      out.write(new byte[] {0, 1, 2, 3});
      out.closeEntry();
    }
  }
}
---------- END SOURCE ----------

CUSTOMER SUBMITTED WORKAROUND :
Build tools which need to clear timestamp metadata must be updated to use 1980-01-01 00:00:02, or any time after that instead of midnight.

See https://github.com/bazelbuild/bazel/pull/10976 for example.

FREQUENCY : always



Comments
Changeset: 99136026 Author: Claes Redestad <redestad@openjdk.org> Date: 2020-06-10 20:53:04 +0000 URL: https://git.openjdk.java.net/lanai/commit/99136026
02-07-2020

URL: https://hg.openjdk.java.net/jdk/jdk/rev/153061a3c51b User: redestad Date: 2020-06-10 18:38:59 +0000
10-06-2020

For many years now I've been recommending that users stop using timestamps in the 20th century. 2010-01-01 is a fine timestamp to use when trying to achieve build determinism. Whereas 1980-01-01 is inviting the sort of trouble seen here. But the current jdk behavior is a small bug. 1980-01-01 00:00:00 is representable.
01-06-2020

The observations on WIndows 10: JDK 11: Fail JDK 15: Fail ILW=MML=P4
29-05-2020