JDK-8241602 : jlink does not produce reproducible jimage files
  • Type: Bug
  • Component: tools
  • Sub-Component: jlink
  • Affected Version: 15
  • Priority: P3
  • Status: Resolved
  • Resolution: Fixed
  • Submitted: 2020-03-25
  • Updated: 2020-09-11
  • Resolved: 2020-05-06
The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.
JDK 13 JDK 15
13.0.5Fixed 15 b22Fixed
Related Reports
Relates :  
JDK-8252730 - jlink does not create reproducible builds on different servers
Relates :  
JDK-8243666 - ModuleHashes attribute generated for JMOD and JAR files depends on timestamps
Description
Consider the following:

$ jlink --add-modules java.se --output image1
$ jlink --add-modules java.se --output image2
$ diff -r image image2
Binary files image1/lib/modules and image2/lib/modules differ

One issue is that the image writer processes the archives in iteration order and that can vary from run to run. That issue is fixed with `jlink.patch` attached to this bug and it least to make the above reproducible on Linux and macOS. There seems to another issue on Solaris (solaris-sparcv9), maybe because it is big endian. The contents of the image files are identical but the index is different:

$ jimage info image1/lib/modules 
 Major Version:  1
 Minor Version:  0
 Flags:          0
 Resource Count: 17925
 Table Length:   17925
 Offsets Size:   71700
 Redirects Size: 71700
 Locations Size: 359375
 Strings Size:   407279
 Index Size:     910082

$ jimage info image2/lib/modules 
 Major Version:  1
 Minor Version:  0
 Flags:          0
 Resource Count: 17925
 Table Length:   17925
 Offsets Size:   71700
 Redirects Size: 71700
 Locations Size: 358493
 Strings Size:   407279
 Index Size:     909200

Windows seems to intermittent, most of the time the image is identical but periodically (1 in 100?), it is different.
Comments
Fix request (13u) on behalf of asemenov@azul.com This fix is necessary for 13u, too. The patch applies cleanly, tier1 tests all pass.
11-09-2020
URL: https://hg.openjdk.java.net/jdk/jdk/rev/bb90831da57e User: jlaskey Date: 2020-05-06 15:50:48 +0000
06-05-2020
Archives are stored in a Set (HashSet) but no hashcode/equals defined in Archive subclasses. ``` diff -r 0e2da3416503 src/jdk.jlink/share/classes/jdk/tools/jlink/internal/DirArchive.java --- a/src/jdk.jlink/share/classes/jdk/tools/jlink/internal/DirArchive.java Tue May 05 15:25:27 2020 -0400 +++ b/src/jdk.jlink/share/classes/jdk/tools/jlink/internal/DirArchive.java Wed May 06 11:42:40 2020 -0300 @@ -153,4 +153,20 @@ private static String getPathName(Path path) { return path.toString().replace(File.separatorChar, '/'); } + + @Override + public int hashCode() { + return Objects.hash(dirPath, moduleName); + } + + @Override + public boolean equals(Object obj) { + if (obj instanceof DirArchive) { + DirArchive other = (DirArchive)obj; + return Objects.equals(dirPath, other.dirPath) && + Objects.equals(moduleName, other.moduleName); + } + + return false; + } } diff -r 0e2da3416503 src/jdk.jlink/share/classes/jdk/tools/jlink/internal/JarArchive.java --- a/src/jdk.jlink/share/classes/jdk/tools/jlink/internal/JarArchive.java Tue May 05 15:25:27 2020 -0400 +++ b/src/jdk.jlink/share/classes/jdk/tools/jlink/internal/JarArchive.java Wed May 06 11:42:40 2020 -0300 @@ -133,4 +133,21 @@ protected JarFile getJarFile() { return jarFile; } + + @Override + public int hashCode() { + return Objects.hash(file, moduleName, version); + } + + @Override + public boolean equals(Object obj) { + if (obj instanceof JarArchive) { + JarArchive other = (JarArchive)obj; + return Objects.equals(file, other.file) && + Objects.equals(moduleName, other.moduleName) && + Objects.equals(version, other.version); + } + + return false; + } } diff -r 0e2da3416503 src/jdk.jlink/share/classes/jdk/tools/jlink/internal/JmodArchive.java --- a/src/jdk.jlink/share/classes/jdk/tools/jlink/internal/JmodArchive.java Tue May 05 15:25:27 2020 -0400 +++ b/src/jdk.jlink/share/classes/jdk/tools/jlink/internal/JmodArchive.java Wed May 06 11:42:40 2020 -0300 @@ -174,4 +174,20 @@ return new JmodEntry(path, resourceName, type, entry); } + + @Override + public int hashCode() { + return Objects.hash(file, moduleName); + } + + @Override + public boolean equals(Object obj) { + if (obj instanceof JmodArchive) { + JmodArchive other = (JmodArchive)obj; + return Objects.equals(file, other.file) && + Objects.equals(moduleName, other.moduleName); + } + + return false; + } } ```
06-05-2020
webrev: http://cr.openjdk.java.net/~jlaskey/8241602/webrev-02 RFR https://mail.openjdk.java.net/pipermail/core-libs-dev/2020-May/066248.html
06-05-2020
All Pass https://mach5.us.oracle.com/mdash/jobs/jlaskey-jdk-20200506-1435-10821733
06-05-2020
I was going to try some further experiments (Why i didn't close the bug out.) Just relaying what I know so far.
01-05-2020
Can you identify where the "file system ordering" comes into this? We should be able to do something with that to make it reproducible.
01-05-2020
So, ... The order is dependent on module dependencies and file system ordering. The only way to create consistent results on SOLARIS is to add the "--order-resources=*" option to the jlink command.
01-05-2020
Should run jimage list --verbose to see if something is missing or changes size between runs.
29-04-2020