JDK-8286712 : jar -M creates unreproducible archives, possibly via extended ZIP attributes
  • Type: Bug
  • Component: tools
  • Sub-Component: jar
  • Affected Version: 11
  • Priority: P4
  • Status: Open
  • Resolution: Unresolved
  • OS: generic
  • CPU: generic
  • Submitted: 2022-05-09
  • Updated: 2022-05-17
Description
A DESCRIPTION OF THE PROBLEM :
Creating a jar file with the -M flag ("does not create a manifest file entry") appears to cause jar(1) to add extended attributes to the last file added to the archive. As files are not added to a .jar in a deterministic order (if, say, a directory is specified on the command-line, it inherits the ordering of the underlying filesystem), this means that even if the files are subsequently sorted by some other tool (I comaintain such a tool), the extended attribute is moved around independent to this hypothetical resorting. This means that jar files created with -M are nondetermistic / non-reproducible

This was first seen in the Debian threeb package: https://salsa.debian.org/reproducible-builds/strip-nondeterminism/-/issues/19

I'm not quite sure what is being stored in this extended attribute (is it required? is it a millisecond timestamp as the source seems to imply?), but oen solution would be to always sort files being added to the jar.

Here is the extended attribute (bX in zipinfo output):

$ zipinfo -l a.jar | grep -i bX
-rw----     2.0 fat     2977 bX     1587 defN 22-May-04 05:04 AdvancedDialog.class

$ zipinfo -l b.jar | grep -i bX
-rw----     2.0 fat     4448 bX     2487 defN 22-May-05 07:05 three_B.class


STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
jar cvf output.jar -C path-to-dir/ .

EXPECTED VERSUS ACTUAL BEHAVIOR :
EXPECTED -
Identical result regardless of underlying filesystem ordering.
Comments
I can try JDK 18 and 19 tomorrow. It will be interesting to see whether JDK-8276764 ("Enable deterministic file content ordering for Jar and Jmod") solved this issue.
17-05-2022
I see that the issue has been marked as affecting Java 11. Would you be able to do a quick test against Java 17 or 18 and see if it's reproducible there too?
17-05-2022
If you run the following command on the attached 'output.jar' file: $ zipinfo -v output.jar the output shows the following 4-byte extra field for the 'File1.class' entry: File1.class ... length of extra field: 4 bytes ... The central-directory extra field contains: - A subfield with ID 0xcafe (unknown) and 0 data bytes. In a normal JAR file with a manifest, that extra field is added to the directory entry for 'META-INF/'. Without a manifest, though, it ends up being added to a random file entry.
17-05-2022
The issue is reproducible. jar command when used with -M adds extended attribute to the last file (the sequence of files added are supposedly taken from the filesystem) or sorted by some other tool (in this case as the submitter maintains the tool). it can be checked with zipinfo tool zipinfo -l a.jar | grep -i bX -rw---- 2.0 fat 2977 bX 1587 defN 22-May-04 05:04 AdvancedDialog.class The issue is raised at https://salsa.debian.org/reproducible-builds/strip-nondeterminism/-/issues/19
13-05-2022