JDK-8186464 : ZipFile cannot read some InfoZip ZIP64 zip files
  • Type: Bug
  • Component: core-libs
  • Sub-Component: java.util.jar
  • Affected Version: 7,8
  • Priority: P3
  • Status: Resolved
  • Resolution: Fixed
  • Submitted: 2017-08-18
  • Updated: 2025-01-31
  • Resolved: 2017-09-20
The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.
JDK 10
10 b25Fixed
Related Reports
Relates :  
JDK-8185896 - Add test for ZIP64
Relates :  
JDK-8334048 - -Xbootclasspath can not read some ZIP64 zip files
Description
ZIP64 zip files created by InfoZip with large entries cannot be read by ZipFile.

 $ for f in OpenZip.java Makefile; do echo --- $f ---; cat $f; done; echo ---; make
--- OpenZip.java ---
public class OpenZip {
    public static void main(String[] args) throws Throwable {
        new java.util.zip.ZipFile(args[0]);
    }
}
--- Makefile ---
FILE = bigfile
ZIP = bigfile.zip
JDK = ~/jdk/jdk10
all:
	fallocate -l 5G $(FILE); zip $(ZIP) $(FILE); zipinfo -t $(ZIP); $(JDK)/bin/javac OpenZip.java; $(JDK)/bin/java OpenZip $(ZIP); rm -f $(FILE) $(ZIP)
---
fallocate -l 5G bigfile; zip bigfile.zip bigfile; zipinfo -t bigfile.zip; ~/jdk/jdk10/bin/javac OpenZip.java; ~/jdk/jdk10/bin/java OpenZip bigfile.zip; rm -f bigfile bigfile.zip
  adding: bigfile (deflated 100%)
1 file, 5368709120 bytes uncompressed, 5210192 bytes compressed:  99.9%
Exception in thread "main" java.util.zip.ZipException: invalid CEN header (bad signature)
	at java.base/java.util.zip.ZipFile$Source.zerror(ZipFile.java:1252)
	at java.base/java.util.zip.ZipFile$Source.initCEN(ZipFile.java:1212)
	at java.base/java.util.zip.ZipFile$Source.<init>(ZipFile.java:997)
	at java.base/java.util.zip.ZipFile$Source.get(ZipFile.java:960)
	at java.base/java.util.zip.ZipFile.<init>(ZipFile.java:216)
	at java.base/java.util.zip.ZipFile.<init>(ZipFile.java:148)
	at java.base/java.util.zip.ZipFile.<init>(ZipFile.java:119)
	at OpenZip.main(OpenZip.java:3)
Comments
A pull request was submitted for review. Branch: master URL: https://git.openjdk.org/jdk8u-dev/pull/452 Date: 2024-02-20 20:40:13 +0000
09-07-2024
At Google we're backporting this. The backport is "trivial" - only release engineering. http://cr.openjdk.java.net/~martin/webrevs/openjdk9/JDK-8186464-zip64-backport/
10-01-2018
URL: http://hg.openjdk.java.net/jdk10/master/rev/723486922bfe User: sherman Date: 2017-09-20 23:45:28 +0000
20-09-2017
Instead of writing a java program to call "new ZipFile", it is sufficient to simply invoke jar. When zip creates a file from so-called "streamed" input, it always creates a ZIP64 zip file, producing a smaller simpler test case. $ (set -x; for f in OpenZip.java Makefile; do cat $f; done; make all; make streamed) +/bin/zsh:1095> f=OpenZip.java +/bin/zsh:1095> cat OpenZip.java public class OpenZip { public static void main(String[] args) throws Throwable { new java.util.zip.ZipFile(args[0]); } } +/bin/zsh:1095> f=Makefile +/bin/zsh:1095> cat Makefile FILE = bigfile ZIP = bigfile.zip JDK = ~/jdk/jdk10 JAVAC = $(JDK)/bin/javac JAVA = $(JDK)/bin/java JAR = $(JDK)/bin/jar all: OpenZip.class fallocate -l 5G $(FILE); zip $(ZIP) $(FILE); zipinfo -t $(ZIP); $(JAVA) OpenZip $(ZIP); $(JAR) tvf $(ZIP); rm -f $(FILE) $(ZIP) streamed: zip $(ZIP) - < Makefile; zipinfo -t $(ZIP); $(JAR) tvf $(ZIP); rm -f $(ZIP) %.class: %.java $(JAVAC) $< clean: rm -f *.class $(FILE) $(ZIP) +/bin/zsh:1095> make all fallocate -l 5G bigfile; zip bigfile.zip bigfile; zipinfo -t bigfile.zip; ~/jdk/jdk10/bin/java OpenZip bigfile.zip; ~/jdk/jdk10/bin/jar tvf bigfile.zip; rm -f bigfile bigfile.zip adding: bigfile (deflated 100%) 1 file, 5368709120 bytes uncompressed, 5210192 bytes compressed: 99.9% Exception in thread "main" java.util.zip.ZipException: invalid CEN header (bad signature) at java.base/java.util.zip.ZipFile$Source.zerror(ZipFile.java:1252) at java.base/java.util.zip.ZipFile$Source.initCEN(ZipFile.java:1212) at java.base/java.util.zip.ZipFile$Source.<init>(ZipFile.java:997) at java.base/java.util.zip.ZipFile$Source.get(ZipFile.java:960) at java.base/java.util.zip.ZipFile.<init>(ZipFile.java:216) at java.base/java.util.zip.ZipFile.<init>(ZipFile.java:148) at java.base/java.util.zip.ZipFile.<init>(ZipFile.java:119) at OpenZip.main(OpenZip.java:3) java.util.zip.ZipException: invalid CEN header (bad signature) at java.base/java.util.zip.ZipFile$Source.zerror(ZipFile.java:1252) at java.base/java.util.zip.ZipFile$Source.initCEN(ZipFile.java:1212) at java.base/java.util.zip.ZipFile$Source.<init>(ZipFile.java:997) at java.base/java.util.zip.ZipFile$Source.get(ZipFile.java:960) at java.base/java.util.zip.ZipFile.<init>(ZipFile.java:216) at java.base/java.util.zip.ZipFile.<init>(ZipFile.java:148) at java.base/java.util.zip.ZipFile.<init>(ZipFile.java:119) at jdk.jartool/sun.tools.jar.Main.list(Main.java:1489) at jdk.jartool/sun.tools.jar.Main.run(Main.java:374) at jdk.jartool/sun.tools.jar.Main.main(Main.java:1670) 39.14s user 1.81s system 100% cpu 40.865 total +/bin/zsh:1095> make streamed zip bigfile.zip - < Makefile; zipinfo -t bigfile.zip; ~/jdk/jdk10/bin/jar tvf bigfile.zip; rm -f bigfile.zip adding: - (deflated 48%) 1 file, 425 bytes uncompressed, 219 bytes compressed: 48.5% java.util.zip.ZipException: invalid CEN header (bad signature) at java.base/java.util.zip.ZipFile$Source.zerror(ZipFile.java:1252) at java.base/java.util.zip.ZipFile$Source.initCEN(ZipFile.java:1212) at java.base/java.util.zip.ZipFile$Source.<init>(ZipFile.java:997) at java.base/java.util.zip.ZipFile$Source.get(ZipFile.java:960) at java.base/java.util.zip.ZipFile.<init>(ZipFile.java:216) at java.base/java.util.zip.ZipFile.<init>(ZipFile.java:148) at java.base/java.util.zip.ZipFile.<init>(ZipFile.java:119) at jdk.jartool/sun.tools.jar.Main.list(Main.java:1489) at jdk.jartool/sun.tools.jar.Main.run(Main.java:374) at jdk.jartool/sun.tools.jar.Main.main(Main.java:1670)
19-08-2017
zip(1) says """zip automatically uses the Zip64 extensions when files larger than 4 GB are added to an archive"""
19-08-2017
We happened to be looking at this code recently and observed it to be incorrect ... because a zip implementation can add a ZIP64 EOCD header for any reason whatsoever. You always have to look for it! But because the last entry might happen to contain data that looks like a ZIP64 EOCD, you had better do a lot of validation on it - check multiple fields (not just the signature) and if anything looks bad, fall back to assuming this was a false trail. yes, reading zip files involves heuristics! appnote.txt has the strong hint """However ZIP64 format may be used regardless of the size of a file""" parse_manifest is appropiately paranoid: /* * Tells whether p appears to be pointing at a valid ZIP64 end header. * Values censiz, cenoff, and entries are the corresponding values * from the non-ZIP64 end header. We perform extra checks to avoid * misidentifying data from the last entry as a ZIP64 end header. */ static jboolean is_zip64_endhdr(int fd, const Byte *p, jlong end64pos, jlong censiz, jlong cenoff, jlong entries) { if (ZIP64_ENDSIG_AT(p)) { jlong censiz64 = ZIP64_ENDSIZ(p); jlong cenoff64 = ZIP64_ENDOFF(p); jlong entries64 = ZIP64_ENDTOT(p); return (censiz64 == censiz || censiz == ZIP64_MAGICVAL) && (cenoff64 == cenoff || cenoff == ZIP64_MAGICVAL) && (entries64 == entries || entries == ZIP64_MAGICCOUNT) && is_valid_end_header(fd, end64pos, censiz64, cenoff64, entries64); } return JNI_FALSE; }
19-08-2017
Hmm, the spec says 4.4.1.4 If one of the fields in the end of central directory record is too small to hold required data, the field should be set to -1 (0xFFFF or 0xFFFFFFFF) and the ZIP64 format record should be created.
19-08-2017
infozip creates the ZIP64END and ZIP64LOCSIG even they are not really necessary ... need double check the info zip spec src/java.base/share/classes/java/util/zip/ZipFile.java *** 1120,1132 **** --- 1120,1134 ---- comment = new byte[comlen]; if (readFullyAt(comment, 0, comlen, end.endpos + ENDHDR) != comlen) { zerror("zip comment read failed"); } } + /* if (end.cenlen == ZIP64_MAGICVAL || end.cenoff == ZIP64_MAGICVAL || end.centot == ZIP64_MAGICCOUNT) + */ { // need to find the zip64 end; try { byte[] loc64 = new byte[ZIP64_LOCHDR]; if (readFullyAt(loc64, 0, loc64.length, end.endpos - ZIP64_LOCHDR)
19-08-2017