JDK-6190518 : Support Zip files with more than 64k entries
  • Type: Bug
  • Component: core-libs
  • Sub-Component: java.util.jar
  • Affected Version: 1.4.2,6
  • Priority: P2
  • Status: Closed
  • Resolution: Duplicate
  • OS: linux,solaris_2.6
  • CPU: x86,sparc
  • Submitted: 2004-11-04
  • Updated: 2010-04-03
  • Resolved: 2005-01-15
The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.
JDK 6
6Resolved
Related Reports
Duplicate :  
Description
JarFile class is not working properly on a large jar file.
  - Entries past a certain point in the file do not get found.
  - Does not list entries above a certain number

The "jar" command itself works fine on this big jar file.

1) JarTest.java

Looks for BackupHistory.xml which is the last entry in file,
 and never finds it.

javac JarTest.java
java
JarTest Main_COR_Session_FULL_8SEP2004_043614_1.jar

Output:
zip is null
Exception in thread "main" java.lang.NullPointerException
        at java.util.zip.ZipFile.getInputStream(ZipFile.java:201)
        at java.util.jar.JarFile.getInputStream(JarFile.java:359)
        at JarTest.main(JarTest.java:29)


2) ListJar.java

List contents of jar file using enumeration jar.entries()
FAILS - only lists 41633 entries, instead of 107169 entries

javac ListJar.java
java
ListJar Main_COR_Session_FULL_8SEP2004_043614_1.jar > biglist

wc -l biglist
   => 41633 biglist

jar tvf <bigjarfile>
  => 107169 entries

3) ListJar2.java

List Jar file contents using ZipInputStream and getNextEntry
WORKS - lists all entries

javac ListJar2.java
java
ListJar2  Main_COR_Session_FULL_8SEP2004_043614_1.jar > correctbiglist

 wc -l correctbiglist
  => 107170 correctbiglist (includes filename at top)


**************************************************************
import java.util.*;
import java.util.zip.*;
import java.util.jar.*;
import java.io.*;

// Look up entry in huge JarFile.  BackupHistory.xml is at end.  Doesn't find it.

public class JarTest {

 public static void main(String[] args) throws Exception {

    String jarFile = args[0];

    JarFile jar = new JarFile(jarFile);

    // not work - can't find this entry even though it's there in the huge Jar file.
    String jarEntry = "BackupHistory.xml";

    ZipEntry zip = jar.getEntry(jarEntry);
    if (zip == null)
        System.out.println ("zip is null");

    InputStream in = jar.getInputStream(zip);

    System.out.println ("getInputStream returns " + in);
  }
}
**************************************************************
import java.util.*;
import java.util.zip.*;
import java.util.jar.*;
import java.io.*;

// List contents of JarFile using jar.entries() enumeration
// FAILS - only lists 41633 entries, instead of 107169

public class ListJar {
  public static void main(String[] args) throws Exception {
    String jarFile = args[0];

    JarFile jar = new JarFile(jarFile);

    Enumeration enum = null;

    enum = jar.entries();

    ZipEntry entry;

    while(enum.hasMoreElements()){

        entry = (ZipEntry)enum.nextElement();

        String entryName = entry.getName();

        System.out.println("  " + entryName);

    }
  }
}
**********************************************************
import java.util.*;
import java.util.zip.*;
import java.util.jar.*;
import java.io.*;

// List Jar file contents using ZipInputStream and getNextEntry
// WORKS - lists all entries

public class ListJar2 {
  public static void main(String[] args) throws Exception {

        String jarFile = args[0];
        System.out.println("Jar File = '" + jarFile + "'");

        String jarEntry = "BackupHistory.xml";

        FileInputStream in = new FileInputStream(jarFile);
        if (in == null) {System.out.println("FileInputStream 'in' is null");};

        ZipInputStream zip = new ZipInputStream(in);
        if (zip == null) {System.out.println("ZipInputStream 'zip' is null");};

        ZipEntry entry;
        while ((entry = zip.getNextEntry()) != null) {
            String entryName = entry.getName();
            System.out.println("  " + entryName);
        }
    }
}
**********************************************************

Please note that the Main_COR_Session_FULL_8SEP2004_043614_1.jar is jar file of 806510725 bytes. It is located at:

/java/jle_build/sko/hp/jarfile_bug/Main_COR_Session_FULL_8SEP2004_043614_1.jar

###@###.### 11/4/04 01:44 GMT
###@###.### 11/4/04 01:54 GMT
###@###.### 11/4/04 01:54 GMT

Comments
EVALUATION The jar file contains 107169 entries, but thinks it has only 41633. The key observation is that 107169 - 41633 = 65536 There is a 2-byte field in the zip file (ENDTOT) containing the number of entries. This field cannot hold the number of entries in this case. A zip implementation does not need to use this field, since it can simply count the entries in the "central directory". The central directory must be read anyways as part of initializing a JarFile, so computing the count independently is not a hardship. In fact, the count is already being computed in the code, but limited by ENDTOT. The fix is to ignore ENDTOT, or use it as simply a hint. Strictly speaking, zip files with more than 64k entries are not valid zip files, but zip implementations can support such zip files without too great a hardship. The Info-Zip implementation is one implementation that does this. It seems that the de-facto zip file format does not in fact have the limit on the number of entries, and Java should be fixed up to match. ###@###.### 2005-1-15 20:11:52 GMT
15-01-2005

SUGGESTED FIX Treat the ENDTOT field in the end header as a hint, or simply ignore it. When reading the central directory, keep track of the count. If necessary, grow dynamic data structures such as the comments array. These changes should be localizable to zip_util.[ch] ###@###.### 2005-1-15 20:11:52 GMT
15-01-2005