JDK-4795136 : CRC check fails for files over 2 GB
  • Type: Bug
  • Component: core-libs
  • Sub-Component: java.util.jar
  • Affected Version: 1.4.0,1.4.1
  • Priority: P4
  • Status: Resolved
  • Resolution: Fixed
  • OS: solaris_7,windows_2000
  • CPU: x86,sparc
  • Submitted: 2002-12-17
  • Updated: 2003-09-12
  • Resolved: 2003-09-12
The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.
Other
5.0 tigerFixed
Related Reports
Relates :  
Relates :  
Relates :  
Description

Name: nt126004			Date: 12/17/2002


FULL PRODUCT VERSION :
java version "1.4.0"
Java(TM) 2 Runtime Environment, Standard Edition (build 1.4.0-b92)
Java HotSpot(TM) Client VM (build 1.4.0-b92, mixed mode)

FULL OPERATING SYSTEM VERSION :
Microsoft Windows 2000 [Version 5.00.2195]


A DESCRIPTION OF THE PROBLEM :
CRC check in always failed when the file is bigger than 2GB.
The code and error show below.




STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
1.create a file bigger than 2GB
2.change the code read file by your created file
3.

EXPECTED VERSUS ACTUAL BEHAVIOR :
it can pass the CRC check and no error throw

ERROR MESSAGES/STACK TRACES THAT OCCUR :
entry:TEST_UF024.RAW/_extern.inf
Before while:801
Finish while loop.
entry:TEST_UF024.RAW/_FUNC001.DAT
Before while:688
IOError:invalid entry size (expected 2147491650 but got -2147475646 bytes)
java.util.zip.ZipException: invalid entry size (expected 2147491650 but got -2147475646 bytes)
        at java.util.zip.ZipInputStream.readEnd(Unknown Source)
        at java.util.zip.ZipInputStream.read(Unknown Source)
        at java.io.FilterInputStream.read(Unknown Source)
        at CRCTest.checkCRCFile(CRCTest.java:31)
        at CRCTest.main(CRCTest.java:56)


REPRODUCIBILITY :
This bug can be reproduced always.

---------- BEGIN SOURCE ----------
import java.util.zip.*;
import java.io.*;

public class CRCTest{

public CRCTest(){}


public void checkCRCFile(String inStrFileAbsolutePath) throws IOException, DataFormatException
{

    final int BUFFER = 1024*1024;
    byte[] buffData = new byte[BUFFER];
    FileInputStream fis = new FileInputStream(inStrFileAbsolutePath);
    BufferedInputStream bis = new BufferedInputStream(fis,BUFFER);
    ZipInputStream zis = new ZipInputStream(bis);


    for(ZipEntry nextZipEntry = zis.getNextEntry(); nextZipEntry != null; nextZipEntry =
zis.getNextEntry())
    {
          System.out.println("entry:"+nextZipEntry);
      if (!nextZipEntry.isDirectory())
      {
        CRC32 crcGenerated = new CRC32();

        int count = zis.read(buffData);

        System.out.println("Before while:"+count);
        while (count != -1) {
            crcGenerated.update(buffData, 0, count);
            count = zis.read(buffData);
        }

        System.out.println("Finish while loop.");

        long lCRC32FromZip = nextZipEntry.getCrc();

        if (lCRC32FromZip == -1)
          throw new DataFormatException("CRC Not present in File " + nextZipEntry.getName());

        if (lCRC32FromZip != crcGenerated.getValue())
          throw new DataFormatException("CRC Error " + nextZipEntry.getName() + " ZipCRC=" +
lCRC32FromZip + " CalculatedCRC=" + crcGenerated.getValue());
      }
    }

    zis.close();
    bis.close();
    fis.close();

  }

   public static void main(String args[]) {
        CRCTest crc = new CRCTest();

        try {
                crc.checkCRCFile("c:\\temp\\TEST_UF024.RAW.ZIP");
                System.out.println("CRC Check success.");
        }catch(IOException ioe){
                System.out.println("IOError:");
				ioe.printStackTrace();
        }catch(DataFormatException fe){
                System.out.println("DataFormatError:");
				fe.printStackTrace();
        }catch(Exception e){
                System.out.println("OtherError:");
				e.printStackTrace();
        }

  }
}
---------- END SOURCE ----------
(Review ID: 178849) 
======================================================================

Comments
CONVERTED DATA BugTraq+ Release Management Values COMMIT TO FIX: tiger FIXED IN: tiger INTEGRATED IN: tiger tiger-b20
14-06-2004

PUBLIC COMMENTS Add support for zip files and jar files between 2GB and 4GB in size.
10-06-2004

EVALUATION Submitter is correct. This limitation will be addressed as resources allow. ###@###.### 2002-12-17 File sizes in zip files are generally stored as unsigned 32-bit quantities. which do not fit into a java `int'. A `long' should have been used instead. For example, public synchronized int getTotalOut() should have been public synchronized long getTotalOut() However, it is too late to change the API, especially for such a "small" fix as to increase the size from 2GB to 4GB. Long term, we want to support file sizes larger than 4GB, and this will require major changes to the Zip File format. PKZIP implements such an extension, and we should investigate their file format. However, we _can_ get support for sizes up to 4GB. Although the return type of getTotalOut and getTotalIn are semantically incorrect, there is no data loss. We simply have to reinterpret the value as unsigned. So it looks like we can fix this without making any API changes. Most of the code in the zip package already uses "long" to represent unsigned 32-bit values, so few changes are necessary. One very good reason to fix this is that currently a user can use the "jar" command to create a jar file that cannot be read by "jar". ###@###.### 2003-04-29
29-04-2003