JDK-6519463 : Unexpected end of ZLIB when using GZIP on some files
  • Type: Bug
  • Component: core-libs
  • Sub-Component: java.util.jar
  • Affected Version: 6
  • Priority: P4
  • Status: Closed
  • Resolution: Cannot Reproduce
  • OS: windows_xp
  • CPU: x86
  • Submitted: 2007-01-30
  • Updated: 2010-05-15
  • Resolved: 2010-05-15
Related Reports
Relates :  
Description
FULL PRODUCT VERSION :
java version "1.6.0"
Java(TM) SE Runtime Environment (build 1.6.0-b105)
Java HotSpot(TM) Client VM (build 1.6.0-b105, mixed mode, sharing)

ADDITIONAL OS VERSION INFORMATION :
Microsoft Windows XP [Version 5.1.2600]

A DESCRIPTION OF THE PROBLEM :
This bug is similar to 4040920, expect it applies to when you use the GZIP streams.

When using gzip (via new Deflater(Deflater.BEST_COMPRESSION, true)), for some files, and EOFException is thrown at the end of inflating.  Although the file is correct, the bug is the EOFException is thrown inconsistently.  For some files it is thrown, other it is not.

  To show the bug, use the following file:
ftp://66.199.149.152/buggedfile.bin

The program does not crash for most other files (for example try java -cp . BuggedEof BuggedEof.java)

STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
Using the provide source code and file enter the commands:

javac BuggedEof.java
java -cp . BuggedEof buggedfile.bin

EXPECTED VERSUS ACTUAL BEHAVIOR :
EXPECTED -
The program is expected to produce buggedfile.bin.z and buggedfile.bin.uncompressed with not exceptions thrown.

buggedfile.bin.uncompressed is expected to be the same as buggedfile.bin.
ACTUAL -
Exception in thread "main" java.io.EOFException: Unexpected end of ZLIB input st
ream
        at java.util.zip.InflaterInputStream.fill(Unknown Source)
        at java.util.zip.InflaterInputStream.read(Unknown Source)
        at java.io.FilterInputStream.read(Unknown Source)
        at BuggedEof.main(BuggedEof.java:62)

REPRODUCIBILITY :
This bug can be reproduced always.

---------- BEGIN SOURCE ----------
// for BuggedEof.java
import java.io.BufferedInputStream;
import java.io.BufferedOutputStream;
import java.io.EOFException;
import java.io.File;
import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.io.IOException;
import java.util.zip.Deflater;
import java.util.zip.DeflaterOutputStream;
import java.util.zip.Inflater;
import java.util.zip.InflaterInputStream;


public class BuggedEof {

    public static void main(String[] args) throws IOException {

        final byte[] buffer = new byte[4096];

        // compress to .z
        File file = new File(args[0]);
        File compressed = new File(file.getAbsolutePath() + ".z");
        compressed.delete();

        FileInputStream fis = new FileInputStream(file);
        BufferedInputStream bis = new BufferedInputStream(fis);

        FileOutputStream fos = new FileOutputStream(compressed);
        BufferedOutputStream bos = new BufferedOutputStream(fos);
        DeflaterOutputStream dos = new DeflaterOutputStream(bos, new Deflater(Deflater.BEST_COMPRESSION, true));

        int read;

        try {
			while((read = bis.read(buffer)) > 0) {
				dos.write(buffer, 0, read);
			}
			dos.finish();
			dos.flush();
		} finally {
			dos.close();
			bos.close();
			fos.close();

			bis.close();
			fis.close();
		}


        // uncompress .z to .uncompressed
        File uncompressed = new File(file.getAbsolutePath() + ".uncompressed");
        uncompressed.delete();

        fis = new FileInputStream(compressed);
        bis = new BufferedInputStream(fis);
        InflaterInputStream iis = new InflaterInputStream(bis, new Inflater(true));

        fos = new FileOutputStream(uncompressed);
        bos = new BufferedOutputStream(fos);

        try {
			while((read = iis.read(buffer)) > 0) {
				bos.write(buffer, 0, read);
			}
		} catch(EOFException eofe) {
			throw eofe;  // comment out line to implement work-around
		} finally {
			bos.close();
			fos.close();

			iis.close();
			bis.close();
			fis.close();
		}
    }
}

---------- END SOURCE ----------

CUSTOMER SUBMITTED WORKAROUND :
Just catch the EOFException and do nothing.  The file uncompresses correctly.

Comment out line 66 to implement the workaround

Comments
EVALUATION The ZipInputStream from the ZipFile overrides the fill() method with comment "to provide an extra dummy byte at the endo fo the input stream. This is required when using the nowrap Inflater option". This workaround obviously should be done in InflaterInputStream for all "nowrap" Inflater. The good news is that it appears that the new zlib1.23 does not have this request for "nowrap" Inflater. So the problem is not reproducible after jdk7-b72, in which we upgraded the underlying zlib to 1.23. So I closed this bug as "not reproducible". Anyone needs the solution for 6u and earlier can reopen this one for the previous releases.
15-05-2010

EVALUATION The root cause is the same as described in 4040920. However, the fix for that bug was done in ZipFile.java. It might have been better done in InflaterInputStream.java.
31-01-2007