JDK-7022417 : JDK6u23,6u24 GZIPInputStream readHeader broke handling of GZIP 'extra fields'
  • Type: Bug
  • Component: core-libs
  • Sub-Component: java.util.jar
  • Affected Version: 6u24
  • Priority: P2
  • Status: Closed
  • Resolution: Duplicate
  • OS: windows_xp
  • CPU: x86
  • Submitted: 2011-02-25
  • Updated: 2012-03-20
  • Resolved: 2011-02-25
Related Reports
Duplicate :  
Description
FULL PRODUCT VERSION :
java version "1.6.0_24"
Java(TM) SE Runtime Environment (build 1.6.0_24-b07)
Java HotSpot(TM) Client VM (build 19.1-b02, mixed mode, sharing)


A DESCRIPTION OF THE PROBLEM :
Changes for 6u23 (probably for bug 4691425) broke GZIPInputStream.readHeader so it no longer properly handles the optional 'extra fields' GZIP data may include.

Specifically, code that used to read:

	// Skip optional extra field
	if ((flg & FEXTRA) == FEXTRA) {
	    skipBytes(in, readUShort(in));
	}

...now reads...

        // Skip optional extra field
        if ((flg & FEXTRA) == FEXTRA) {
            skipBytes(in, readUShort(in));
            int m = readUShort(in);
            skipBytes(in, m);
            n += m + 2;
        }

It looks to me like a sloppy edit that meant to completely replace the first skipBytes(), but didn't. As a result, too many bytes are skipped (and the 'n' this method is trying to report can't be right). Depending on the subsequent data misinterpreted as a length, and subsequent overskip, several things could go wrong, including an error "java.util.zip.ZipException: invalid stored block lengths", an unexpected EOFException, and possibly other corrupted output/checksum warnings.

I believe removing the leftover, untallied skipBytes() line will fix.

STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
Try to use JDK6u23/JDK6u24 GZIPInputStream on GZIP data including any 'extra fields'. (See attached code for small example.)

EXPECTED VERSUS ACTUAL BEHAVIOR :
EXPECTED -
Expected result is receiving the uncompressed data, as worked (at least for the first member encountered) pre-JDK6u23.
ACTUAL -
An exception is thrown. Some legal inputs might also return corrupted data, if the skip happened to land at an offset where the compressed data looked legal for a while.

ERROR MESSAGES/STACK TRACES THAT OCCUR :
The test below will generate:

Exception in thread "main" java.util.zip.ZipException: invalid stored block lengths
	at java.util.zip.InflaterInputStream.read(InflaterInputStream.java:147)
	at java.util.zip.GZIPInputStream.read(GZIPInputStream.java:92)
	at java.util.zip.InflaterInputStream.read(InflaterInputStream.java:105)
	at GZIPTest.main(GZIPTest.java:49)

As first encountered, I received an unexpected EOFException (because my data at the time caused the bug to skip past end of compressed data).

It's probably possible to receive corrupt uncompressed data for a while, then receive a GZIP checksum error or other indication of corruption after a while.

REPRODUCIBILITY :
This bug can be reproduced always.

---------- BEGIN SOURCE ----------
import java.io.ByteArrayInputStream;
import java.io.IOException;
import java.util.zip.GZIPInputStream;

/**
 * Demonstrate JDK6u23-JDK6u24 GZIPInputStream FEXTRA bug.
 *
 * Pre-JDK6u23, will print zero length of both GZIP inputs. JDK6u23-24,
 * will work on input without extra-field but throw exception for input with
 * extra-field.
 */
public class GZIPTest {
    // Identical to what GZIPOutputStream creates with zero-length input
    static byte[] empty_gz = {
              0x1f, (byte)0x8b,       // ID1, ID2: GZIP MAGIC
              0x08,                   // CM: DEFLATE
              0x00,                   // FLG: NO FLAGS
              0x00, 0x00, 0x00, 0x00, // MTIME
              0x00,                   // XFL
              0x00,                   // OS
              0x03, 0x00,             // compressed block
              0x00, 0x00, 0x00, 0x00, // CRC32
              0x00, 0x00, 0x00, 0x00, // ISIZE
    };
    
    // Above with added, legal (RFC1952) "extra field"
    static byte[] empty_extra_gz = {
            0x1f, (byte)0x8b,       // ID1, ID2: GZIP MAGIC
            0x08,                   // CM: DEFLATE
            0x04,                   // FLG: FEXTRA (0x04) set
            0x00, 0x00, 0x00, 0x00, // MTIME
            0x00,                   // XFL
            0x00,                   // OS
            
            0x04, 0x00,             // XLEN: 4 bytes of "extra field"
            0x00, 0x00, 0x00, 0x00, // SI1, SI2, LEN: irrelevant
            
            0x03, 0x00,             // compressed block
            0x00, 0x00, 0x00, 0x00, // CRC32
            0x00, 0x00, 0x00, 0x00, // ISIZE
  };
 
    public static void main(String [] args) throws IOException {
        GZIPInputStream in = new GZIPInputStream(new ByteArrayInputStream(empty_gz));
        long emptyCount = 0;
        while(in.read()>-1) {
            emptyCount++;
        }
        System.out.println("empty_gz:"+emptyCount);
        GZIPInputStream in2 = new GZIPInputStream(new ByteArrayInputStream(empty_extra_gz));
        long emptyExtraCount = 0;
        while(in2.read()>-1) {
            emptyExtraCount++;
        }
        System.out.println("empty_extra_gz:"+emptyExtraCount);
    }
}
---------- END SOURCE ----------

CUSTOMER SUBMITTED WORKAROUND :
Move back to pre-JDK6u23 releases. Copy the GZIPInputStream code elsewhere and remove t