United StatesChange Country, Oracle Worldwide Web Sites Communities I am a... I want to...
JDK-6728376 : Wrong error handling in Java_java_util_zip_Deflater_deflateBytes leads to size 0 if compress fails

Details
Type:
Bug
Submit Date:
2008-07-22
Status:
Resolved
Updated Date:
2011-07-27
Project Name:
JDK
Resolved Date:
2009-06-22
Component:
core-libs
OS:
solaris_10
Sub-Component:
java.util.jar
CPU:
generic
Priority:
P3
Resolution:
Fixed
Affected Versions:
6u7
Fixed Versions:

Related Reports
Backport:
Backport:

Sub Tasks

Description
Here is a small testcase that uses java.util.zip.Deflater


//****************************************************
import java.util.Random;
import java.util.zip.Deflater; 

public class TestDeflater {

      public static void main(String[] args) {

            int max_byte_length = 200000000;
            byte[] inputData = new byte[max_byte_length];
            Random random = new Random();
            random.nextBytes(inputData);

            System.out.println("dataSize:" + inputData.length);

            Deflater compressor = new Deflater(Deflater.BEST_COMPRESSION);
            compressor.setInput(inputData);
            compressor.finish();

      
            byte[] tempBuf = new byte[inputData.length];
            int compressedLength = compressor.deflate(tempBuf);

            System.out.println("compressedSize:" + compressedLength);
            System.out.println("finishedFlag:"+compressor.finished());

            compressor.end();
      }
}
//****************************************************


This is how it works normally:

$ java TestDeflater     
dataSize:200000000
compressedSize:200000000
finishedFlag:false


However, if either virtual memory (swap space) is scarce or the Java
process is about to hit the (4GB on Solaris32 / 2GB on Windows/Linux) 
process size limit, then we will see this:

$ ulimit -d 100000  //simulating low memory condition
$ java TestDeflater
dataSize:200000000
compressedSize:0    <<<<< 
finishedFlag:false

The expected behaviour should be to throw an OOME instead.

                                    

Comments
SUGGESTED FIX

In Inflater.c and Deflater.c, change this code pattern:
            in_buf = (jbyte *) malloc(this_len);
            if (in_buf == 0) {
                return 0;
            }

to:
            in_buf = (jbyte *) malloc(this_len);
            if (in_buf == 0) {
                JNU_ThrowOutOfMemoryError(env, 0);
            }
Note that in some cases, previously-allocated memory should be freed before the OOME is thrown.
                                     
2008-07-22
EVALUATION

There are a few places in both Deflater.c and Inflater.c which have the problem noted in the comments.  There are also places in those files from which an OOME is thrown via     JNU_ThrowOutOfMemoryError(env, 0);
                                     
2008-07-22



Hardware and Software, Engineered to Work Together