JDK-8327687 : GZIPOutputStream infinite loop
  • Type: Bug
  • Component: core-libs
  • Sub-Component: java.util.jar
  • Affected Version: 8,11,17,21
  • Priority: P4
  • Status: Closed
  • Resolution: Not an Issue
  • OS: generic
  • CPU: generic
  • Submitted: 2024-03-07
  • Updated: 2024-06-10
  • Resolved: 2024-06-10
Related Reports
Relates :  
Description
A DESCRIPTION OF THE PROBLEM :
If used incorrectly, GZIPOutputStream may hang in an infinite loop instead of throwing an exception.

If there are two threads, one writes bytes to GZIPOutputStream, the other calls the finish() method at this point in time, then at certain timings, both threads can hang in an infinite loop. Provided GZIPOutputStreamTest.java has an example that hangs reliably on all tested JDK versions.

The reason is in the Deflater class, as you can see from the provided DeflaterTest.java that after calling the finish() method there is a point in time when the needsInput() method will return true, and if you actually pass additional bytes using the setInput method, then Deflater will go into the state after a certain number of calls of deflate(), when needsInput() and finished() will both return false, and there will be no more bytes available for compression. In GZIPOutputStream, the Deflater.needsInput() and Deflater.finished() methods are used in infinite loops and can hang. Probably setInput method should throw an error if it is called after a call to finish(), or needsInput() should always return false after a call to finish().

This issue was discovered when using javax.servlet.AsyncContext with incorrect synchronization in a program that asynchronously serves data via a long-lived http GET request with gzip enabled, and another thread called AsyncContext.complete() without waiting for the first thread to finish writing the response.

STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
Run provided GZIPOutputStreamTest

EXPECTED VERSUS ACTUAL BEHAVIOR :
EXPECTED -
The program ends with an error in the log
ACTUAL -
The program freezes in endless loops

---------- BEGIN SOURCE ----------
// GZIPOutputStreamTest.java
import java.io.IOException;
import java.io.OutputStream;
import java.util.Random;
import java.util.zip.GZIPOutputStream;

public class GZIPOutputStreamTest
{
    public static void main(String[] args) throws Exception
    {
        GZIPOutputStream s = new GZIPOutputStream(new OutputStream()
        {
            int i = 0;

            @Override
            public void write(int b) throws IOException
            {
                ++i;
                System.out.println("WRITE " + Thread.currentThread() + " " + i);

                if (i == 1027)
                {
                    try
                    {
                        Thread.sleep(100);
                    }
                    catch (InterruptedException e)
                    {
                        throw new RuntimeException(e);
                    }
                }
            }
        });

        Thread t1 = new Thread(() -> {
            Random rand = new Random();

            for (int i = 0; i < 10; ++i)
            {
                try
                {
                    byte[] input = new byte[4096];
                    rand.nextBytes(input);
                    s.write(input);
                }
                catch (Exception e)
                {
                    throw new RuntimeException(e);
                }
            }
        });
        t1.setName("T1");


        Thread t2 = new Thread(() -> {
            try
            {
                s.finish();
            }
            catch (IOException e)
            {
                throw new RuntimeException(e);
            }
        });
        t2.setName("T2");


        t1.start();
        Thread.sleep(50);
        t2.start();


        t1.join();
        t2.join();
    }
}


// DeflaterTest.java
import java.util.Random;
import java.util.zip.Deflater;

public class DeflaterTest
{
    public static void main(String[] args)
    {
        Deflater d = new Deflater();

        while (true)
        {
            addInput(d);
            deflate(d);

            if (!d.needsInput())
                break;
        }

        System.out.println("finish()");
        d.finish();

        while (true)
        {
            deflate(d);

            if (d.needsInput())
            {
                addInput(d);
                break;
            }
        }

        for (int i = 0; i < 100; ++i)
        {
            deflate(d);
        }
    }

    private static void addInput(Deflater d)
    {
        if (d.needsInput())
        {
            byte[] input = new byte[4096];
            Random rand = new Random();
            rand.nextBytes(input);
            d.setInput(input);
            System.out.println("setInput " + input.length + " bytes");
        }
    }

    private static void deflate(Deflater d)
    {
        byte[] output = new byte[512];
        int len = d.deflate(output);
        System.out.println("Generated " + len + " bytes, needsInput = " + d.needsInput() + ", finished = " + d.finished());
    }
}

---------- END SOURCE ----------

FREQUENCY : always



Comments
As noted GZIPOutputStream isn't specified to be thread safe and isn't expected to be used concurrently by multiple threads.
10-06-2024

This looks like user error, GZIPOutputStream is not specified to be thread safe. Also it doesn't specify behavior for closing the output stream when another thread is writing to the stream.
22-03-2024

I can't find it in the javadoc nor the implementation of java.util.zip.GZIPOutputStream - does this class allow multi-threaded concurrent access? The implementation code in GZIPOutputStream doesn't have any thread safety checks in place.
22-03-2024

Seems to be easy to reproduce. Stack trace shows hanging in both threads. CPU usage suggests at least one thread is in an endless busy loop.
08-03-2024

Observations on Windows 10 ----------------------------------------- JDK 1.8.0_391-b13 : Failed (No exception thrown, stuck in a loop) JDK 11.0.21+9-LTS-193 : Failed JDK 17.0.4+11-LTS-179 : Failed JDK 21.0.3-ea+2-LTS-132 : Failed JDK 23-ea+6 : Failed
08-03-2024