JDK-6806875 : Young memory "leak" in LinkedBlockingQueue
  • Type: Bug
  • Component: core-libs
  • Sub-Component: java.util.concurrent
  • Affected Version: 6u11
  • Priority: P4
  • Status: Closed
  • Resolution: Duplicate
  • OS: windows_vista
  • CPU: x86
  • Submitted: 2009-02-18
  • Updated: 2011-02-16
  • Resolved: 2009-02-18
Related Reports
Duplicate :  
Description
FULL PRODUCT VERSION :
java version "1.6.0_11"
Java(TM) SE Runtime Environment (build 1.6.0_11-b03)
Java HotSpot(TM) 64-Bit Server VM (build 11.0-b16, mixed mode)

ADDITIONAL OS VERSION INFORMATION :
Microsoft Windows [version 6.0.6001]

A DESCRIPTION OF THE PROBLEM :
The method extract() doesn't "help" the GC by dereferencing head.next prior to changing the head node (with head.next = null;).

Consequence is quite damaging as any LinkedBlockingQueue that survives a full gc gets its 'head' node in old gen memory.
As the link is never broken, any node created hereafter will end up in old gen (being referenced by an old gen object), consuming both minor gc time AND old gen memory (16 bytes per node for 32 bits, 32 bytes for 64 bits JVM).

It results that any java program using LinkedBlockingQueue for long term queue will NOT be able to keep a constant memory usage through minor GC, nor will it be able to avoid full gc.
In our application (extensive use of queues), those nodes end up consuming more old gen memory than the rest of the application.

Note that LinkedList doesn't follow LinkedBlockingQueue logic, and DO null next/prev node links.
Issue that raised & validated on concurrency-interest ML:
http://concurrency.markmail.org/search/?q=#query:+page:1+mid:ew73inrwglvziurl+state:results

Also the issue is even more dramatic for G1 (tested on JVM 1.7 b44 with -XX:+UnlockExperimentalVMOptions -XX:+UseG1GC) which seem to dislike this "not coding" behavior:
Test 1 took: 5456ms
Test 2 took: 5575ms
(performs as well as JDK6 with null assignment version)

STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
Run the following test.


EXPECTED VERSUS ACTUAL BEHAVIOR :
EXPECTED -
If using a modified LinkedBlockingQueue (performing null assignment, aka chain break):

Test 1 took: 1629ms
Test 2 took: 1592ms

ACTUAL -
When using JDK LinkedBlockingQueue, I get 47 full GC !

Test 1 took: 1595ms
Test 2 took: 2639ms
Exception in thread "main" java.lang.AssertionError: Full GC has
occured during the test: 47 times
       at TestLBQ.main(TestLBQ.java:72)

REPRODUCIBILITY :
This bug can be reproduced always.

---------- BEGIN SOURCE ----------
import java.lang.management.ManagementFactory;
import java.util.concurrent.LinkedBlockingQueue;

import com.sun.management.GarbageCollectorMXBean;

public class TestLBQ
{
    static GarbageCollectorMXBean fullgcx;
    static
    {
        for (GarbageCollectorMXBean gcx : ManagementFactory.getGarbageCollectorMXBeans().toArray(new GarbageCollectorMXBean[2]))
        {
            if (gcx == null) continue; // G1 MXBeans aren't defined yet it seems
            if ("PS MarkSweep".equals(gcx.getName()) || "MarkSweepCompact".equals(gcx.getName()))
            {
                fullgcx = gcx;
            }
            else if ("ConcurrentMarkSweep".equals(gcx.getName()))
            {
                throw new AssertionError("Test to be executed with PS MarkSweep (standard old collector)");
            }
        }
    }

    public static void main(String[] args) throws InterruptedException
    {
        long fullgc_count_ref=0, fullgc_count=0, start, end;

        // first test, create the queue after GC, head node will be in young gen
        System.gc();
        LinkedBlockingQueue q = new LinkedBlockingQueue();
        if (fullgcx != null) fullgc_count_ref = fullgcx.getCollectionCount();
        Object DUMMY = new Object();
        start = System.currentTimeMillis();
        for (int i = 0; i < 10000000; i++)
        {
            q.offer(DUMMY);
            q.take();
        }
        System.out.println("Test 1 took: " + (System.currentTimeMillis()-start) + "ms");
        if (fullgcx != null) fullgc_count = fullgcx.getCollectionCount();
        if (fullgc_count > fullgc_count_ref)
            throw new AssertionError("Full GC has occured during the test: " + (fullgc_count-fullgc_count_ref) + " times"); // not thrown

        // SAME test, but create the queue before GC, head node will be in old gen
        q = new LinkedBlockingQueue();
        System.gc();
        if (fullgcx != null) fullgc_count_ref = fullgcx.getCollectionCount();
        start = System.currentTimeMillis();
        for (int i = 0; i < 10000000; i++)
        {
            q.offer(DUMMY);
            q.take();
        }
        System.out.println("Test 2 took: " + (System.currentTimeMillis()-start) + "ms");
        if (fullgcx != null) fullgc_count = fullgcx.getCollectionCount();
        if (fullgc_count > fullgc_count_ref)
            throw new AssertionError("Full GC has occured during the test: " + (fullgc_count-fullgc_count_ref) + " times"); // ERROR
    }
}


---------- END SOURCE ----------

CUSTOMER SUBMITTED WORKAROUND :
Do not use LinkedBlockingQueue from JDK in any long term queue.
Replace by custom modified queue that null links.