United StatesChange Country, Oracle Worldwide Web Sites Communities I am a... I want to...
JDK-7161229 : PriorityBlockingQueue keeps hard reference to last removed element

Details
Type:
Bug
Submit Date:
2012-04-13
Status:
Closed
Updated Date:
2013-04-11
Project Name:
JDK
Resolved Date:
2012-07-24
Component:
core-libs
OS:
windows_7
Sub-Component:
java.util.concurrent
CPU:
x86
Priority:
P2
Resolution:
Fixed
Affected Versions:
7
Fixed Versions:

Related Reports
Backport:
Backport:
Duplicate:
Relates:

Sub Tasks

Description
FULL PRODUCT VERSION :
java version "1.7.0_02"
Java(TM) SE Runtime Environment (build 1.7.0_02-b13)
Java HotSpot(TM) 64-Bit Server VM (build 22.0-b10, mixed mode)

ADDITIONAL OS VERSION INFORMATION :
Microsoft Windows [Version 6.1.7601]

EXTRA RELEVANT SYSTEM CONFIGURATION :
Dell E6420, 8GB RAM, 256GB SSD

A DESCRIPTION OF THE PROBLEM :
  PriorityBlockingQueue appears to keep a hard reference to the final queue element when it's removed via take() or poll() . The element will not be garbage collected until/unless another element is put().

This seems to be new in Java 7. Looking at the source, extract() dutifully nulls out the removed element from the internal queue field, and then calls siftDownComparable() or siftDownUsingComparator() which sets it back into the queue.

REGRESSION.  Last worked in version 6u31

STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
- Construct a PriorityBlockingQueue
- Put an element
- Take that element, leaving an empty queue
- Now grab a heap dump and analyze it. Or inspect the internal queue field of the PriorityBlockingQueue via reflection or a debugger.

EXPECTED VERSUS ACTUAL BEHAVIOR :
EXPECTED -
I was expecting PriorityBlockingQueue to release the object once it was removed.
ACTUAL -
Both the heap dump and live inspection show that the internal queue field has a hard reference to the object that was removed. In our case, the object is a document that may be hundreds of megabytes in size and we have no safe way to null out this reference. This is causing OutOfMemoryErrors in our application.

REPRODUCIBILITY :
This bug can be reproduced always.

---------- BEGIN SOURCE ----------
import java.io.IOException;
import java.lang.reflect.Field;
import java.util.Arrays;
import java.util.concurrent.PriorityBlockingQueue;

// Trivial example that demonstrates PriorityBlockingQueue holding onto removed objects
public class PriorityBlockingQueueTest
{
    public static void main(String[] args) throws IOException, InterruptedException
    {
        PriorityBlockingQueue<Integer> testQueue = new PriorityBlockingQueue<>();
        testQueue.put(new Integer(123));
        testQueue.take();

        if (!testQueue.isEmpty())
            throw new IllegalStateException("Queue should be empty!");

        try {
            Field privateField = PriorityBlockingQueue.class.getDeclaredField("queue");
            privateField.setAccessible(true);
            Object[] queue = (Object[]) privateField.get(testQueue);
            System.out.println("queue = " + Arrays.toString(queue));
        } catch (NoSuchFieldException e) {
            // Ignore
        } catch (IllegalAccessException e) {
            // Ignore
        }
    }
}
---------- END SOURCE ----------

CUSTOMER SUBMITTED WORKAROUND :
Any workaround would be very convoluted... we'd need to obtain the internal lock via reflection, null out any element(s) past the current size, and release the lock.

                                    

Comments
EVALUATION

http://hg.openjdk.java.net/hsx/hotspot-comp/jdk/rev/94b525ce3653
                                     
2012-08-14
EVALUATION

Changeset: 94b525ce3653
Author:    dholmes
Date:      2012-06-27 01:36 -0400
URL:       http://hg.openjdk.java.net/jdk8/tl/jdk/rev/94b525ce3653

7161229: PriorityBlockingQueue keeps hard reference to last removed element
Reviewed-by: dholmes, forax, alanb
Contributed-by: Doug Lea <###@###.###>

! src/share/classes/java/util/concurrent/PriorityBlockingQueue.java
! test/java/util/concurrent/BlockingQueue/LastElement.java
                                     
2012-07-02
EVALUATION

As per the description after a delete operation the array contents are sifted-down. This is done by saving the last entry, nulling it, the calling the sift operation. The final part of which is store the saved last entry into the last position.

When removing the only entry, the saved entry is the last entry and the size and index are both zero leading to a degenerative case in the sift operation where all that gets executed is the final store of the saved result - hence we write the removed element back into the array.

Possible simple fix is to only siftdown when the new size of the queue is > 0.
                                     
2012-06-09



Hardware and Software, Engineered to Work Together