JDK-7161229 : PriorityBlockingQueue keeps hard reference to last removed element
  • Type: Bug
  • Component: core-libs
  • Sub-Component: java.util.concurrent
  • Affected Version: 7
  • Priority: P2
  • Status: Closed
  • Resolution: Fixed
  • OS: windows_7
  • CPU: x86
  • Submitted: 2012-04-13
  • Updated: 2013-06-26
  • Resolved: 2012-07-24
The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.
JDK 7 JDK 8
7u40Fixed 8 b47Fixed
Related Reports
Duplicate :  
Relates :  
Description
FULL PRODUCT VERSION :
java version "1.7.0_02"
Java(TM) SE Runtime Environment (build 1.7.0_02-b13)
Java HotSpot(TM) 64-Bit Server VM (build 22.0-b10, mixed mode)

ADDITIONAL OS VERSION INFORMATION :
Microsoft Windows [Version 6.1.7601]

EXTRA RELEVANT SYSTEM CONFIGURATION :
Dell E6420, 8GB RAM, 256GB SSD

A DESCRIPTION OF THE PROBLEM :
  PriorityBlockingQueue appears to keep a hard reference to the final queue element when it's removed via take() or poll() . The element will not be garbage collected until/unless another element is put().

This seems to be new in Java 7. Looking at the source, extract() dutifully nulls out the removed element from the internal queue field, and then calls siftDownComparable() or siftDownUsingComparator() which sets it back into the queue.

REGRESSION.  Last worked in version 6u31

STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
- Construct a PriorityBlockingQueue
- Put an element
- Take that element, leaving an empty queue
- Now grab a heap dump and analyze it. Or inspect the internal queue field of the PriorityBlockingQueue via reflection or a debugger.

EXPECTED VERSUS ACTUAL BEHAVIOR :
EXPECTED -
I was expecting PriorityBlockingQueue to release the object once it was removed.
ACTUAL -
Both the heap dump and live inspection show that the internal queue field has a hard reference to the object that was removed. In our case, the object is a document that may be hundreds of megabytes in size and we have no safe way to null out this reference. This is causing OutOfMemoryErrors in our application.

REPRODUCIBILITY :
This bug can be reproduced always.

---------- BEGIN SOURCE ----------
import java.io.IOException;
import java.lang.reflect.Field;
import java.util.Arrays;
import java.util.concurrent.PriorityBlockingQueue;

// Trivial example that demonstrates PriorityBlockingQueue holding onto removed objects
public class PriorityBlockingQueueTest
{
    public static void main(String[] args) throws IOException, InterruptedException
    {
        PriorityBlockingQueue<Integer> testQueue = new PriorityBlockingQueue<>();
        testQueue.put(new Integer(123));
        testQueue.take();

        if (!testQueue.isEmpty())
            throw new IllegalStateException("Queue should be empty!");

        try {
            Field privateField = PriorityBlockingQueue.class.getDeclaredField("queue");
            privateField.setAccessible(true);
            Object[] queue = (Object[]) privateField.get(testQueue);
            System.out.println("queue = " + Arrays.toString(queue));
        } catch (NoSuchFieldException e) {
            // Ignore
        } catch (IllegalAccessException e) {
            // Ignore
        }
    }
}
---------- END SOURCE ----------

CUSTOMER SUBMITTED WORKAROUND :
Any workaround would be very convoluted... we'd need to obtain the internal lock via reflection, null out any element(s) past the current size, and release the lock.

Comments
EVALUATION http://hg.openjdk.java.net/hsx/hotspot-comp/jdk/rev/94b525ce3653
2012-08-14

EVALUATION Changeset: 94b525ce3653 Author: dholmes Date: 2012-06-27 01:36 -0400 URL: http://hg.openjdk.java.net/jdk8/tl/jdk/rev/94b525ce3653 7161229: PriorityBlockingQueue keeps hard reference to last removed element Reviewed-by: dholmes, forax, alanb Contributed-by: Doug Lea <###@###.###> ! src/share/classes/java/util/concurrent/PriorityBlockingQueue.java ! test/java/util/concurrent/BlockingQueue/LastElement.java
2012-07-02

EVALUATION As per the description after a delete operation the array contents are sifted-down. This is done by saving the last entry, nulling it, the calling the sift operation. The final part of which is store the saved last entry into the last position. When removing the only entry, the saved entry is the last entry and the size and index are both zero leading to a degenerative case in the sift operation where all that gets executed is the final store of the saved result - hence we write the removed element back into the array. Possible simple fix is to only siftdown when the new size of the queue is > 0.
2012-06-09