JDK-6210541 : (bf) sun.nio.ch.Util.java: releaseTemporaryDirectBuffer() SoftReference GC
  • Type: Bug
  • Component: core-libs
  • Sub-Component: java.nio
  • Affected Version: 1.4.2_05,6
  • Priority: P4
  • Status: Closed
  • Resolution: Won't Fix
  • OS: linux,solaris_8
  • CPU: x86,sparc
  • Submitted: 2004-12-20
  • Updated: 2024-04-12
  • Resolved: 2017-07-20
Related Reports
Duplicate :  
Duplicate :  
Duplicate :  
Description
FULL PRODUCT VERSION :


ADDITIONAL OS VERSION INFORMATION :
Solaris 8

A DESCRIPTION OF THE PROBLEM :
In JDK 1.4.2_05 the releaseTemporaryDirectBuffer() was modified to avoid replacement of smaller DirectByteBuffers but that was re-added in JDK 1.4.2_06 and is present for the JDK 1.5.0 and JDK 1.6.0 source trees.

While I understand the advantage, I don't believe it fully takes care of potential memory issues off the C-Heap as the SoftReference is to my understanding within the scope of the Java Heap garbage collection entirely independent of the C-Heap. Thus, the very small SoftReference will in general hold a larger C-Heap reference to physical memory and while the java.nio.Bits.java reserveMemory() request will attempt a System.gc() on low C-Heap space, it is highly probable that the JVM GC will not be low on Java Heap memory and thus not release the SoftReferences in the Threads holding the DirectByteBuffer allocations.

With the present configuration, once a large buffer has been allocated in a thread it is unlikely to ever disappear if the Java Heap is appropriately sized and there are a number of threads holding large buffers as the SoftRerence doesn't need to be cleaned up necessarily.


ERROR MESSAGES/STACK TRACES THAT OCCUR :
OutOfMemoryError

REPRODUCIBILITY :
This bug can be reproduced always.

CUSTOMER SUBMITTED WORKAROUND :
Managing the way the application handles threads and direct byte buffers this can be worked around.
###@###.### 2004-12-20 13:19:41 GMT

Comments
EVALUATION Just one update on this issue. In jdk7, the buffer cache implementation is an array of buffers. Soft references are not used. As previously stated, the only remaining issue is short lived threads that do I/O with buffers backed by arrays in the heap. We don't have a solution for this case at this time.
31-08-2010

EVALUATION There are a number of improvements to this code already in jdk7. In particular, the scatter/gather operations use the buffer cache and excess buffers are released rather than being unreferenced. The remaining issue is short lived threads that do I/O with heap buffers. We don't currently have a way to free the buffers in the buffer cache in a timely manner when the thread exits.
05-08-2009

EVALUATION The gathering write and scattering read also needs to be re-examined - these copy the contents of slow buffers into newly allocated direct buffers. When the I/O completes these direct buffers are unreferenced and will consume native heap until reference processing happens. The scatter/gather cases could re-use the thread-local buffer cache and excessive buffers could be released after the I/O completes.
07-11-2005

EVALUATION I've closed a number of bugs as duplicates of this bug. This means that this bug is now a more general bug to re-examine the temporary direct buffer cache. The current implementation is fine for most applications but there will be cases and configurations where it is not ideal. These include cases where there are a lot threads that do some I/O with a non-direct buffer, cases where there are short-lived threads doing I/O, and cases where entries in the cache are replaced with ever larger buffers (delayed referencing processing can mean that the memory for the replaced buffers isn't freed in a timely manner).
14-10-2005