JDK-5046110 : (bf) Direct memory cannot be unreserved while reserving thread sleeps
  • Type: Bug
  • Component: core-libs
  • Sub-Component: java.nio
  • Affected Version: 1.4.2
  • Priority: P2
  • Status: Closed
  • Resolution: Fixed
  • OS: windows_xp
  • CPU: x86
  • Submitted: 2004-05-12
  • Updated: 2004-07-22
  • Resolved: 2004-05-28
The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.
Other
1.4.2_06 06Fixed
Related Reports
Relates :  
Description

Name: jl125535			Date: 05/12/2004


FULL PRODUCT VERSION :
J2SE v1.5beta, 1.4.2_04

ADDITIONAL OS VERSION INFORMATION :
Windows, Linux, Solaris

A DESCRIPTION OF THE PROBLEM :
I think the implemetation of java.nio.Bits#reserveMemory is not good.
Here is a part of java.nio.Bits.java source code for v1.5beta.

 1. static synchronized void reserveMemory(long size) {
     ...
 6.   if (reservedMemory + size > maxMemory) {
 7.     System.gc();
 8.     try {
 9.       Thread.sleep(100);
10.     } catch (InterruptedException x) {
11.       // Restore interrupt status
12.       Thread.currentThread().interrupt();
13.     }
14.     if (reservedMemory + size > maxMemory)
15.       throw new OutOfMemoryError("Direct buffer memory");
16.     }
17.   reservedMemory += size;
18. }

I propose changing sleep(100) into Object.wait(100) or Object.wait() at line 9.

[Reason]

The reserveMemory method is "static synchronized".

It seems like the "sleep(100)" at line 9 is called so that that the
reservedMemory field used at line 14 can wait for the ReferenceHandler
thread to call java.nio.Bits#unreserveMemory(long).

1. static synchronized void unreserveMemory(long size) {
2.   if (reservedMemory > 0) {
3.     reservedMemory -= size;
4.     assert (reservedMemory > -1);
5.   }
6. }

However, because the sleep does not lock a Bits.class object,
and java.nio.Bits#unreserveMemory is also "static synchronied",
java.nio.Bits#unreserveMemory can not run while the
java.nio.Bits#reserveMemory is sleeping.

Here is a proposed implementaion of these methods.
 - change sleep(100) into wait(100) in reserveMemory, and
 - add notifyAll() as last statement in unreserveMemory

 1. static synchronized void reserveMemory(long size) {
    ...
 6.   if (reservedMemory + size > maxMemory) {
 7.     System.gc();
 8.     try {
 9.       Bits.class.wait(100);
10.     } catch (InterruptedException x) {
11.       // Restore interrupt status
12.       Thread.currentThread().interrupt();
13.     }
14.     if (reservedMemory + size > maxMemory)
15.       throw new OutOfMemoryError("Direct buffer memory");
16.     }
17.	reservedMemory += size;
18.  }

1. static synchronized void unreserveMemory(long size) {
2.   if (reservedMemory > 0) {
3.     reservedMemory -= size;
4.     assert (reservedMemory > -1);
5.   }
6.   Bits.class.notifyAll();
7. }

I evaluate these implementations by using the same sample
source code as those of 4879883.
Even with -XX:MexDirectMemorySize=8k, new implementation work fine.

----
import java.nio.ByteBuffer;
public class ByteBufferTest {

  public static void main(String[] args) {
    for (;;) {
      ByteBuffer buf = ByteBuffer.allocateDirect(8192);
    }
  }
}
----

I hope both of v1.4.2_05 and v1.5 release has these implementations,
or more good implemetations.


REPRODUCIBILITY :
This bug can be reproduced always.
(Incident Review ID: 265100) 
======================================================================

Comments
CONVERTED DATA BugTraq+ Release Management Values COMMIT TO FIX: 1.4.2_06 generic tiger-rc FIXED IN: 1.4.2_06 tiger-rc INTEGRATED IN: 1.4.2_06 tiger-b54 tiger-rc VERIFIED IN: 1.4.2_06
23-07-2004

SUGGESTED FIX --- /tmp/geta8207 2004-05-17 09:41:52.000000000 -0700 +++ Bits.java 2004-05-17 09:40:25.000000000 -0700 @@ -607,23 +607,32 @@ // These methods should be called whenever direct memory is allocated or // freed. They allow the user to control the amount of direct memory // which a process may access. All sizes are specified in bytes. - static synchronized void reserveMemory(long size) { - if (!memoryLimitSet && VM.isBooted()) { - maxMemory = VM.maxDirectMemory(); - memoryLimitSet = true; - } - if (size > maxMemory - reservedMemory) { - System.gc(); - try { - Thread.sleep(100); - } catch (InterruptedException x) { - // Restore interrupt status - Thread.currentThread().interrupt(); + static void reserveMemory(long size) { + + synchronized (Bits.class) { + if (!memoryLimitSet && VM.isBooted()) { + maxMemory = VM.maxDirectMemory(); + memoryLimitSet = true; } + if (size <= maxMemory - reservedMemory) { + reservedMemory += size; + return; + } + } + + System.gc(); + try { + Thread.sleep(100); + } catch (InterruptedException x) { + // Restore interrupt status + Thread.currentThread().interrupt(); + } + synchronized (Bits.class) { if (reservedMemory + size > maxMemory) throw new OutOfMemoryError("Direct buffer memory"); + reservedMemory += size; } - reservedMemory += size; + } static synchronized void unreserveMemory(long size) {
23-07-2004

EVALUATION In order to avoid gratuitous OutOfMemoryErrors the fix for 4879883 introduced a 100ms pause after the last-ditch invocation of System.gc. Unfortunately that code, as pointed out by the submitter, does not release the Bits.class monitor. This prevents the reference-handler thread, or any other code, from unreserving direct memory. The fix suggested by the submitter will only work in some cases since it can wake up too early: If the first chunk of memory that's released is smaller than the request being made by the reserving thread then the allocation will still fail. It'd be better for the reserving thread to release the monitor, wait 100ms no matter what (modulo interrupts), then re-acquire the lock and retry the allocation (see patch in suggested-fix section). -- ###@###.### 2004/5/17
05-12-0183