JDK-4879883 : (bf) Allocating direct buffers causes OutOfMemoryError
  • Type: Bug
  • Component: core-libs
  • Sub-Component: java.nio
  • Affected Version: 3.5sp1,1.4.1,1.4.2
  • Priority: P3
  • Status: Resolved
  • Resolution: Fixed
  • OS:
    generic,solaris_7,solaris_9,windows_xp generic,solaris_7,solaris_9,windows_xp
  • CPU: generic,x86,sparc
  • Submitted: 2003-06-17
  • Updated: 2017-05-16
  • Resolved: 2003-12-19
The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.
Other
1.4.2_05 05Fixed
Related Reports
Duplicate :  
Relates :  
Relates :  
Relates :  
Description
Name: rmT116609			Date: 06/17/2003


FULL PRODUCT VERSION :
java version "1.4.2-beta"
Java(TM) 2 Runtime Environment, Standard Edition (build 1.4.2-beta-b19)
Java HotSpot(TM) Client VM (build 1.4.2-beta-b19, mixed mode)


FULL OS VERSION :
SunOS pantar 5.7 Generic_106541-17 sun4u sparc SUNW,Ultra-4


A DESCRIPTION OF THE PROBLEM :
Allocating a direct byte buffer may give an OutOfMemoryError, even when plenty of memory is available.  Although not used in the test case below, in my application this occurs when reading a socket channel:

 java.lang.OutOfMemoryError
     at java.nio.Bits.reserveMemory(Bits.java:618)
     at java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:95)
     at java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:285)
     at sun.nio.ch.Util.getTemporaryDirectBuffer(Util.java:54)
     at sun.nio.ch.IOUtil.read(IOUtil.java:205)
     at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:207)
     at sun.nio.ch.SocketAdaptor$SocketInputStream.read(SocketAdaptor.java:171)
     at sun.nio.ch.ChannelInputStream.read(ChannelInputStream.java:86)


STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
Compile source below and run it:

javac ByteBufferTest.java

java ByteBufferTest

EXPECTED VERSUS ACTUAL BEHAVIOR :
EXPECTED -
  Program keeps running forever (this is what happens with JDK 1.4.1_02).
ACTUAL -
See exception below.

ERROR MESSAGES/STACK TRACES THAT OCCUR :
Exception in thread "main" java.lang.OutOfMemoryError
        at java.nio.Bits.reserveMemory(Bits.java:618)
        at java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:95)
        at java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:285)
        at ByteBufferTest.main(ByteBufferTest.java:7)


REPRODUCIBILITY :
This bug can be reproduced always.

---------- BEGIN SOURCE ----------
import java.nio.ByteBuffer;

public class ByteBufferTest {

  public static void main(String[] args) {
    for (;;) {
      ByteBuffer buf = ByteBuffer.allocateDirect(8192);
    }
  }
}

---------- END SOURCE ----------
(Review ID: 188204) 
======================================================================

Name: jl125535			Date: 08/20/2003


It seems that the direct buffers should be able to access the  32 bits (or at least 31 bits) that the underlying OS can provide.

For example,  with jdk1.4.1 under windows it is possible to map a file into memory that is no larger than 1.6 mb *minus* the amount of memory already allocated to the VM.
This is regardless of the physical memory available.

Why not have access to the full 2-4 GB via a memory mapped file?
Better yet, within the constraints of the os, given that sun.misc.Unsafe is manipulating long pointers anyway, there seems to be no reason that *EACH* MappedByteBuffer should not each have a limit of 4 GB.

JUSTIFICATION :
+Large data-instensive applications can get extreme performance improvements with access to large amounts of data in cheap memory.
+Memory mapped access to large files is not available

Multiple 4GB direct buffers should give most people access to all the memory they need, without the need for 64 bit pointers.
(Review ID: 184727)
======================================================================

Comments
CONVERTED DATA BugTraq+ Release Management Values COMMIT TO FIX: 1.4.2_05 generic tiger-beta FIXED IN: 1.4.2_05 tiger-beta INTEGRATED IN: 1.4.2_05 tiger-b32 tiger-beta
14-06-2004

EVALUATION As of 1.4.2 there is a limit to the amount of direct memory which an application can allocate. The default value for this limit is 64MB. You may change it via the VM flag -XX:MaxDirectMemorySize, which takes a size parameter just like the other memory-sizing flags, e.g., % java -XX:MaxDirectMemorySize=256M MyClass Setting a fixed default limit was, in hindsight, a mistake. The default should instead be proportional to the amount of memory dedicated to the Java heap. The default will be changed so that it's roughly equal to the heap's maximum size, i.e, the value of Runtime.getRuntime().maxMemory(). -- ###@###.### 2003/8/23
08-11-0189