JDK-6302804 : Hotspot VM dies ungraceful death when C heap is exhausted in various places.
  • Type: Bug
  • Component: hotspot
  • Sub-Component: runtime
  • Affected Version:
    osol_2009.06,solaris_11,hs19,5.0u13,6,6u2,6u5,6u10,6u11,6u23,7 osol_2009.06,solaris_11,hs19,5.0u13,6,6u2,6u5,6u10,6u11,6u23,7
  • Priority: P3
  • Status: Closed
  • Resolution: Fixed
  • OS:
    generic,linux,linux_redhat_4.0,open_solaris,solaris_nevada,solaris_10,windows,windows_xp generic,linux,linux_redhat_4.0,open_solaris,solaris_nevada,solaris_10,windows,windows_xp
  • CPU: generic,x86,sparc
  • Submitted: 2005-07-27
  • Updated: 2017-03-30
  • Resolved: 2011-03-08
The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.
JDK 6 JDK 7 Other
6u25Fixed 7Fixed hs20Fixed
Related Reports
Duplicate :  
Duplicate :  
Duplicate :  
Duplicate :  
Duplicate :  
Duplicate :  
Duplicate :  
Duplicate :  
Duplicate :  
Relates :  
Relates :  
JDK        	    : Mustang b44
Platform[s]         : Fails On:  winXP Prof.
Failing Test [s]    : CTE_REGTEST/Generic/4765019/ThreadTest.java

    Test source location:

    jtr file location:

    How to reproduce:
    - Set JAVA_HOME to Mustang b44 windows-i586
    - cd /net/jdk/export/jpse04/Regression/1.6.0/test/CTE_REGTEST/Generic/4765019     
    - /net/koori.sfbay/onestop/jct-tools/2.1.6/archive/fcs/binaries/win32/bin/jtreg -r:/tmp -w:/tmp ThreadTest.java

    Test output:
Exception in thread "main" java.lang.OutOfMemoryError: unable to create new native thread
Exception in thread "main" java.lang.OutOfMemoryError: unable to create new native thread
        at java.lang.Thread.start0(Native Method)
        at java.lang.Thread.start(Thread.java:587)
        at ThreadTest.main(ThreadTest.java:17)
Unrecognized VM option '+UseDefaultStackSize'
Unrecognized VM option '+UseDefaultStackSize'
Could not create the Java virtual machine.
java full version "1.6.0-ea-b44"
+ grep Unrecognized VM option test.out 
+ [ 1 != 0 ] 
+ + wc -l test.out 
+ awk {print $1} 
+ grep OutOfMemoryError test.out 
+ [ 0 = 0 ] 
+ + grep -n OutOfMemoryError test.out 
+ cut -f1 -d : 
+ + expr 7111 - 1 
+ + head -7110 test.out 
+ tail -1 
+ tail test.out 
+ VAL1=7109 
+ [ 7109 = 0 ] 
+ Execute_Command -Xss256K -XX:+UseDefaultStackSize 0 22616 
+ cd Y:/dtf/unified/knight-ws/exec/regression-cte-reg-XPpro-2005-07-25-10-50-09-0981/results/workDir/scratch 
+ VAL=0 
+ rm -f test.out 
+ C:/jdk/b44/windows-i586/jdk1.6.0\\bin\\java -Xss256K -XX:+UseDefaultStackSize ThreadTest 
+ sleep 10 
+ [ 0 = 0 ] 
+ + ps 
+ grep ThreadTest 
+ grep -v grep 
+ awk {print $1} 
+ sleep 120 
+ kill -s 9 
kill: Execute_Command 3: Y:\\dtf\\unified\\knight-ws\\suites\\regression\\cte-test\\CTE_REGTEST\\Generic\\4765019\\Test4765019.sh 128: Usage: kill -l [status]
        kill [-s SIGNAL] job ...
+ chmod 777 ThreadTest.class test.out 
+ set -x 
+ grep Unrecognized VM option test.out 
+ [ 0 != 0 ] 
+ tail test.out 
+ VAL3=0 
+ + echo 7109 
+ awk {AA = $1; BB = AA * 0.9; print BB} 
+ [ 0 -ge 7109 -o 0 -ge 6398.1 ] 
[: Y:\\dtf\\unified\\knight-ws\\suites\\regression\\cte-test\\CTE_REGTEST\\Generic\\4765019\\Test4765019.sh 141: unknown operator in arithmetic expression "6398.1" near 8.
result: Failed. Execution failed: exit code 1
The hs_err* file is incomplete because it gets an error while printing out all of the threads, but if I comment out that part of vmError.cpp, I can get a stack trace:

Stack: [0xf5da0000,0xf5de0000),  sp=0xf5ddeb0c,  free space=250k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
V  [libjvm.so+0x8f9c47] void VMError::report_and_die()+0x597
V  [libjvm.so+0x38cbee] void report_vm_out_of_memory(const char*,int,unsigned,const char*)+0xae
V  [libjvm.so+0x1ee067] void*Chunk::operator new(unsigned,unsigned)+0xf7
V  [libjvm.so+0x1ee90f] void*Arena::grow(unsigned)+0x4f
V  [libjvm.so+0x1ef9ef] void*Arena::Amalloc(unsigned)+0xbf
V  [libjvm.so+0x330aef] Node_Stack::Node_Stack(int)+0x7f
V  [libjvm.so+0x6bdbd4] MStack::MStack(int)+0x34
V  [libjvm.so+0x6baa06] void Matcher::find_shared(Node*)+0x56
V  [libjvm.so+0x6b61a5] void Matcher::match()+0x705
V  [libjvm.so+0x32d255] void Compile::Code_Gen()+0xc5
V  [libjvm.so+0x32990c] Compile::Compile(ciEnv*,C2Compiler*,ciMethod*,int,bool)+0xd1c
V  [libjvm.so+0x24ffe7] void C2Compiler::compile_method(ciEnv*,ciMethod*,int)+0x77
V  [libjvm.so+0x33860b] void CompileBroker::invoke_compiler_on_method(CompileTask*)+0x3d7
V  [libjvm.so+0x337e2c] void CompileBroker::compiler_thread_loop()+0x3a4
V  [libjvm.so+0x88e046] void compiler_thread_entry(JavaThread*,Thread*)+0x66
V  [libjvm.so+0x8891cc] void JavaThread::thread_main_inner()+0x15c
V  [libjvm.so+0x88905e] void JavaThread::run()+0x11e
V  [libjvm.so+0x732c2e] java_start+0x19e
C  [libc.so.1+0x9f6f8] _thr_setup+0x4e
C  [libc.so.1+0x9f9e0] _lwp_start+0x0

The compiler is still running when the VM runs out of memory, and blows up trying to allocate memory for compilation.

To get this on x86 -server run:  java -mx10m ThreadTest

EVALUATION http://hg.openjdk.java.net/jdk7/hotspot/hotspot/rev/36c186bcc085

EVALUATION I changed the error reporting so that out of swap and out of native (C heap) memory doesn't look like a VM assert or look like a java.lang.OutOfMemoryError exception that someone could catch. The suggested new wording is in the bug link Evaluation section. I'm open to rewording if people have strong preferences. Also, if solaris runs out of swap space (generally by filling up /tmp), access to thread stacks generate a bus error with ENOMEM errno. I can check for this in the solaris signal handler and give the out of C heap message. I couldn't get Linux or windows to fail the same way, so didn't apply similar changes there.

EVALUATION http://hg.openjdk.java.net/jdk7/hotspot-rt/hotspot/rev/36c186bcc085

EVALUATION Today the vast majority of service calls are due to the fact that the 4g process size limit for 32 bit has been hit. There is almost never a shortage in RAM/swap. It might be a good idea to explicitly mention that fact. E.g. by adding a "possible reasons" section: # Possible reasons: # In 32 bit mode, the 4g process size limit was hit # The system was out of physical RAM or swap space # Possible solutions: .... The message "out of swap space?" which we get today is a red herring

EVALUATION Customers do hit this problem but it won't be fixed in the next release. There's a fair bit of work to gracefully handle C heap exhaustion in the VM.

EVALUATION This is really an enhancement to oom handling where the C heap is exhausted and we need more space. It might be a long time before we fix this. This is not a customer generated request.

EVALUATION It's not just the compiler thread. Any thread that tries to allocate memory in an arena when the created threads have eaten up all of memory, will crash: Stack: [0xf5de2000,0xf5e32000), sp=0xf5e30938, free space=314k Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code) V [libjvm.so+0x8f9c87] void VMError::report_and_die()+0x597 V [libjvm.so+0x38cc2e] void report_vm_out_of_memory(const char*,int,unsigned,const char*)+0xae V [libjvm.so+0x1efef1] void*ChunkPool::allocate(unsigned)+0x101 V [libjvm.so+0x1edff1] void*Chunk::operator new(unsigned,unsigned)+0x81 V [libjvm.so+0x1ee90f] void*Arena::grow(unsigned)+0x4f V [libjvm.so+0x1ef9ef] void*Arena::Amalloc(unsigned)+0xbf V [libjvm.so+0x7e62e5] char*ResourceArea::allocate_bytes(unsigned)+0xc5 V [libjvm.so+0x7e5c71] char*resource_allocate_bytes(unsigned)+0x41 V [libjvm.so+0x45e06b] void*GenericGrowableArray::raw_allocate(int)+0x4b V [libjvm.so+0x4ca2f1] GrowableArray<methodOop>::GrowableArray(int,bool)+0x61 V [libjvm.so+0x6561ec] int klassVtable::get_num_mirandas(klassOop,objArrayOop,objArrayOop)+0x5c V [libjvm.so+0x652d2d] void klassVtable::compute_vtable_size_and_num_mirandas(int&,int&,klassOop,objArrayOop,AccessFlags,oop,symbolOop,objArrayOop)+0x2ad This is a day one problem and is too risky to change the behaviour of the VM for mustang. OutOfMemoryError is thrown and a message in the assert. Are there more bugs open that request better oom handling? # An unexpected error has been detected by Java Runtime Environment: # # java.lang.OutOfMemoryError: requested 32764 bytes for ChunkPool::allocate. Out of swap space? #