JDK-6506133 : Hotspot JVM Out of Memory Error in JDK 1.6RC1 but not in JDK 1.5.0_09
  • Type: Bug
  • Component: hotspot
  • Sub-Component: gc
  • Affected Version: 6
  • Priority: P3
  • Status: Closed
  • Resolution: Fixed
  • OS: windows_2003
  • CPU: x86
  • Submitted: 2006-12-19
  • Updated: 2011-03-08
  • Resolved: 2011-03-08
The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.
JDK 6 JDK 7 Other
6u10Fixed 7Fixed hs11Fixed
Related Reports
Relates :  
Description
FULL PRODUCT VERSION :
java version "1.6.0-rc"
Java(TM) SE Runtime Environment (build 1.6.0-rc-b104)
Java HotSpot(TM) Client VM (build 1.6.0-rc-b104, mixed mode, sharing)

FULL OS VERSION :
Microsoft Windows [Version 5.2.3790]

A DESCRIPTION OF THE PROBLEM :
It appears that something has changed between Java 1.5.0_09 and Java 1.6RC that is preventing our application from allocating large amounts of heap space.  In Java 1.5 the attached program would run well past when "17000" was output with 1400m allocated to the JVM.  In Java 1.6RC the counter of the attached program doesn't even get to 2000.

THE PROBLEM WAS REPRODUCIBLE WITH -Xint FLAG: Yes

THE PROBLEM WAS REPRODUCIBLE WITH -server FLAG: Yes

STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
Run the attached code sample with the vm options "-Xms1400m -Xmx1400m -server"

EXPECTED VERSUS ACTUAL BEHAVIOR :
Expected:
Code is able to execute in Java 1.6 with roughly the same memory requirements of 1.5

Actual:
Processing fails almost immeadiatly in the 1.6 JVM vs completing in the 1.5 JVM
ERROR MESSAGES/STACK TRACES THAT OCCUR :
#
# An unexpected error has been detected by Java Runtime Environment:
#
# java.lang.OutOfMemoryError: requested 83886080 bytes for GrET in C:\BUILD_AREA\jdk6\hotspot\src\share\vm\utilities\growableArray.cpp. Out of swap space?
#
#  Internal Error (414C4C4F434154494F4E0E494E4C494E450E4850500017), pid=896, tid=1948
#
# Java VM: Java HotSpot(TM) Server VM (1.6.0-rc-b104 mixed mode)
# If you would like to submit a bug report, please visit:
#   http://java.sun.com/webapps/bugreport/crash.jsp
#

---------------  T H R E A D  ---------------

Current thread (0x0028e800):  GCTaskThread [id=1948]

Stack: [0x5f810000,0x5f860000)
[error occurred during error reporting, step 110, id 0xc0000005]


---------------  P R O C E S S  ---------------

Java Threads: ( => current thread )
  0x5f985000 JavaThread "Low Memory Detector" daemon [_thread_blocked, id=360]
  0x5f983000 JavaThread "CompilerThread1" daemon [_thread_blocked, id=3836]
  0x5f97e800 JavaThread "CompilerThread0" daemon [_thread_blocked, id=2040]
  0x5f95f400 JavaThread "Attach Listener" daemon [_thread_blocked, id=132]
  0x5f97dc00 JavaThread "Signal Dispatcher" daemon [_thread_blocked, id=2768]
  0x5f95b400 JavaThread "Finalizer" daemon [_thread_blocked, id=1448]
  0x5f957c00 JavaThread "Reference Handler" daemon [_thread_blocked, id=1368]
  0x00286000 JavaThread "main" [_thread_blocked, id=2584]

Other Threads:
  0x5f970c00 VMThread [id=2732]
  0x5f986400 WatcherThread [id=1340]

VM state:at safepoint (normal execution)

VM Mutex/Monitor currently owned by a thread:  ([mutex/lock_event])
[0x00284c98/0x00000710] Threads_lock - owner thread: 0x5f970c00
[0x00284e38/0x000006d0] Heap_lock - owner thread: 0x00286000

Heap
 PSYoungGen      total 139328K, used 139328K [0x55650000, 0x5f1d0000, 0x5f1d0000)
  eden space 119424K, 100% used [0x55650000,0x5caf0000,0x5caf0000)
  from space 19904K, 100% used [0x5caf0000,0x5de60000,0x5de60000)
  to   space 19904K, 0% used [0x5de60000,0x5de60000,0x5f1d0000)
 PSOldGen        total 1274368K, used 189996K [0x079d0000, 0x55650000, 0x55650000)
  object space 1274368K, 14% used [0x079d0000,0x1335b0f8,0x55650000)
 PSPermGen       total 16384K, used 1664K [0x039d0000, 0x049d0000, 0x079d0000)
  object space 16384K, 10% used [0x039d0000,0x03b700a0,0x049d0000)

Dynamic libraries:
0x00400000 - 0x00423000 	C:\dev\jdk1.6.0RC\bin\java.exe
0x7c800000 - 0x7c8c0000 	C:\WINDOWS\system32\ntdll.dll
0x77e40000 - 0x77f42000 	C:\WINDOWS\system32\kernel32.dll
0x77f50000 - 0x77fec000 	C:\WINDOWS\system32\ADVAPI32.dll
0x77c50000 - 0x77cef000 	C:\WINDOWS\system32\RPCRT4.dll
0x7c340000 - 0x7c396000 	C:\dev\jdk1.6.0RC\jre\bin\msvcr71.dll
0x6dac0000 - 0x6ddf5000 	C:\dev\jdk1.6.0RC\jre\bin\server\jvm.dll
0x77380000 - 0x77412000 	C:\WINDOWS\system32\USER32.dll
0x77c00000 - 0x77c48000 	C:\WINDOWS\system32\GDI32.dll
0x76aa0000 - 0x76acd000 	C:\WINDOWS\system32\WINMM.dll
0x71bc0000 - 0x71bc8000 	C:\WINDOWS\system32\rdpsnd.dll
0x771f0000 - 0x77201000 	C:\WINDOWS\system32\WINSTA.dll
0x77ba0000 - 0x77bfa000 	C:\WINDOWS\system32\msvcrt.dll
0x71c40000 - 0x71c98000 	C:\WINDOWS\system32\NETAPI32.dll
0x76b70000 - 0x76b7b000 	C:\WINDOWS\system32\PSAPI.DLL
0x6d310000 - 0x6d318000 	C:\dev\jdk1.6.0RC\jre\bin\hpi.dll
0x6d770000 - 0x6d77c000 	C:\dev\jdk1.6.0RC\jre\bin\verify.dll
0x6d3b0000 - 0x6d3cf000 	C:\dev\jdk1.6.0RC\jre\bin\java.dll
0x6d7b0000 - 0x6d7bf000 	C:\dev\jdk1.6.0RC\jre\bin\zip.dll

VM Arguments:
jvm_args: -Xms1400m -Xmx1400m
java_command: MemTest
Launcher Type: SUN_STANDARD

Environment Variables:
JAVA_HOME=C:\dev\jdk1.6.0RC
PATH=C:\dev\jdk1.6.0RC\bin;c:\Perl\bin;C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\System32\Wbem;C:\Program Files\Microsoft SQL Server\80\Tools\BINN;C:\Program Files\cvsnt;C:\dev\build\apache-ant-1.6.5\bin
USERNAME=nwest
OS=Windows_NT
PROCESSOR_IDENTIFIER=x86 Family 15 Model 2 Stepping 9, GenuineIntel



---------------  S Y S T E M  ---------------

OS: Windows Server 2003 family Build 3790 Service Pack 1

CPU:total 4 family 15, cmov, cx8, fxsr, mmx, sse, sse2, ht

Memory: 4k page, physical 2097151k(2097151k free), swap 4194303k(4194303k free)

vm_info: Java HotSpot(TM) Server VM (1.6.0-rc-b104) for windows-x86, built on Nov  1 2006 00:39:24 by "java_re" with unknown MS VC++:1310



REPRODUCIBILITY :
This bug can be reproduced always.

---------- BEGIN SOURCE ----------
	public static void main(String[] args) {
		Bucket[] recs = new Bucket[20000];
		int num = 17000;

		for(int i=0; i<recs.length; i++){
			recs[i] = new Bucket();
		}

		for(int i=0; i<num; i++){
			IDObject o = new IDObject();
			for (Bucket rec : recs) {
				rec.setIdObject(o);
			}
			if(i%1000 == 0){
				System.out.println(i);
			}
		}

	}

	private static class IDObject{
		int id=0;
	}

	private static class Bucket {

		List<IDObject> list = new ArrayList<IDObject>();

		void setIdObject(IDObject ob){
			list.add(ob);
		}
	}
---------- END SOURCE ----------

CUSTOMER SUBMITTED WORKAROUND :
Use Java 1.5.0_09

Comments
EVALUATION I actually managed to reproduce the problem on a SPARC box. The trick was to use a small number of GC threads (in my case: 2) to ensure that each GC thread does a lot of work. The cmd line parameter I used is the following one: java -XX:ParallelGCThreads=2 -verbosegc -Xms2g -Xmx2g -XX:+UseParallelGC MemTest If larger number of threads are used (say 8 or 16, as I was running on big boxes) the OOM never happened. The fix outlined in the Suggested Fix section actually does fix the issue and allows the small test to complete. Interestingly, the last GC before the OOM happened, the overflow stacks grew to 40M entries (which is 160MB!!!) each. With the fix the overflow stacks never grow to more than 2K entries, and that's only in 2-3 GCs.
25-09-2007

SUGGESTED FIX I've revamped parallel scavenge to deal with this problem. There are two main changes that help avoid the OOM error. - First, we now chunk arrays we scan. For any non-small array, we only process part of it and push the rest on the stack. This way, we will never push all the entries of an array on the stack one after the other, which reduces the probability that we'll need to hit the overflow stack. The way we do the chunking is similar to what ParNew does: we use the length field on the from-image of the object to mark how much we've scanned and we push on the stack an oop to the from-image. - Second, we now drain the stacks more aggressively when scanning the card table. It was the case that we would scan large chunks of the card table and push many entries, before attempting to drain the stacks. This also contributed to the problem in question.
25-09-2007

EVALUATION Moved this entry to the "Comments" section.
03-01-2007

EVALUATION I actully can't reproduce the bug in-house on both Windows Server 2003 and XP systems.
02-01-2007