JDK-7112413 : JVM Crash, possibly GC-related
  • Type: Bug
  • Component: hotspot
  • Sub-Component: gc
  • Affected Version: 7
  • Priority: P4
  • Status: Closed
  • Resolution: Fixed
  • OS: windows_2008
  • CPU: x86
  • Submitted: 2011-11-16
  • Updated: 2013-10-23
  • Resolved: 2012-03-24
The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.
JDK 6 JDK 7 JDK 8 Other
6u71Fixed 7u4Fixed 8Fixed hs23Fixed
Description
FULL PRODUCT VERSION :
vm_info: Java HotSpot(TM) 64-Bit Server VM (21.1-b02) for windows-amd64 JRE (1.7.0_01-b08), built on Oct  3 2011 01:39:25 by "java_re" with unknown MS VC++:1600

FULL OS VERSION :
Microsoft Windows [Version 6.1.7601]

EXTRA RELEVANT SYSTEM CONFIGURATION :
DELL Server R910, 32GB

A DESCRIPTION OF THE PROBLEM :
Maybe a unsuitable combo of GC- and memory-related XX options, for which there is no proper documentation anyway.


THE PROBLEM WAS REPRODUCIBLE WITH -Xint FLAG: Did not try

THE PROBLEM WAS REPRODUCIBLE WITH -server FLAG: Yes

STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
we try to minimize GC latency while avoiding bad ParNew performance on hardware with many threads/cores

EXPECTED VERSUS ACTUAL BEHAVIOR :
expcted: no crash
actual: crash
ERROR MESSAGES/STACK TRACES THAT OCCUR :
this text field does not allow nearly enough characters to hold the content of hs_err files. Maybe somebody fix this.

#
# A fatal error has been detected by the Java Runtime Environment:
#
#  EXCEPTION_ACCESS_VIOLATION (0xc0000005) at pc=0x0000000002bd89ea, pid=4360, tid=560
#
# JRE version: 7.0_01-b08
# Java VM: Java HotSpot(TM) 64-Bit Server VM (21.1-b02 mixed mode windows-amd64 compressed oops)
# Problematic frame:
# J  java.util.HashMap.put(Ljava/lang/Object;Ljava/lang/Object;)Ljava/lang/Object;
#
# Core dump written. Default location: D:\icubic\rhelbing\workspace\Server Head\hs_err_pid4360.mdmp
#
# If you would like to submit a bug report, please visit:
#   http://bugreport.sun.com/bugreport/crash.jsp
#

---------------  T H R E A D  ---------------

Current thread (0x0000000002941800):  JavaThread "main" [_thread_in_Java, id=560, stack(0x0000000002740000,0x0000000002940000)]

siginfo: ExceptionCode=0xc0000005, writing address 0x000000000797b95c

Registers:
RAX=0x0000000004cf8000, RBX=0x000000057f001b98, RCX=0x00000000b20ebbef, RDX=0x000000059075df78
RSP=0x000000000293e4f0, RBP=0x000000059075df68, RSI=0x0000000002c8395c, RDI=0x00000000fe030996
R8 =0x000000057f001bd0, R9 =0x000000059072b910, R10=0x0000000000038a37, R11=0x0000000012338a37
R12=0x0000000000000000, R13=0x00000005eead5b60, R14=0x00000000b20ebbed, R15=0x0000000002941800
RIP=0x0000000002bd89ea, EFLAGS=0x0000000000010202

  Top of Stack: (sp=0x000000000293e4f0)
0x000000000293e4f0:   000000057f001bd0 000000057f001b98
0x000000000293e500:   00000005fe5dddbf 12338a37fe5dddbf
0x000000000293e510:   b20ebbed00038a37 000000059072b910
0x000000000293e520:   000000059075df68 0000000002f325dc
0x000000000293e530:   00000005e7d3bde0 000000058eba5758
0x000000000293e540:   000000058eba5758 00000005eacca430
0x000000000293e550:   000000058eba5758 0000000590cb0f60
0x000000000293e560:   00000005ef00cf60 0000000002d3cd50
0x000000000293e570:   00000005eacca488 000000058eba57c8
0x000000000293e580:   000000057f001b98 000000058e92fbb8
0x000000000293e590:   000000058eba5758 0000000590cb0f48
0x000000000293e5a0:   00000005b1d01d9e 000000058eba57c8
0x000000000293e5b0:   00000005ee55fa68 000000058eba58a0
0x000000000293e5c0:   0000000000000000 000000058eba5860
0x000000000293e5d0:   000000058eba5758 0000000590cb0f60
0x000000000293e5e0:   00000005eacca430 000000057f001b98

Instructions: (pc=0x0000000002bd89ea)
0x0000000002bd89ca:   48 8b d1 48 c1 e2 03 48 c1 ee 09 48 8b 5c 24 08
0x0000000002bd89da:   48 8b c3 48 c1 e8 03 41 89 41 14 b8 00 80 cf 04
0x0000000002bd89ea:   44 88 24 30 81 ff 20 1d 02 fe 75 6c 48 8b c2 eb
0x0000000002bd89fa:   5b 4c 8b 04 24 41 8b 49 18 85 05 f7 75 54 fd 85


Register to memory mapping:

RAX=0x0000000004cf8000 is an unknown value
RBX=0x000000057f001b98 is an oop
org.hibernate.engine.EntityEntry
 - klass: 'org/hibernate/engine/EntityEntry'
RCX=0x00000000b20ebbef is an unknown value
RDX=0x000000059075df78 is an oop
org.hibernate.engine.EntityEntry
 - klass: 'org/hibernate/engine/EntityEntry'
RSP=0x000000000293e4f0 is pointing into the stack for thread: 0x0000000002941800
RBP=0x000000059075df68 is an oop
org.hibernate.util.IdentityMap$IdentityKey
 - klass: 'org/hibernate/util/IdentityMap$IdentityKey'
RSI=0x0000000002c7e950 [CodeBlob (0x0000000002c7e950)]
Framesize: 60
RDI=0x00000000fe030996 is an unknown value
R8 =0x000000057f001bd0 is an oop
org.hibernate.util.IdentityMap$IdentityKey
 - klass: 'org/hibernate/util/IdentityMap$IdentityKey'
R9 =0x000000059072b910 is an oop
java.util.LinkedHashMap$Entry
 - klass: 'java/util/LinkedHashMap$Entry'
R10=0x0000000000038a37 is an unknown value
R11=0x0000000012338a37 is an unknown value
R12=0x0000000000000000 is an unknown value
R13=0x00000005eead5b60 is an oop
java.util.LinkedHashMap
 - klass: 'java/util/LinkedHashMap'
R14=0x00000000b20ebbed is an unknown value
R15=0x0000000002941800 is a thread

Stack: [0x0000000002740000,0x0000000002940000],  sp=0x000000000293e4f0,  free space=2041k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
J  java.util.HashMap.put(Ljava/lang/Object;Ljava/lang/Object;)Ljava/lang/Object;


---------------  P R O C E S S  ---------------

Java Threads: ( => current thread )
  0x000000001980c800 JavaThread "PerformanceMonitor" [_thread_blocked, id=2176, stack(0x000000001fe40000,0x0000000020040000)]
  0x0000000019806000 JavaThread "ThreadMonitor" [_thread_blocked, id=4432, stack(0x000000001f550000,0x000000001f750000)]
  0x0000000019804800 JavaThread "LatencyStats" [_thread_blocked, id=5080, stack(0x000000001ec10000,0x000000001ee10000)]
  0x000000001980d800 JavaThread "PerformanceStats" [_thread_blocked, id=4464, stack(0x000000001e9e0000,0x000000001ebe0000)]
  0x0000000019801800 JavaThread "RFQManagerThread" [_thread_blocked, id=4416, stack(0x000000001e7e0000,0x000000001e9e0000)]
  0x000000001980d000 JavaThread "NotificationManager" [_thread_blocked, id=4428, stack(0x000000001de60000,0x000000001e060000)]
  0x0000000019809800 JavaThread "Model Timer" [_thread_in_vm, id=4056, stack(0x000000001dc10000,0x000000001de10000)]
  0x000000001980a000 JavaThread "AsyncAppender-Dispatcher-Thread-8" daemon [_thread_blocked, id=3872, stack(0x0000000020770000,0x0000000020970000)]
  0x0000000019806800 JavaThread "AsyncAppender-Dispatcher-Thread-7" daemon [_thread_blocked, id=2188, stack(0x000000001f830000,0x000000001fa30000)]
  0x0000000019807000 JavaThread "AsyncAppender-Dispatcher-Thread-6" daemon [_thread_blocked, id=1284, stack(0x000000001fb60000,0x000000001fd60000)]
  0x0000000019805000 JavaThread "AsyncAppender-Dispatcher-Thread-5" daemon [_thread_blocked, id=5108, stack(0x000000001f220000,0x000000001f420000)]
  0x0000000019956800 JavaThread "TelnetAppender Accept" [_thread_in_native, id=3840, stack(0x000000001e0d0000,0x000000001e2d0000)]
  0x00000000199e5000 JavaThread "AsyncAppender-Dispatcher-Thread-3" daemon [_thread_blocked, id=5088, stack(0x000000001d790000,0x000000001d990000)]
  0x000000001962b000 JavaThread "StatsCounterTimer" [_thread_blocked, id=5084, stack(0x000000001d200000,0x000000001d400000)]
  0x0000000019579000 JavaThread "AWT-Windows" daemon [_thread_in_native, id=4852, stack(0x000000001d400000,0x000000001d600000)]
  0x000000001962a800 JavaThread "Java2D Disposer" daemon [_thread_blocked, id=5068, stack(0x000000001cec0000,0x000000001d0c0000)]
  0x00000000195cf800 JavaThread "RMI TCP Accept-0" daemon [_thread_in_native, id=5060, stack(0x000000001cb90000,0x000000001cd90000)]
  0x00000000195ce800 JavaThread "RMI TCP Accept-44443" daemon [_thread_in_native, id=5048, stack(0x000000001c890000,0x000000001ca90000)]
  0x00000000195bf800 JavaThread "RMI TCP Accept-0" daemon [_thread_in_native, id=5040, stack(0x000000001c5b0000,0x000000001c7b0000)]
  0x0000000019270000 JavaThread "Service Thread" daemon [_thread_blocked, id=3988, stack(0x000000001c220000,0x000000001c420000)]
  0x0000000019263000 JavaThread "C2 CompilerThread1" daemon [_thread_blocked, id=5036, stack(0x000000001bf90000,0x000000001c090000)]
  0x0000000019261800 JavaThread "C2 CompilerThread0" daemon [_thread_blocked, id=5032, stack(0x000000001b550000,0x000000001b650000)]
  0x0000000019253000 JavaThread "JDWP Command Reader" daemon [_thread_in_native, id=2440, stack(0x000000001bc70000,0x000000001be70000)]
  0x0000000019250000 JavaThread "JDWP Event Helper Thread" daemon [_thread_blocked, id=2268, stack(0x000000001ba00000,0x000000001bc00000)]
  0x0000000019241800 JavaThread "JDWP Transport Listener: dt_socket" daemon [_thread_blocked, id=4052, stack(0x000000001b6a0000,0x000000001b8a0000)]
  0x0000000019237800 JavaThread "Attach Listener" daemon [_thread_blocked, id=1996, stack(0x000000001aeb0000,0x000000001b0b0000)]
  0x0000000019236800 JavaThread "Signal Dispatcher" daemon [_thread_blocked, id=916, stack(0x000000001abb0000,0x000000001adb0000)]
  0x0000000019235800 JavaThread "Surrogate Locker Thread (Concurrent GC)" daemon [_thread_blocked, id=3572, stack(0x000000001a990000,0x000000001ab90000)]
  0x00000000191b7000 JavaThread "Finalizer" daemon [_thread_blocked, id=4912, stack(0x000000001a6b0000,0x000000001a8b0000)]
  0x00000000191b0000 JavaThread "Reference Handler" daemon [_thread_blocked, id=4948, stack(0x000000001a350000,0x000000001a550000)]
=>0x0000000002941800 JavaThread "main" [_thread_in_Java, id=560, stack(0x0000000002740000,0x0000000002940000)]

Other Threads:
  0x00000000191a9000 VMThread [stack: 0x000000001a0d0000,0x000000001a1d0000] [id=3656]
  0x000000001960c800 WatcherThread [stack: 0x000000001cd90000,0x000000001ce90000] [id=5064]

VM state:synchronizing (normal execution)

VM Mutex/Monitor currently owned by a thread:  ([mutex/lock_event])
[0x000000000013e7f0] Safepoint_lock - owner thread: 0x00000000191a9000
[0x000000000013e870] Threads_lock - owner thread: 0x00000000191a9000

Heap
 adaptive size par new generation total 241344K, used 53407K [0x000000057f000000, 0x00000005964c0000, 0x00000005e7000000)
  eden space 193280K,   2% used [0x000000057f000000, 0x000000057f537cc8, 0x000000058acc0000)
  from space 48064K, 100% used [0x000000058e800000, 0x00000005916f0000, 0x00000005916f0000)
  to   space 48064K,   0% used [0x000000058acc0000, 0x000000058acc0000, 0x000000058dbb0000)
 concurrent mark-sweep generation total 349440K, used 194444K [0x00000005e7000000, 0x00000005fc540000, 0x00000007f0000000)
 concurrent-mark-sweep perm gen total 262144K, used 49944K [0x00000007f0000000, 0x0000000800000000, 0x0000000800000000)

Code Cache  [0x0000000002a40000, 0x0000000002ff0000, 0x0000000005a40000)
 total_blobs=2052 nmethods=1412 adapters=598 free_code_cache=43595Kb largest_free_block=44383168



REPRODUCIBILITY :
This bug can be reproduced always.

---------- BEGIN SOURCE ----------
only tried with out software...
here the VM options in case they get chopped off the error message text field:

VM Arguments:
jvm_args: -agentlib:jdwp=transport=dt_socket,suspend=y,address=localhost:49280 -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.port=44443 -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Dserver.jmx -Djava.library.path=..\MMKF4Java\dll -Xss2m -Xmx10000M -XX:PermSize=256m -XX:+UseConcMarkSweepGC -XX:MaxGCPauseMillis=50 -XX:+UseAdaptiveSizePolicy -XX:ParallelGCThreads=20 -XX:+BindGCTaskThreadsToCPUs -XX:+UseGCTaskAffinity -XX:+UseCompressedOops -XX:+DoEscapeAnalysis -Xloggc:logs/gc.log -verbose:gc -XX:+PrintGCDateStamps -XX:+PrintGCDetails -XX:+UnlockDiagnosticVMOptions -XX:+PrintFlagsFinal -XX:+LogVMOutput -XX:LogFile=logs/jvm.log -Dfile.encoding=Cp1252


---------- END SOURCE ----------

CUSTOMER SUBMITTED WORKAROUND :
try any other arbitrary combo of undocumented vm arguments

Comments
EVALUATION http://hg.openjdk.java.net/lambda/lambda/hotspot/rev/a5244e07b761
22-03-2012

EVALUATION http://hg.openjdk.java.net/hsx/hotspot-gc/hotspot/rev/a5244e07b761
26-01-2012

EVALUATION Do the latter (disable and print a warning).
17-01-2012

EVALUATION The option -XX:+UseAdaptiveSizePolicy does not work with the CMS collector (-XX:+UseConcMarkSweepGC). This option combination should be detected and the JVM should either reject it and print an appropriate message, or disable UseAdaptiveSizePolicy and issue a warning that it is doing so.
16-11-2011

WORK AROUND Remove the option -XX:+UseAdaptiveSizePolicy.
16-11-2011