FULL PRODUCT VERSION :
java version "1.6.0_19"
Java(TM) SE Runtime Environment (build 1.6.0_19-b04)
Java HotSpot(TM) Client VM (build 16.2-b04, mixed mode, sharing)
FULL OS VERSION :
Windows XP
EXTRA RELEVANT SYSTEM CONFIGURATION :
a dozen jdbc connections to Oracle and MSSQL and lots of threads running static synchronized methods
A DESCRIPTION OF THE PROBLEM :
was debugging and going step by step through the code
THE PROBLEM WAS REPRODUCIBLE WITH -Xint FLAG: Did not try
THE PROBLEM WAS REPRODUCIBLE WITH -server FLAG: Did not try
STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
step by step debugging of application
EXPECTED VERSUS ACTUAL BEHAVIOR :
normal step by step execution
ERROR MESSAGES/STACK TRACES THAT OCCUR :
#
# A fatal error has been detected by the Java Runtime Environment:
#
# EXCEPTION_ACCESS_VIOLATION (0xc0000005) at pc=0x6d802ef9, pid=3836, tid=2084
#
# JRE version: 6.0_19-b04
# Java VM: Java HotSpot(TM) Client VM (16.2-b04 mixed mode windows-x86 )
# Problematic frame:
# V [jvm.dll+0x2ef9]
#
# An error report file with more information is saved as:
# H:\task04AM_Downloader\Workspace\bmo-fius\etc\hs_err_pid3836.log
#
# If you would like to submit a bug report, please visit:
# http://java.sun.com/webapps/bugreport/crash.jsp
#
REPRODUCIBILITY :
This bug can be reproduced occasionally.
---------- BEGIN SOURCE ----------
#
# A fatal error has been detected by the Java Runtime Environment:
#
# EXCEPTION_ACCESS_VIOLATION (0xc0000005) at pc=0x6d802ef9, pid=3836, tid=2084
#
# JRE version: 6.0_19-b04
# Java VM: Java HotSpot(TM) Client VM (16.2-b04 mixed mode windows-x86 )
# Problematic frame:
# V [jvm.dll+0x2ef9]
#
# If you would like to submit a bug report, please visit:
# http://java.sun.com/webapps/bugreport/crash.jsp
#
--------------- T H R E A D ---------------
Current thread (0x16b21800): VMThread [stack: 0x16bb0000,0x16c00000] [id=2084]
siginfo: ExceptionCode=0xc0000005, reading address 0x0003b20c
Registers:
EAX=0x0003b208, EBX=0x003ae770, ECX=0x131b9708, EDX=0x131b9708
ESP=0x16bff558, EBP=0x16bff55c, ESI=0x0003b208, EDI=0x131b9708
EIP=0x6d802ef9, EFLAGS=0x00010202
Top of Stack: (sp=0x16bff558)
0x16bff558: 131b9708 16bff56c 6d802eec 0003b208
0x16bff568: 6d9af3c2 16bff584 6d820b0b 131b9708
0x16bff578: 16bff5d4 16bff5d4 003abb28 16bff590
0x16bff588: 6d8d467b 131b9958 16bff5a0 6d8ce3d3
0x16bff598: 131b9958 003ae538 16bff5ac 6d8ce183
0x16bff5a8: 003ae730 16bff5c4 6d883124 16bff5d4
0x16bff5b8: 00000000 003abb80 131b9958 16bff5e0
0x16bff5c8: 6d8cdf25 16bff5d4 00000000 6da1cbf0
Instructions: (pc=0x6d802ef9)
0x6d802ee9: 00 00 00 c3 cc cc cc 55 8b ec 56 8b 75 08 8b d1
0x6d802ef9: 8b 4e 04 8b c1 c1 f8 02 85 c9 89 4d 08 7f 2c 7d
Stack: [0x16bb0000,0x16c00000], sp=0x16bff558, free space=13d16bff0cck
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
V [jvm.dll+0x2ef9]
V [jvm.dll+0x2eec]
V [jvm.dll+0x20b0b]
V [jvm.dll+0xd467b]
V [jvm.dll+0xce3d3]
V [jvm.dll+0xce183]
V [jvm.dll+0x83124]
V [jvm.dll+0xcdf25]
V [jvm.dll+0xc8a07]
V [jvm.dll+0x16b2b5]
V [jvm.dll+0x21385]
V [jvm.dll+0xcaf66]
V [jvm.dll+0xcdcf5]
V [jvm.dll+0x1795fb]
V [jvm.dll+0x17988d]
V [jvm.dll+0x179ad3]
V [jvm.dll+0x179b74]
V [jvm.dll+0xd8d76]
V [jvm.dll+0x16b764]
V [jvm.dll+0xb36b9]
V [jvm.dll+0xb37d2]
V [jvm.dll+0x1db7e5]
V [jvm.dll+0x1dbbba]
V [jvm.dll+0x1a1d94]
V [jvm.dll+0xc84cf]
V [jvm.dll+0xca2ce]
V [jvm.dll+0xca52b]
V [jvm.dll+0xce57a]
V [jvm.dll+0x1d8762]
V [jvm.dll+0xc94b8]
V [jvm.dll+0x7e7cb]
V [jvm.dll+0x1f45ab]
V [jvm.dll+0x1f7043]
V [jvm.dll+0x1f62de]
V [jvm.dll+0x1f662c]
V [jvm.dll+0x1f6a52]
V [jvm.dll+0x17f96c]
C [msvcr71.dll+0x9565]
C [kernel32.dll+0xb729]
VM_Operation (0x1785f05c): GenCollectForAllocation, mode: safepoint, requested by thread 0x17169400
--------------- P R O C E S S ---------------
Java Threads: ( => current thread )
0x17169400
[error occurred during error reporting (printing all threads), id 0xc0000005]
VM state:at safepoint (normal execution)
VM Mutex/Monitor currently owned by a thread: ([mutex/lock_event])
[0x003a6fe8] Threads_lock - owner thread: 0x16b21800
[0x003a73f8] Heap_lock - owner thread: 0x17169400
Heap
def new generation total 12800K, used 843K [0x02970000, 0x03750000, 0x07ec0000)
eden space 11392K, 0% used [0x02970000, 0x02970000, 0x03490000)
from space 1408K, 59% used [0x035f0000, 0x036c2fe8, 0x03750000)
to space 1408K, 0% used [0x03490000, 0x03490000, 0x035f0000)
tenured generation total 28324K, used 28231K [0x07ec0000, 0x09a69000, 0x12970000)
the space 28324K, 99% used [0x07ec0000, 0x09a51f60, 0x08e39600, 0x09a69000)
compacting perm gen total 12288K, used 8961K [0x12970000, 0x13570000, 0x16970000)
the space 12288K, 72% used [0x12970000, 0x132305d8, 0x1322d600, 0x13570000)
No shared spaces configured.
Dynamic libraries:
0x00400000 - 0x00424000 C:\Program Files\Java\jre6\bin\javaw.exe
0x7c900000 - 0x7c9b2000 C:\WINDOWS\system32\ntdll.dll
0x7c800000 - 0x7c8f6000 C:\WINDOWS\system32\kernel32.dll
0x77dd0000 - 0x77e6b000 C:\WINDOWS\system32\ADVAPI32.dll
0x77e70000 - 0x77f03000 C:\WINDOWS\system32\RPCRT4.dll
0x77fe0000 - 0x77ff1000 C:\WINDOWS\system32\Secur32.dll
0x7e410000 - 0x7e4a1000 C:\WINDOWS\system32\USER32.dll
0x77f10000 - 0x77f59000 C:\WINDOWS\system32\GDI32.dll
0x7c340000 - 0x7c396000 C:\Program Files\Java\jre6\bin\msvcr71.dll
0x6d800000 - 0x6da97000 C:\Program Files\Java\jre6\bin\client\jvm.dll
0x76b40000 - 0x76b6d000 C:\WINDOWS\system32\WINMM.dll
0x6d7b0000 - 0x6d7bc000 C:\Program Files\Java\jre6\bin\verify.dll
0x6d330000 - 0x6d34f000 C:\Program Files\Java\jre6\bin\java.dll
0x6d290000 - 0x6d298000 C:\Program Files\Java\jre6\bin\hpi.dll
0x76bf0000 - 0x76bfb000 C:\WINDOWS\system32\PSAPI.DLL
0x6d380000 - 0x6d3a9000 C:\Program Files\Java\jre6\bin\jdwp.dll
0x6d6a0000 - 0x6d6a6000 C:\Program Files\Java\jre6\bin\npt.dll
0x6d7f0000 - 0x6d7ff000 C:\Program Files\Java\jre6\bin\zip.dll
0x6d200000 - 0x6d207000 C:\Program Files\Java\jre6\bin\dt_socket.dll
0x71ab0000 - 0x71ac7000 C:\WINDOWS\system32\WS2_32.dll
0x77c10000 - 0x77c68000 C:\WINDOWS\system32\msvcrt.dll
0x71aa0000 - 0x71aa8000 C:\WINDOWS\system32\WS2HELP.dll
0x71a50000 - 0x71a8f000 C:\WINDOWS\System32\mswsock.dll
0x76f20000 - 0x76f47000 C:\WINDOWS\system32\DNSAPI.dll
0x76fb0000 - 0x76fb8000 C:\WINDOWS\System32\winrnr.dll
0x76f60000 - 0x76f8c000 C:\WINDOWS\system32\WLDAP32.dll
0x76fc0000 - 0x76fc6000 C:\WINDOWS\system32\rasadhlp.dll
0x662b0000 - 0x66308000 C:\WINDOWS\system32\hnetcfg.dll
0x71a90000 - 0x71a98000 C:\WINDOWS\System32\wshtcpip.dll
0x6d610000 - 0x6d623000 C:\Program Files\Java\jre6\bin\net.dll
0x68000000 - 0x68036000 C:\WINDOWS\system32\rsaenh.dll
0x769c0000 - 0x76a74000 C:\WINDOWS\system32\USERENV.dll
0x5b860000 - 0x5b8b5000 C:\WINDOWS\system32\netapi32.dll
0x6d790000 - 0x6d798000 C:\Program Files\Java\jre6\bin\sunmscapi.dll
0x77a80000 - 0x77b15000 C:\WINDOWS\system32\CRYPT32.dll
0x77b20000 - 0x77b32000 C:\WINDOWS\system32\MSASN1.dll
VM Arguments:
jvm_args: -agentlib:jdwp=transport=dt_socket,suspend=y,address=localhost:3819 -Xms5m -Xmx256m -Dfile.encoding=Cp1252
java_command: com.bmo.fius.interfaces.AccountMasterDownloader START
Launcher Type: SUN_STANDARD
Environment Variables:
JAVA_HOME=C:\Program Files\Java\jdk1.6.0_19JRE_HOME=C:\Program Files\Rational\RationalSDLC\Common\Java5.0\jre
CLASSPATH=.;C:\Program Files\IBM\WebSphere MQ\Java\lib\com.ibm.mqjms.jar;C:\Program Files\IBM\WebSphere MQ\Java\lib\com.ibm.mq.jar;C:\Program Files\Java\jre6\lib\ext\QTJava.zip;C:\Program Files\Rational\RationalSDLC\ClearQuest\cqjni.jar
PATH=C:/Program Files/Java/jdk1.6.0_19/bin/../jre/bin/client;C:/Program Files/Java/jdk1.6.0_19/bin/../jre/bin;C:\Program Files\Rational\RationalSDLC\common;C:\Program Files\Rational\RationalSDLC\ClearCase\bin;;C:\apache-ant-1.7.1\\bin;C:\Program Files\IBM\WebSphere MQ\Java\lib;C:\ORACLE\product\10.2.0\client_1\BIN\;C:\Program Files\Java\jdk1.6.0_19\bin;C:\Program Files\Java\jre6\bin;C:\Program Files\Business Objects\Crystal Reports 11\Common\3.5\bin\NOTES\;C:\Program Files\Business Objects\Crystal Reports 11\Common\3.5\bin\NOTES\DATA\;C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\System32\Wbem;C:\Program Files\WinZip\;C:\Program Files\Common Files\OTG;C:\PROGRA~1\DISKEE~1\DISKEE~1\;C:\ORANT\BIN\;C:\Program Files\Oracle\jre\1.1.7\bin;C:\Program Files\Oracle\jre\1.3.1\bin;C:\ORACLE\product\10.2.0\client_1\bin;C:\WINDOWS\system32\;C:\WINDOWS\system32\Wbem;C:\Program Files\Windows Imaging\;C:\Program Files\IBM\WebSphere MQ\bin;C:\Program Files\IBM\WebSphere MQ\tools\c\samples\bin;c:\Program Files\Microsoft SQL Server\90\Tools\binn\;C:\Program Files\QuickTime\QTSystem\;
USERNAME=rwolf01
OS=Windows_NT
PROCESSOR_IDENTIFIER=x86 Family 6 Model 23 Stepping 10, GenuineIntel
--------------- S Y S T E M ---------------
OS: Windows XP Build 2600 Service Pack 3
CPU:total 2 (2 cores per cpu, 1 threads per core) family 6 model 23 stepping 10, cmov, cx8, fxsr, mmx, sse, sse2, sse3, ssse3, sse4.1
Memory: 4k page, physical 2061864k(691452k free), swap 3996920k(2472976k free)
vm_info: Java HotSpot(TM) Client VM (16.2-b04) for windows-x86 JRE (1.6.0_19-b04), built on Mar 9 2010 00:48:41 by "java_re" with MS VC++ 7.1 (VS2003)
time: Wed Dec 08 16:36:57 2010
elapsed time: 745 seconds
---------- END SOURCE ----------
CUSTOMER SUBMITTED WORKAROUND :
just run it again inside eclipse's debug mode
Decoded stack trace:
Stack: [0x16bb0000,0x16c00000], sp=0x16bff558, free space=13d16bff0cck
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
V [jvm.dll+0x2ef9];; ?size_given_klass@oopDesc@@QAEHPAVKlass@@@Z+0x9
V [jvm.dll+0x2eec];; ?size@oopDesc@@QAEHXZ+0xc
V [jvm.dll+0x20b0b];; ?block_start_unsafe@BlockOffsetArrayContigSpace@@UBEPAVHeapWord@@PBX@Z+0x9b
V [jvm.dll+0xd467b];; ?block_start_const@OffsetTableContigSpace@@UBEPAVHeapWord@@PBX@Z+0x1b
V [jvm.dll+0xce3d3];; ?do_space@GenerationBlockStartClosure@@UAEXPAVSpace@@@Z+0x23
V [jvm.dll+0xce183];; ?space_iterate@OneContigSpaceCardGeneration@@UAEXPAVSpaceClosure@@_N@Z+0x13
V [jvm.dll+0x83124];; ?space_iterate@CompactingPermGenGen@@UAEXPAVSpaceClosure@@_N@Z+0x14
V [jvm.dll+0xcdf25];; ?block_start@Generation@@UBEPAVHeapWord@@PBX@Z+0x25
V [jvm.dll+0xc8a07];; ?block_start@GenCollectedHeap@@UBEPAVHeapWord@@PBX@Z+0x67
V [jvm.dll+0x16b2b5];; ?method_from_bcp@methodOopDesc@@SAPAV1@PAE@Z+0x15
V [jvm.dll+0x21385];; ?non_breakpoint_code_at@Bytecodes@@SA?AW4Code@1@PAEPAVmethodOopDesc@@@Z+0x15
V [jvm.dll+0xcaf66];; ?next@BytecodeStream@@QAE?AW4Code@Bytecodes@@XZ+0x36
V [jvm.dll+0xcdcf5];; ?compute_map@GenerateOopMap@@QAEXPAVThread@@@Z+0x165
V [jvm.dll+0x1795fb];; ?compute_map@OopMapForCacheEntry@@QAEXPAVThread@@@Z+0x7b
V [jvm.dll+0x17988d];; ?fill@OopMapCacheEntry@@IAEXVmethodHandle@@H@Z+0xcd
V [jvm.dll+0x179ad3];; ?compute_one_oop_map@OopMapCache@@SAXVmethodHandle@@HPAVInterpreterOopMap@@@Z+0x53
V [jvm.dll+0x179b74];; ?lookup@OopMapCache@@QAEXVmethodHandle@@HPAVInterpreterOopMap@@@Z+0x84
V [jvm.dll+0xd8d76];; ?mask_for@instanceKlass@@QAEXVmethodHandle@@HPAVInterpreterOopMap@@@Z+0x66
V [jvm.dll+0x16b764];; ?mask_for@methodOopDesc@@QAEXHPAVInterpreterOopMap@@@Z+0x54
V [jvm.dll+0xb36b9];; ?oops_interpreted_do@frame@@QAEXPAVOopClosure@@PBVRegisterMap@@_N@Z+0x2c9
V [jvm.dll+0xb37d2];; ?oops_do_internal@frame@@AAEXPAVOopClosure@@PAVRegisterMap@@_N@Z+0x22
V [jvm.dll+0x1db7e5];; ?oops_do@JavaThread@@QAEXPAVOopClosure@@@Z+0x115
V [jvm.dll+0x1dbbba];; ?oops_do@Threads@@SAXPAVOopClosure@@@Z+0x1a
V [jvm.dll+0x1a1d94];; ?process_strong_roots@SharedHeap@@QAEX_NW4ScanningOption@1@PAVOopClosure@@PAVOopsInGenClosure@@@Z+0x84
V [jvm.dll+0xc84cf];; ?gen_process_strong_roots@GenCollectedHeap@@QAEXH_N0W4ScanningOption@SharedHeap@@PAVOopsInGenClosure@@2@Z+0x1f
V [jvm.dll+0xca2ce];; ?mark_sweep_phase3@GenMarkSweep@@CAXH@Z+0xce
V [jvm.dll+0xca52b];; ?invoke_at_safepoint@GenMarkSweep@@SAXHPAVReferenceProcessor@@_N@Z+0xab
V [jvm.dll+0xce57a];; ?collect@OneContigSpaceCardGeneration@@UAEX_N0I0@Z+0x3a
V [jvm.dll+0x1d8762];; ?collect@TenuredGeneration@@UAEX_N0I0@Z+0x22
V [jvm.dll+0xc94b8];; ?do_collection@GenCollectedHeap@@IAEX_N0I0H@Z+0x3c8
V [jvm.dll+0x7e7cb];; ?satisfy_failed_allocation@GenCollectorPolicy@@UAEPAVHeapWord@@I_N@Z+0xdb
V [jvm.dll+0x1f45ab];; ?doit@VM_GenCollectForAllocation@@UAEXXZ+0x3b
V [jvm.dll+0x1f7043];; ?evaluate@VM_Operation@@QAEXXZ+0x33
V [jvm.dll+0x1f62de];; ?evaluate_operation@VMThread@@AAEXPAVVM_Operation@@@Z+0x5e
V [jvm.dll+0x1f662c];; ?loop@VMThread@@QAEXXZ+0x2bc
V [jvm.dll+0x1f6a52];; ?run@VMThread@@UAEXXZ+0x62
V [jvm.dll+0x17f96c];; ?java_start@@YGIPAVThread@@@Z+0x8c
C [msvcr71.dll+0x9565]
C [kernel32.dll+0xb729]