Repeatable, non-deterministic, SIGSEGV crash on quad-processor machines
Comments
EVALUATION
Since problem is no longer reproducible with RedHat 3 update 5 and later, closing this
bug. Would need more information to proceed, including data from runs with custom JVM
sent to customer.
26-06-2007
EVALUATION
Additional testing at Chicago Mercantile Exchange indicated the problem is readily reproducible with RedHat 3 update 4, but *not* reproducible using update 5 or update 8. They are migrating to either update 5 or 8, and so the escalation associated with this bug will be closed. Will leave this bug open at a lower priority; hope to get more information from CME.
13-10-2006
EVALUATION
Was able to get several intact hs_err files from the truncated attachment and get
stack traces. Most often we die while scavenging in drain_stacks(), but there is
also a stack trace from a full gc, and one trying to handle an implicit exception
in a (JIT-) compiled method.
1.5.0_05: there are 5 unique stack traces. The stack trace shown for pid 10573
was also seen in 1.5.0_04 and 1.5.0_03. It is also nearly identical to the
trace from pid 10830 except for the offset in drain_stacks(). Similar comment
applies to 8761 and 10184.
error-quad-fadvap01c-1.5.0_05/hs_err_pid10573.log.stack:
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
V [libjvm.so+0x4310d8];; PSPromotionManager::drain_stacks()+0xc8
V [libjvm.so+0x4334f2];; OldToYoungRootsTask::do_it(GCTaskManager*, unsigned int)+0x52
V [libjvm.so+0x20229f];; GCTaskThread::run()+0x17f
V [libjvm.so+0x3f7718];; _start(Thread*)+0x178
C [libpthread.so.0+0x4dec]
The above trace also occurred in 1.5.0_03 and 1.5.0_04.
error-quad-fadvap01c-1.5.0_05/hs_err_pid10830.log.stack:
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
V [libjvm.so+0x431047];; PSPromotionManager::drain_stacks()+0x37
V [libjvm.so+0x4334f2];; OldToYoungRootsTask::do_it(GCTaskManager*, unsigned int)+0x52
V [libjvm.so+0x20229f];; GCTaskThread::run()+0x17f
V [libjvm.so+0x3f7718];; _start(Thread*)+0x178
C [libpthread.so.0+0x4dec]
error-quad-fadvap01c-1.5.0_05/hs_err_pid6931.log.stack:
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
V [libjvm.so+0x4333fd];; StealTask::do_it(GCTaskManager*, unsigned int)+0x8d
V [libjvm.so+0x20229f];; GCTaskThread::run()+0x17f
V [libjvm.so+0x3f7718];; _start(Thread*)+0x178
C [libpthread.so.0+0x4dec]
error-quad-fadvap01c-1.5.0_05/hs_err_pid10184.log.stack:
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
V [libjvm.so+0x43115d];; PSPromotionManager::drain_stacks()+0x14d
V [libjvm.so+0x43340f];; StealTask::do_it(GCTaskManager*, unsigned int)+0x9f
V [libjvm.so+0x20229f];; GCTaskThread::run()+0x17f
V [libjvm.so+0x3f7718];; _start(Thread*)+0x178
C [libpthread.so.0+0x4dec]
error-quad-fadvap01c-1.5.0_05/hs_err_pid8761.log.stack:
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
V [libjvm.so+0x4310d8];; PSPromotionManager::drain_stacks()+0xc8
V [libjvm.so+0x43340f];; StealTask::do_it(GCTaskManager*, unsigned int)+0x9f
V [libjvm.so+0x20229f];; GCTaskThread::run()+0x17f
V [libjvm.so+0x3f7718];; _start(Thread*)+0x178
C [libpthread.so.0+0x4dec]
The above trace also occurred in 1.5.0_04.
1.5.0_04: of the 3 unique stack traces in the 1.5.0_04 data, only one is
substantially different from the 1.5.0_05 traces.
error-quad-fadvap01c-1.5.0_04/hs_err_pid5336.log.stack:
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
V [libjvm.so+0x4b7e53];; VMError::report_and_die()+0x213
V [libjvm.so+0x1ba45c];; report_fatal(char const*, int, char const*)+0x3c
V [libjvm.so+0x3dd31e];; nmethod::continuation_for_implicit_exception(unsigned char*)+0x6e
V [libjvm.so+0x450c2b];; SharedRuntime::continuation_for_implicit_exception(JavaThread*, unsigned char*, SharedRuntime::ImplicitExceptionKind)+0x32b
V [libjvm.so+0x3f2da6];; JVM_handle_linux_signal+0x4b6
V [libjvm.so+0x3f00c4];; signalHandler(int, siginfo*, void*)+0x24
Which corresponds to
# An unexpected error has been detected by HotSpot Virtual Machine:
#
# Internal Error (4E4D4554484F440E4350500521), pid=5336, tid=1209007024;;
;; Error ID is nmethod.cpp, 1313
#
# Java VM: Java HotSpot(TM) Server VM (1.5.0_04-b05 mixed mode)
nmethod.cpp:1313 is
guarantee(cont_offset != 0, "unhandled implicit exception in compiled code");
1.5.0_03: the only trace not seen w/later updates is a crash during a full gc.
error-quad-fadvap01c-1.5.0_03/hs_err_pid7707.log.stack:
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
V [libjvm.so+0x3b532a];; MarkSweep::follow_stack()+0x3a
V [libjvm.so+0x422efe];; PSMarkSweep::mark_sweep_phase1(int&, int)+0x9e
V [libjvm.so+0x42220e];; PSMarkSweep::invoke_no_policy(int*, int)+0x32e
V [libjvm.so+0x426a95];; PSScavenge::invoke(int*)+0xd5
V [libjvm.so+0x3f90cf];; ParallelScavengeHeap::failed_mem_allocate(int*, unsigned int, int, int)+0x4f
V [libjvm.so+0x4bb0c2];; VM_ParallelGCFailedAllocation::doit()+0x42
V [libjvm.so+0x4baca6];; VM_Operation::evaluate()+0x46
V [libjvm.so+0x4ba317];; VMThread::evaluate_operation(VM_Operation*)+0x47
V [libjvm.so+0x4ba525];; VMThread::loop()+0x165
V [libjvm.so+0x4ba130];; VMThread::run()+0x80
V [libjvm.so+0x3eca78];; _start(Thread*)+0x178
C [libpthread.so.0+0x4dec]