Name: pa48320 Date: 07/07/2003
We have been seeing this problem intermittently during our nightly regression. What happens in that the JRE just hangs. You can see that the process is still alive using ps or top but it does not respond to "kill -QUIT" to generate a stack trace. Connecting to that process using gdb, we can see that the stack shown below. The stack indicates that the JVM is responding to an unexpected exception, presumably trying to generate the hs_pid log but then it blocks while trying to get the current time. This makes it very hard to debug the unexpected exception that was raised and consumed by the JRE since the hs_pid log is never generated.
Finally, we have seen this only since we moved to JRE 1.4.1.02. We haven't seen it with 1.3.1.07.
here is the gdb stack trace:
#0 0xff29f160 in _lwp_sema_wait () from /usr/lib/libc.so.1
#1 0xff359acc in _park () from /usr/lib/libthread.so.1
#2 0xff359794 in _swtch () from /usr/lib/libthread.so.1
#3 0xff35b150 in _mutex_adaptive_lock () from /usr/lib/libthread.so.1
#4 0xff35ae88 in _cmutex_lock () from /usr/lib/libthread.so.1
#5 0xff242b1c in free () from /usr/lib/libc.so.1
#6 0xff2534fc in tzcpy () from /usr/lib/libc.so.1
#7 0xff253460 in getzname () from /usr/lib/libc.so.1
#8 0xff252f64 in _ltzset_u () from /usr/lib/libc.so.1
#9 0xff25212c in localtime_u () from /usr/lib/libc.so.1
#10 0xff252334 in localtime_r () from /usr/lib/libc.so.1
#11 0xfe338e3c in __1cCosSreport_fatal_error6FpnMoutputStream_pCi_v_ ()
from /ade_autofs/ade/OUI_MAIN_SOLARIS.rdd/030624/jre14/lib/sparc/client/libjvm.so
#12 0xfe339284 in __1cCosbBhandle_unexpected_exception6FpnGThread_ipCpv_v_ ()
from /ade_autofs/ade/OUI_MAIN_SOLARIS.rdd/030624/jre14/lib/sparc/client/libjvm.so
#13 0xfe33d418 in JVM_handle_solaris_signal ()
from /ade_autofs/ade/OUI_MAIN_SOLARIS.rdd/030624/jre14/lib/sparc/client/libjvm.so
#14 0xff36b82c in __sighndlr () from /usr/lib/libthread.so.1
#15 <signal handler called>
#16 0xff2427a0 in t_delete () from /usr/lib/libc.so.1
#17 0xff242394 in realfree () from /usr/lib/libc.so.1
#18 0xff242c68 in cleanfree () from /usr/lib/libc.so.1
#19 0xff241d9c in _malloc_unlocked () from /usr/lib/libc.so.1
#20 0xff241c90 in malloc () from /usr/lib/libc.so.1
#21 0xfe0492f8 in __1cCosGmalloc6FI_pv_ ()
from /ade_autofs/ade/OUI_MAIN_SOLARIS.rdd/030624/jre14/lib/sparc/client/libjvm.so
#22 0xfe049298 in __1cICHeapObj2n6FI_pv_ ()
from /ade_autofs/ade/OUI_MAIN_SOLARIS.rdd/030624/jre14/lib/sparc/client/libjvm.so
#23 0xfe0a0cf4 in __1cQSystemDictionaryYadd_klass_or_placeholder6FpnNsymbolOopDesc_pnHoopDesc_4_v_ ()
from /ade_autofs/ade/OUI_MAIN_SOLARIS.rdd/030624/jre14/lib/sparc/client/libjvm.so
#24 0xfe09f8fc in __1cQSystemDictionarybEresolve_instance_class_or_null6FnMsymbo
lHandle_nGHandle_2pnGThread__pnMklassOopDesc__ ()
from /ade_autofs/ade/OUI_MAIN_SOLARIS.rdd/030624/jre14/lib/sparc/client/libjvm.so
#25 0xfe09f220 in __1cQSystemDictionaryPresolve_or_null6FnMsymbolHandle_nGHandle_2pnGThread__pnMklassOopDesc__ ()
from /ade_autofs/ade/OUI_MAIN_SOLARIS.rdd/030624/jre14/lib/sparc/client/libjvm.so
#26 0xfe09ede0 in __1cQSystemDictionaryPresolve_or_fail6FnMsymbolHandle_nGHandle_2ipnGThread__pnMklassOopDesc__ ()
from /ade_autofs/ade/OUI_MAIN_SOLARIS.rdd/030624/jre14/lib/sparc/client/libjvm.so
#27 0xfe0d7154 in __1cTconstantPoolOopDescNklass_at_impl6FnSconstantPoolHandle_ipnGThread__pnMklassOopDesc__ ()
from /ade_autofs/ade/OUI_MAIN_SOLARIS.rdd/030624/jre14/lib/sparc/client/libjvm.so
#28 0xfe0dcb6c in __1cSInterpreterRuntimeE_new6FpnKJavaThread_pnTconstantPoolOopDesc_i_v_ ()
from /ade_autofs/ade/OUI_MAIN_SOLARIS.rdd/030624/jre14/lib/sparc/client/libjvm.so
#29 0xfa41585c in ?? ()
#30 0xfa405c6c in ?? ()
#31 0xfa405c6c in ?? ()
#32 0xfa405c6c in ?? ()
#33 0xfa405b5c in ?? ()
#34 0xfa406024 in ?? ()
#35 0xfa405b5c in ?? ()
#36 0xfa406024 in ?? ()
#37 0xfa405b5c in ?? ()
#38 0xfa405b5c in ?? ()
#39 0xfa405b5c in ?? ()
#40 0xfa405c6c in ?? ()
#41 0xfa405b5c in ?? ()
#42 0xfa400120 in ?? ()
#43 0xfe0d4bf4 in __1cJJavaCallsLcall_helper6FpnJJavaValue_pnMmethodHandle_pnRJavaCallArguments_pnGThread__v_ ()
from /ade_autofs/ade/OUI_MAIN_SOLARIS.rdd/030624/jre14/lib/sparc/client/libjvm.so
#44 0xfe0ecb04 in __1cRjni_invoke_static6FpnHJNIEnv__pnJJavaValue_pnI_jobject_nL
JNICallType_pnK_jmethodID_pnSJNI_ArgumentPusher_pnGThread__v_ ()
from /ade_autofs/ade/OUI_MAIN_SOLARIS.rdd/030624/jre14/lib/sparc/client/libjvm.so
#45 0xfe18985c in jni_CallStaticVoidMethod ()
from /ade_autofs/ade/OUI_MAIN_SOLARIS.rdd/030624/jre14/lib/sparc/client/libjvm.so
#46 0x12540 in main ()
================================================================================
Thread info:
14 LWP 2 0xff29eab0 in _signotifywait () from /usr/lib/libc.so.1
13 LWP 3 0xff29f160 in _lwp_sema_wait () from /usr/lib/libc.so.1
12 LWP 4 0xff29c654 in door_restart () from /usr/lib/libc.so.1
11 LWP 5 0xff29f114 in ___lwp_cond_wait () from /usr/lib/libc.so.1
10 LWP 6 0xff29f114 in ___lwp_cond_wait () from /usr/lib/libc.so.1
9 LWP 7 0xff29f114 in ___lwp_cond_wait () from /usr/lib/libc.so.1
8 LWP 8 0xff29f09c in ___lwp_mutex_lock () from /usr/lib/libc.so.1
7 LWP 9 0xff29f114 in ___lwp_cond_wait () from /usr/lib/libc.so.1
6 LWP 10 0xff29f09c in ___lwp_mutex_lock () from /usr/lib/libc.so.1
5 LWP 11 0xff29f114 in ___lwp_cond_wait () from /usr/lib/libc.so.1
4 LWP 12 0xff29f160 in _lwp_sema_wait () from /usr/lib/libc.so.1
3 LWP 15 0xff29f160 in _lwp_sema_wait () from /usr/lib/libc.so.1
2 LWP 14 0xff29f160 in _lwp_sema_wait () from /usr/lib/libc.so.1
* 1 LWP 1 0xff29f160 in _lwp_sema_wait () from /usr/lib/libc.so.1
================================================================================
(Review ID: 189812)
======================================================================