The problem I see is a hang in one of the tests that my group (DBTG) runs as part of nightly testing of the Derby (Java DB) database. The test is a multithreaded (10 load threads) stress test that is supposed to run for 10 minutes, and is run twice every night on two branches. The hang occurs now and then, 7-8 times the past month, and we have seen it on both branches of the product. It has been observed when running on the latest build of Mustang (b104) on Solaris 10 on x86 (v20s). 

We test with jdk1.4 and jdk1.5, on Solaris 8 and 9, on x86 and Sparc, but have not observed this hang in any other configuration.

Failed attempts at debugging with the jdk tools made me suspect there might be a bug in the jvm, so instead I have generated a core file using gcore, it can be found here: /net/khepri32/export/home/tmp/cores/siv08.670
The core file is from a run on siv08.norway.sun.com:
vo136787@siv08:~> uname -a
SunOS siv08 5.10 Generic_118855-14 i86pc i386 i86pc

The behaviour of the hang is that the jvm-process will be using 98-99% cpu, it can not be connected to with jconsole or jdb. Using jinfo to print for instance system properties somehow resolves the hang and the process continues to run. Connecting with jstack fails, but when using the force option I get a list of threads but no stack (this also resolves the hang):

Deadlock Detection:

No deadlocks found.

Thread t@25: (state = IN_VM)


Thread t@24: (state = IN_JAVA)


Thread t@23: (state = IN_VM)


Thread t@22: (state = IN_VM)


Thread t@21: (state = IN_VM)


Thread t@20: (state = IN_VM)


Thread t@19: (state = IN_VM)


Thread t@18: (state = IN_JAVA)


Thread t@17: (state = IN_VM)


Thread t@16: (state = IN_VM)


Thread t@15: (state = IN_VM)


Thread t@14: (state = BLOCKED)


Thread t@13: (state = BLOCKED)


Thread t@8: (state = BLOCKED)


Thread t@7: (state = BLOCKED)


Thread t@6: (state = BLOCKED)


Thread t@2: (state = BLOCKED)

Below follows some output from dbx:

vo136787@siv08:~> /opt/studio11/SUNWspro/bin/dbx /usr/local/java/jdk1.6.0_b104/bin/java /tmp/siv08.670
<snip>
t@null (l@25) terminated by signal 0 (UNKNOWN SIGNAL)
0xfef309b7: ___lwp_mutex_timedlock+0x0007:      jae      ___lwp_mutex_timedlock+0x15    [ 0xfef309c5, .+0xe ]
(dbx) where
=>[1] ___lwp_mutex_timedlock(0x80727c0), at 0xfef309b7
  [2] Mutex::lock_without_safepoint_check(0x8072768), at 0xfe893a13
  [3] SafepointSynchronize::block(0x8748400), at 0xfe8ddbce
  [4] ObjectMonitor::wait(0x816a0d8, 0xbb8, 0x0, 0x1, 0x8748400), at 0xfec94008
  [5] ObjectSynchronizer::wait(0x8748db4, 0xbb8, 0x0, 0x8748400, 0x1a1), at 0xfe94184b
  [6] JVM_MonitorWait(0x87484e8, 0xf264261c, 0xbb8, 0x0, 0xf73d8370, 0xf26425e4, 0x0, 0xf264261c, 0xf3273ac0, 0x0, 0xf32058b8, 0x0, 0xf2642620, 0xfad8c0e8, 0xfb965aec, 0x0, 0xfb4081f9, 0xbb8, 0x0, 0xfad8c0e8), at 0xfe94157c
  [7] 0xfb40a578(0x0, 0xfb4081f9, 0xbb8, 0x0, 0xfad8c0e8, 0xbb8), at 0xfb40a578
  [8] 0xfb965aec(0xfab85830, 0xf99c18e0, 0xf73d83d0, 0x0, 0x0, 0x0), at 0xfb965aec
(dbx) threads
      t@1  a  l@1   ?()   running          in  __lwp_wait()
      t@2  a  l@2   JavaMain()   running          in  ___lwp_mutex_timedlock()
      t@3  b  l@3   java_start()   running          in  ___lwp_cond_wait()
      t@4  b  l@4   java_start()   running          in  ___lwp_cond_wait()
      t@5  b  l@5   java_start()   running          in  ___lwp_cond_wait()
      t@6  b  l@6   java_start()   running          in  ___lwp_cond_wait()
      t@7  b  l@7   java_start()   running          in  ___lwp_cond_wait()
      t@8  b  l@8   java_start()   sleep on 0xfee1ffd0  in  __lwp_park()
      t@9  b  l@9   java_start()   running          in  ___lwp_cond_wait()
     t@10  b l@10   java_start()   running          in  ___lwp_cond_wait()
     t@11  b l@11   java_start()   running          in  ___lwp_cond_wait()
     t@12  b l@12   java_start()   running          in  ___lwp_mutex_timedlock()
     t@13  b l@13   java_start()   running          in  ___lwp_cond_wait()
     t@14  b l@14   java_start()   running          in  ___lwp_cond_wait()
     t@15  b l@15   java_start()   running          in  ___lwp_mutex_timedlock()
     t@16  b l@16   java_start()   running          in  inflate()
     t@17  b l@17   java_start()   running          in  ___lwp_mutex_timedlock()
     t@18  b l@18   java_start()   running          in  Stall()
     t@19  b l@19   java_start()   running          in  ___lwp_cond_wait()
     t@20  b l@20   java_start()   running          in  Stall()
     t@21  b l@21   java_start()   running          in  inflate()
     t@22  b l@22   java_start()   running          in  inflate()
     t@23  b l@23   java_start()   running          in  ___lwp_mutex_timedlock()
     t@24  b l@24   java_start()   running          in  ___lwp_mutex_timedlock()
     t@25  b l@25   java_start()   running          in  ___lwp_mutex_timedlock()