FULL PRODUCT VERSION :
java version "1.6.0-rc"
Java(TM) SE Runtime Environment (build 1.6.0-rc-b96)
Java HotSpot(TM) 64-Bit Server VM (build 1.6.0-rc-b96, mixed mode)

FULL OS VERSION :
Linux XXXX 2.6.17-1.2142_FC4smp #1 SMP Tue Jul 11 22:59:20 EDT 2006 x86_64 x86_64 x86_64 GNU/Linux

EXTRA RELEVANT SYSTEM CONFIGURATION :
It is an SMP system with 2 processors

A DESCRIPTION OF THE PROBLEM :
When one of the threads has a lower priority than other threads (set using Thread.setPriority()) in a JVM, and they all share a synchronized statement, then we could find a temporarily deadlocked system due to priority inversion.

I had a piece of code in which almost all the threads would stop making progress at random times for a few seconds. During this time, the JVM would be eating up all the CPU on the machine. I wrote a script to send it SIGSTOP signal when the CPU gets maxed out and used gdb to look into the suspended process. Here is what I found:

Many threads in the JVM are running the code (in the JDK source) in hotspot/src/share/vm/runtime/synchronizer.cpp ObjectSynchronizer::inflate().
One of the threads is running in
hotspot/src/share/vm/runtime/synchronizer.cpp ObjectSynchronizer::omAlloc().

The thread running in omAlloc() has a lower priority. All the other threads with higher
priority are spinning in the following code in inflate() waiting for lower priority thread to
finish its job and unset the INFLATING flag:

 if (mark == markOopDesc::INFLATING()) {
         Stall() ;
         TEVENT (Inflate: spin while INFLATING) ;
         continue ;
}

It takes a while to linux to schedule the lower priority thread while higher priority
threads a running and hence the appearance of the program making no progress.

If the higher priority threads had realtime priorities (SCHED_RR or SCHED_FIFO in linux)
then this would result in a permanent deadlock.

The fundamental cause seems to be the code which simply spins. It might be better
to spin a certain number of times, and then block. Otherwise using java in programs
with threads with different priorities seems risky.


THE PROBLEM WAS REPRODUCIBLE WITH -Xint FLAG: Did not try

THE PROBLEM WAS REPRODUCIBLE WITH -server FLAG: Yes

STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
I have a piece of code which allowed me to figure out the problem. But it is very likely
that this code will work only on a machine with the exact same specs as my machine (CPU speed etc.).

The problem is pretty obvious looking at the JDK source code in
hotspot/src/share/vm/runtime/synchronizer.cpp

EXPECTED VERSUS ACTUAL BEHAVIOR :
  Program freezes temporarily while consuming all the CPU.
REPRODUCIBILITY :
This bug can be reproduced often.

CUSTOMER SUBMITTED WORKAROUND :
Don't use different thread priorities.