JDK-8176204 : [DOC] ThreadMXBean Fails to Detect ReentrantReadWriteLock Deadlock
  • Type: Bug
  • Component: core-svc
  • Sub-Component: java.lang.management
  • Affected Version: 8,9
  • Priority: P3
  • Status: Resolved
  • Resolution: Fixed
  • OS: generic
  • CPU: generic
  • Submitted: 2017-03-02
  • Updated: 2017-05-17
  • Resolved: 2017-04-10
The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.
JDK 10 JDK 9
10Fixed 9 b165Fixed
Related Reports
Relates :  
Description
FULL PRODUCT VERSION :
java version "1.8.0_111"
Java(TM) SE Runtime Environment (build 1.8.0_111-b14)
Java HotSpot(TM) 64-Bit Server VM (build 25.111-b14, mixed mode)

ADDITIONAL OS VERSION INFORMATION :
Microsoft Windows [Version 6.1.7601]

A DESCRIPTION OF THE PROBLEM :
ThreadMXBean.findDeadlockedThreads() fails to detect a deadlock involving ReentrantReadWriteLock.  Here are the steps to reproduce the deadlock.

ReentrantReadWriteLock lock1, lock2;

- Thread A: lock1.readLock().lock();
- Thread B: lock2.readLock().lock();
- Thread C: lock1.writeLock().lock();
- Thread D: lock2.writeLock().lock();
- Thread A: lock2.readLock().lock();
- Thread B: lock1.readLock().lock();

The first 2 operations succeed.  The rest of the operations cause the threads to block.

ThreadMXBean doesn't account for Thread C and D causing Thread A and B to block.  Hence, ThreadMXBean doesn't report a deadlock.


STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
Fix compiler errors in the source code by adding imports and "throws" statements.  Run the code.

EXPECTED VERSUS ACTUAL BEHAVIOR :
EXPECTED -
bean.findDeadlockedThreads() should return a long[] of the threads involved in the deadlock.  "Deadlock detected" should be printed to the screen.
ACTUAL -
Nothing is printed to the screen.

REPRODUCIBILITY :
This bug can be reproduced always.

---------- BEGIN SOURCE ----------
public class Deadlock
{
   public static void main(String arg[])
   {
      ReentrantReadWriteLock lock1, lock2;
      ThreadMXBean bean;

      lock1 = new ReentrantReadWriteLock();
      lock2 = new ReentrantReadWriteLock();

      new Thread(() -> deadlocker(lock1, lock2)).start();
      new Thread(() -> deadlocker(lock2, lock1)).start();
      new Thread(() -> lockUnlock(lock1.writeLock())).start();
      new Thread(() -> lockUnlock(lock2.writeLock())).start();

      bean = ManagementFactory.getThreadMXBean();

      while (bean.findDeadlockedThreads() == null)
         Thread.sleep(1000);

      System.out.println("Deadlock detected");
   }

   private static void deadlocker(ReentrantReadWriteLock lock1, ReentrantReadWriteLock lock2)               // TODO - Remove this
   {
      lock1.readLock().lock();
      lockUnlock(lock2.readLock());
   }

   private static void lockUnlock(Lock lock)
   {
      while (true)
      {
         lock.lock();
         lock.unlock();
      }
   }
}
---------- END SOURCE ----------

CUSTOMER SUBMITTED WORKAROUND :
None found yet.  I just figured out how to reproduce the deadlock and not have it detected by ThreadMXBean.  I now need to write my own deadlock detector so I can figure out how to resolve my deadlock issue.


Comments
Documentation fixes are very important and do not fit neatly into ILW or P1-P5. The current documentation is incorrect and can mislead the user to expect something that is not provided and then to file bug reports. So the sooner documentation is fixed the better - it is very much a quality issue. As a doc issue this is not subject to the RDP2 approval process and any priority doc issue can be fixed during RDP2. "P3-P5 bugs whose fixes only affect tests, documentation, or demos may be fixed until the first GA candidate build, on 2017/6/22. Please make sure that such bugs have a "noreg-self", "noreg-doc", or "noreg-demo" label, as appropriate."
28-03-2017

Why is this simple doc fix (not a spec change!) deferred to 10?
28-03-2017

David's rewording is progress. It seems hard to come up with clear and precise wording since read-lock and write-lock are "part of the same object".
14-03-2017

Transferring back to core-svc->java.lang.management as this is a documentation error in: jdk/src/java.management/share/classes/java/lang/management/LockInfo.java Suggested fix: diff -r e559d0c0985a src/java.management/share/classes/java/lang/management/LockInfo.java --- a/src/java.management/share/classes/java/lang/management/LockInfo.java +++ b/src/java.management/share/classes/java/lang/management/LockInfo.java @@ -38,8 +38,8 @@ * a synchronizer that may be exclusively owned by a thread and uses * {@link AbstractOwnableSynchronizer AbstractOwnableSynchronizer} * (or its subclass) to implement its synchronization property. - * {@link ReentrantLock ReentrantLock} and - * {@link ReentrantReadWriteLock ReentrantReadWriteLock} are + * {@link ReentrantLock ReentrantLock} and the write-lock (but not + * the read-lock) of {@link ReentrantReadWriteLock ReentrantReadWriteLock} are * two examples of ownable synchronizers provided by the platform. * * <h3><a name="MappedType">MXBean Mapping</a></h3> This makes it clear that only the write-lock of ReentrantReadWriteLock is amenable to deadlock detection, not the read-lock.
14-03-2017

If the test is changed to only use writeLocks - which are "ownable synchronizers" then deadlock detection works as expected: Found one Java-level deadlock: ============================= "Thread-3": waiting for ownable synchronizer 0xd1d6bed8, (a java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync), which is held by "Thread-1" "Thread-1": waiting for ownable synchronizer 0xd1d6bd08, (a java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync), which is held by "Thread-0" "Thread-0": waiting for ownable synchronizer 0xd1d6bed8, (a java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync), which is held by "Thread-1" Java stack information for the threads listed above: =================================================== "Thread-3": at sun.misc.Unsafe.park(Native Method) - parking to wait for <0xd1d6bed8> (a java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836) at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:870) at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1199) at java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.lock(ReentrantReadWriteLock.java:943) at Deadlock.lockUnlock(Deadlock.java:37) at Deadlock.lambda$main$3(Deadlock.java:16) at Deadlock$$Lambda$4/13921446.run(Unknown Source) at java.lang.Thread.run(Thread.java:744) "Thread-1": at sun.misc.Unsafe.park(Native Method) - parking to wait for <0xd1d6bd08> (a java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836) at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:870) at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1199) at java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.lock(ReentrantReadWriteLock.java:943) at Deadlock.lockUnlock(Deadlock.java:37) at Deadlock.deadlocker(Deadlock.java:30) at Deadlock.lambda$main$1(Deadlock.java:14) at Deadlock$$Lambda$2/1557092.run(Unknown Source) at java.lang.Thread.run(Thread.java:744) "Thread-0": at sun.misc.Unsafe.park(Native Method) - parking to wait for <0xd1d6bed8> (a java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836) at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:870) at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1199) at java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.lock(ReentrantReadWriteLock.java:943) at Deadlock.lockUnlock(Deadlock.java:37) at Deadlock.deadlocker(Deadlock.java:30) at Deadlock.lambda$main$0(Deadlock.java:13) at Deadlock$$Lambda$1/32404285.run(Unknown Source) at java.lang.Thread.run(Thread.java:744) Found 1 deadlock.
14-03-2017

[~dholmes], please provide correct verbiage for documentation, then transfer ownership to doc.
13-03-2017

Thanks Martin. My position is that this is not something that should be expected to work - a fact that should be much more clearly documented - and that we have no plans for doing any improvements in this area, even assuming it is feasible to detect the problematic scenario (which currently is not because reading threads are not tracked)..
13-03-2017

The entire ownable synchronizer mechanism was clearly only designed for exclusive access, so RRWL write locks (but not read locks) are covered, so the reporter's results are expected. We could certainly improve the documentation for the deadlock detection feature. As to whether to extend the current mechanism to also cover read-write deadlocks, that sounds like a fair amount of engineering work, and I don't know how valuable it would be for users. The overhead we currently have for recording read locks is already distressing. Can we implement read-write deadlock detection without making the overhead worse? (I'm not planning to work on this)
10-03-2017

Martin: what do you think about this?
10-03-2017

Also see similar issue in JDK-8157409 relating to the thread dump deadlock detection logic.
10-03-2017

The documentation for the deadlock detection capabilities is incorrect. ThreadMXBean states: long[] findDeadlockedThreads() Finds cycles of threads that are in deadlock waiting to acquire object monitors or ownable synchronizers. Where "ownable synchronizers" links to the doc for java.lang.management.LockInfo, which states: "An ownable synchronizer is a synchronizer that may be exclusively owned by a thread and uses AbstractOwnableSynchronizer (or its subclass) to implement its synchronization property. ReentrantLock and ReentrantReadWriteLock are two examples of ownable synchronizers provided by the platform. " --- But this is very misleading. Being an "ownable synchronizer" is a necessary but not suffiicent condition for deadlock detection to be applicable.
10-03-2017

Deadlock detection logic is pretty basic. Trying to determine deadlocks involving read/write locks is much more complicated.
10-03-2017

ILW = M (Fails only on ReentrantReadWriteLock), H (Always reproducible), H (no workaround) = P2
10-03-2017

Issue is reproducible both 8u121 and 9, ThreadMXBean.findDeadlockedThreads() fails to detect a deadlock 8uxx (Including 8u121) 9 ea b157 - Fail
06-03-2017