Bug ID: JDK-8270085 Suspend during block transition may deadlock if lock held

Versions (Unresolved/Resolved/Fixed)

The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.

JDK 17	JDK 18
17 b33Fixed	18Fixed

Some locks allow a JavaThread to enter a blocked state.
If the JavaThread owning the lock is suspended on the back edge when returning to the VM state,
it may cause a deadlock.

An example of this is a JavaThread having the Compile_lock and then grabbing the VMOperation_lock,
the JavaThread may transition to a blocked state due to contentions or if the JavaThread waits on the
VMOperation_lock.

The resumer might call a path which needs the Compile_lock before it resumes the suspended JavaThread.
But since the Compile_lock will not be unlocked until the suspended JavaThread is resumed, we deadlock.

Changeset: e7f90093 Author: Patricio Chilano Mateo <pchilanomate@openjdk.org> Date: 2021-07-22 14:30:19 +0000 URL: https://git.openjdk.java.net/jdk17/commit/e7f9009315f1fdf33d6539c413c224f925c9ffb0
22-07-2021
Fix request for JDK 17 is approved.
21-07-2021
Copied here from JDK-8260062: rehn Robbin Ehn added a comment - 6 days ago We are calling deopt all marked with the Compile_lock held, thus asking the VM thread to handshake all threads. While we wait for that in blocked state, the JavaThread gets suspended.
08-07-2021
Copied here from JDK-8260062: David Holmes added a comment - 6 days ago > It seem like one JavaThread is suspended while holding Compile_lock. Just to state the obvious we should never allow a thread to suspend whilst holding any internal VM resources. Is this a case of nested locks?
08-07-2021
Copied here from JDK-8260062: Robbin Ehn added a comment - 1 week ago We had one on Linux and got some data. It seem like one JavaThread is suspended while holding Compile_lock. The JavaThread that is suppose to resume the suspended thread is waiting for Compile_lock. We use to avoid the suspend flag when in block_trans when leaving safepoint. Since the Windows timeouts are older than suspend changes, I believe those were infra problems. Linux one is the issue with suspend while holding Compile_lock. Note: The comment about Windows is for the failures in JDK-8260062.
08-07-2021
I think we are now checking for suspension in too many places. Suspension checks used to only occur in relation to acquisition of Java monitors, when returning to Java, or when entering the VM from native (IIRC). Checking for suspension on all thread-blocked situations is wrong - we must never suspend whilst holding a VM lock.
08-07-2021
The added test (SuspendBlocked.java) timeouts as expected on jdk master.
08-07-2021
WIP/Testing: https://github.com/robehn/jdk/tree/suspend-block-trans
08-07-2021

Duplicate :	JDK-8260062 - runtime/handshake/HandshakeDirectTest.java timed out
Relates :	JDK-8260062 - runtime/handshake/HandshakeDirectTest.java timed out
Relates :	JDK-8271251 - JavaThread::java_suspend() fails with "fatal error: Illegal threadstate encountered: 6"