JDK-8247972 : incorrect implementation of JVM TI GetObjectMonitorUsage
  • Type: Bug
  • Component: hotspot
  • Sub-Component: jvmti
  • Affected Version: 6,7,8,11,15,16
  • Priority: P4
  • Status: Resolved
  • Resolution: Fixed
  • Submitted: 2020-06-22
  • Updated: 2024-03-21
  • Resolved: 2024-03-12
The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.
JDK 23
23 b14Fixed
Related Reports
CSR :  
Duplicate :  
Relates :  
Relates :  
Relates :  
Sub Tasks
JDK-8325314 :  
Description
The specification for the jvmtiMonitorUsage structures is defined as follows:

 jvmtiMonitorUsage - Object monitor usage information
 Field	Type	Description
owner	jthread	The thread owning this monitor, or NULL if unused
entry_count	jint	The number of times the owning thread has entered the monitor
waiter_count	jint	The number of threads waiting to own this monitor
waiters	jthread*	The waiter_count waiting threads
notify_waiter_count	jint	The number of threads waiting to be notified by this monitor
notify_waiters	jthread*	The notify_waiter_count threads waiting to be notified 

Looking at:
- waiter_count: The number of threads waiting to own this monitor
- notify_waiter_count: The number of threads waiting to be notified by this monitor

a reasonable developer would reasonably expect that these mean exactly what they state, in particular that "waiter_count" is the number of threads currently blocked acquiring the monitor in question. While "notify_waiter_count" would be the number of threads that have called Object.wait() on this monitor and which are still in the wait-set awaiting notification. However, due to what seems to be a historical misunderstanding that is not the case. The "waiter count" (and thus "waiters") is the number of threads blocked entering the monitor plus the number of threads in the wait-set:

    nWant = mon->contentions(); // # of threads contending for monitor
    nWait = mon->waiters();     // # of threads in Object.wait()
    ret.waiter_count = nWant + nWait;
    ret.notify_waiter_count = nWait;

ref: jvmtiEnvBase.cpp - JvmtiEnvBase::get_object_monitor_usage

The historical misunderstanding can be seen in the commentary of JDK-4546581.

---
The JVM/DI GetMonitorInfo() API returns a pointer to the following struct:

typedef struct {
    jthread owner;
    jint entry_count;
    jint waiter_count;
    jthread *waiters;
} JVMDI_monitor_info;

The owner field is pretty obvious, but the other fields are not. The
entry_count field could be the number of times the monitor is entered
by the owning thread (recursion count) or it could be the number of
threads waiting to enter (contention count). The waiter_count field
specified the number of jthread pointers in the waiters array. However,
is the waiters array the threads waiting to enter the monitor, the
threads doing an Object.wait() or both?

The resolve this mystery, I'm using the JDI spec for ObjectReference:

public int entryCount() 
    Returns the number times this object's monitor has been entered by
    the current owning thread.

public List waitingThreads()
    Returns a List containing a ThreadReference for each thread currently
    waiting for this object's monitor. See
    ThreadReference.currentContendedMonitor() for information about when
    a thread is considered to be waiting for a monitor.

ThreadReference.currentContendedMonitor() says:
    The thread can be waiting for a monitor through entry into a
    synchronized method, the synchronized statement, or Object.wait(long).

I'm planning to implement the GetMonitorInfo() API to meet the expectations of JDI's use of the interface.
---

The misunderstanding starts to creep in here because monitor contention can arise through three paths:

"The thread can be waiting for a monitor through entry into a synchronized method, the synchronized statement, or Object.wait(long)."

The reference to Object.wait is of course referring to the process by which a call to wait() first releases then monitor, then when the thread is notified (or times out or is interrupted) it must reacquire the monitor. That re-acquisition can be a source of contention.

This detail is clearly understood as per the next comment:

---
GetCurrentContendedMonitor() returns a jobject for two conditions:

- the specified thread is waiting to enter
- the specified thread is waiting to regain through java.lang.Object.wait

The first condition is clear. The second needs clarification. An
Object.wait() call has two parts: the wait for notification (or
timeout) part and the reenter part. I believe the phrase "waiting to
regain" is intended to apply to the reenter part.
---

But it then continues:

---
However, the JDI spec needs to be checked to see what is expected by the higher layers.

ThreadReference.currentContendedMonitor() says:

    Returns an ObjectReference for the monitor, if any, for which this
    thread is currently waiting. The thread can be waiting for a monitor
    through entry into a synchronized method, the synchronized statement,
    or Object.wait(long). The status() method can be used to differentiate
    between the first two cases and the third.

ThreadReference.status() says:

    Returns the thread's status. If the thread is not suspended the thread's
    current status is returned. If the thread is suspended, the thread's
    status before the suspension is returned (or THREAD_STATUS_UNKNOWN if
    this information is not available. isSuspended() can be used to determine
    if the thread has been suspended.

ThreadReference.THREAD_STATUS_WAIT says:
    Thread is waiting - Thread.wait() or JVM_MonitorWait() was called

Looks like the JDI layer is expecting that a call into Object.wait() will
result in the thread showing a contended monitor.
---

And here we have the false conclusion that leads to threads awaiting notification in Object.wait being included in the "waiters_count". It was based on the incorrect statement:

"The status() method can be used to differentiate  between the first two cases and the third."

In fact the thread status cannot make any such distinction. If THREAD_STATUS_WAIT is applied a thread in the wait-set or trying to re-acquire the monitor after being notified, then you cannot tell from that status whether we have monitor contention or not. And if THREAD_STATUS_MONITOR is applied to the case of the thread trying to re-acquire the monitor after being notified (which it should be but I haven't checked) then you cannot distinguish the cases at all.

The end result is that it was wrongly determined that the waiters_count should include the threads contending to acquire the monitor AND those waiting for notification.

This mistake has been around since the first implementation of JVM TI was introduced and so we cannot correct it after all this time. Instead we need to clarify the specification so that it matches what the implementation actually does.

That said, it is possible that outside of OpenJDK this specification has been implemented correctly and such a specification change would invalidate that implementation.
Comments
Changeset: b92440f9 Author: Serguei Spitsyn <sspitsyn@openjdk.org> Date: 2024-03-12 08:55:28 +0000 URL: https://git.openjdk.org/jdk/commit/b92440f9b1f41643bf9649ca192e405a9d6c026a
12-03-2024

Dan, thank you for the history!
13-02-2024

Serguei's quotes from JDK-4546581 show my thinking when I implemented the JVM/DI GetMonitorInfo() API more than 20 years ago. During that time frame Swamy was implementing the JVM/TI version of the API and he was planning to base it on my work on the JVM/DI GetMonitorInfo() API. When I looked though my notes archive for JDK-4546581, I found that I relied on the JDI APIs to determine what the JVM/DI APIs should do. I did not find anything to indicate that I looked at JDWP spec to see what it did. I did find notes showing the JDWP tests that were run on the new JDI API changes and the fact that those tests passed when we integrated the JVM/DI API changes.
07-02-2024

The Pull Request listed in the comment above fixes the implementation to match the JVMTI GetObjectMonitorUsage spec.
02-02-2024

A pull request was submitted for review. URL: https://git.openjdk.org/jdk/pull/17680 Date: 2024-02-02 00:44:00 +0000
02-02-2024

Moved to 22.
18-04-2023

[~dholmes] Thank you for catching this problem! > This mistake has been around since the first implementation of JVM TI was introduced and so > we cannot correct it after all this time. Instead we need to clarify the specification so that > it matches what the implementation actually does. > > That said, it is possible that outside of OpenJDK this specification has been implemented correctly and such a > specification change would invalidate that implementation. In fact, I don't like the idea to put the burden of our mistake on the other implementors and make them incorrect while they correctly implemented this API in the first place. Theoretically, it'd be better to correct the implementation to match the spec. But I agree, it is hard to estimate the incompatibility impact of such change. Need to think a little bit. Moving to 18.
21-05-2021