Summary
-------
Improve the scalability of Java code that uses `synchronized` methods and statements by arranging for virtual threads that block in such constructs to release their underlying [platform threads] for use by other virtual threads. This will eliminate nearly all cases of virtual threads being [pinned][jep444-pinning] to platform threads, which severely restricts the number of virtual threads available to handle an application's workload.
Goals
-----
- Enable existing Java libraries to scale well with virtual threads without having to change them not to use `synchronized` methods and statements.
- Improve the diagnostics that identify the remaining situations in which virtual threads fail to release platform threads.
Motivation
----------
Virtual threads, which were introduced in [Java 21] via [JEP 444], are lightweight threads that are provided by the JDK rather than the operating system (OS). Virtual threads significantly reduce the effort of developing, maintaining, and observing high-throughput concurrent applications by enabling applications to use huge numbers of threads. The basic model of virtual threads is as follows:
- To do useful work, a thread must be *scheduled*, that is, assigned for execution on a processor core. For platform threads, which are implemented as OS threads, the JDK relies on the scheduler in the OS. For virtual threads, by contrast, the JDK has its own scheduler. Rather than assign virtual threads to processor cores directly, the JDK's scheduler assigns virtual threads to platform threads, which are then scheduled by the OS as usual.
- To run code in a virtual thread, the JDK's scheduler assigns the virtual thread for execution on a platform thread by *mounting* the virtual thread on the platform thread. This makes the platform thread become the *carrier* of the virtual thread. Later, after running some code, the virtual thread can *unmount* from its carrier. At that point the platform thread is released so that the JDK's scheduler can mount a different virtual thread on it, thereby making it a carrier again.
- A virtual thread unmounts when performing a blocking operation such as I/O. Later, when the blocking operation is ready to complete because, e.g., bytes were received on a socket, the operation submits the virtual thread back to the JDK's scheduler. The scheduler mounts the virtual thread on a platform thread to resume running code.
Virtual threads are mounted and unmounted frequently and transparently, without blocking any platform threads.
### Virtual threads are pinned in `synchronized` methods
Unfortunately, a virtual thread cannot unmount when it runs code inside a `synchronized` method. Consider the following `synchronized` method, which reads bytes from a socket:
```
synchronized byte[] getData() {
byte[] buf = ...;
int nread = socket.getInputStream().read(buf); // Can block here
...
}
```
If the `read` method blocks because there are no bytes available, we would like the virtual thread that is running `getData` to unmount from its carrier. This would release a platform thread so that the JDK's scheduler can mount a different virtual thread on it. Unfortunately, because `getData` is `synchronized`, the JVM *pins* the virtual thread that is running `getData` to its carrier. Pinning prevents the virtual thread from unmounting. Consequently, the `read` method blocks not only the virtual thread but also its carrier, and hence the underlying OS thread, until bytes are available to read.
### The reason for pinning
The `synchronized` keyword in the Java programming language is defined in terms of [*monitors*](https://en.wikipedia.org/wiki/Monitor_(synchronization))*:* Every object is associated with a monitor that can be acquired (i.e., locked), held for a time, and then released (i.e., unlocked). Only one thread at a time may hold an object's monitor. For a thread to run a `synchronized` instance method, the thread first acquires the monitor associated with the instance; when the method is finished, the thread releases the monitor.
To implement the `synchronized` keyword, the JVM tracks which thread currently holds an object's monitor. Unfortunately, it tracks which platform thread holds the monitor, not which virtual thread. When a virtual thread runs a `synchronized` instance method and acquires the monitor associated with the instance, the JVM records the virtual thread's carrier platform thread as holding the monitor — not the virtual thread itself.
If a virtual thread were to unmount inside a `synchronized` instance method, the JDK's scheduler would soon mount some other virtual thread on the now-free platform thread. That other virtual thread, because of its carrier, would be viewed by the JVM as holding the monitor associated with the instance. Code running in that thread would be able to call other `synchronized` methods on the instance, or release the monitor associated with the instance. Mutual exclusion would be lost. Accordingly, the JVM actively prevents a virtual thread from unmounting inside a ` synchronized` method.
### More pinning
If a virtual thread invokes a `synchronized` instance method and the monitor associated with the instance is held by another thread, then the virtual thread must block since only one thread at a time may hold the monitor. We would like the virtual thread to unmount from its carrier and release that platform thread to the JDK scheduler. Unfortunately, if the monitor is already held by another thread then the virtual thread blocks in the JVM until the carrier acquires the monitor.
Moreover, when a virtual thread is inside a `synchronized` instance method and it invokes [`Object.wait()`](https://docs.oracle.com/en/java/javase/23/docs/api/java.base/java/lang/Object.html#wait()) on the object, then the virtual thread blocks in the JVM until awakened with [`Object.notify()`](https://docs.oracle.com/en/java/javase/23/docs/api/java.base/java/lang/Object.html#notify()) and the carrier re-acquires the monitor. The virtual thread is pinned because it is executing inside a `synchronized` method, and further pinned because its carrier is blocked in the JVM.
The foregoing discussion applies, with appropriate changes, to `synchronized` `static` methods, which synchronize on the monitor associated with the `Class` object for the method's class, and to `synchronized` statements, which synchronize on the monitor associated with a specified object.
### Overcoming pinning
Frequent pinning for long durations can harm scalability. It can lead to starvation or even deadlock, when no virtual threads can run because all of the platform threads available to the JDK's scheduler are either pinned by virtual threads or blocked in the JVM. To avoid these problems, the maintainers of many libraries have modified their code to use [`java.util.concurrent` locks][jucl] — which do not pin virtual threads — instead of `synchronized` methods and statements.
It should not, however, be necessary to abandon `synchronized` methods and statements in order to enjoy the scalability benefits of virtual threads. The JVM's implementation of the `synchronized` keyword should allow a virtual thread to unmount when inside a `synchronized` method or statement, or when blocked on a monitor. This would enable the broader adoption of virtual threads.
Description
-----------
We will change the JVM's implementation of the `synchronized` keyword so that virtual threads can acquire, hold, and release monitors, independently of their carriers. The mounting and unmounting operations will do the bookkeeping necessary to allow a virtual thread to unmount and re-mount when inside a `synchronized` method or statement, or when waiting on a monitor.
Blocking to acquire a monitor will unmount a virtual thread and release its carrier to the JDK's scheduler. When the monitor is released, and the JVM selects the virtual thread to continue, the JVM will submit the virtual thread to the scheduler. The scheduler will mount the virtual thread, perhaps on a different carrier, to resume execution and try again to acquire the monitor.
The `Object.wait()` method, and its timed-wait variants, will similarly unmount a virtual thread when waiting and blocking to re-acquire a monitor. When awakened with `Object.notify()`, and the monitor is released, and the JVM selects the virtual thread to continue, the JVM will submit the virtual thread to the scheduler to resume execution.
### Diagnosing remaining cases of pinning
A [`jdk.VirtualThreadPinned`](https://openjdk.org/jeps/444#JDK-Flight-Recorder-JFR) event is recorded by JDK Flight Recorder (JFR) whenever a virtual thread blocks inside a `synchronized` method. This event has been useful to identify code that would benefit from being changed to make less use of `synchronized` methods and statements, to not block while inside such constructs, or to replace such constructs with `java.util.concurrent` locks.
This JFR event will no longer be needed for that purpose once the `synchronized` keyword no longer pins virtual threads, but we will retain it for other pinning situations. In particular, if a virtual thread calls native code, either through a `native` method or the [Foreign Function & Memory API][ffm], and that native code calls back to Java code that performs a blocking operation or blocks on a monitor, then the virtual thread will be pinned. We will therefore change the JVM to issue a `jdk.VirtualThreadPinned` event in these cases, and we will enhance the event itself to convey both the reason why the virtual thread is pinned and the identity of the carrier thread.
### The system property `jdk.tracePinnedThreads` is no longer needed
The system property `jdk.tracePinnedThreads`, introduced by [JEP 444], causes a stack trace to be printed whenever a virtual thread blocks inside a `synchronized` method, though not when a virtual thread blocks to acquire a monitor or wait in `Object.wait()`.
This system property will no longer be needed once the `synchronized` keyword no longer pins virtual threads. It has, in addition, proved to be [problematic](https://bugs.openjdk.org/browse/JDK-8322846) since the stack traces are printed while executing critical code. We will therefore remove this system property; setting it on the command line will have no effect.
### Choosing between `synchronized` and `java.util.concurrent.locks`
Once the `synchronized` keyword no longer pins virtual threads, you can choose between `synchronized` and the APIs in the [`java.util.concurrent.locks`][jucl] package based solely upon which best solves the problem at hand.
As background, the `java.util.concurrent.locks` package defines APIs for locking and waiting that are distinct from, and more flexible than, the built-in `synchronized` keyword. The [`ReentrantLock`] API behaves the same as `synchronized`. The [`Condition`] API is the equivalent of the `Object.wait()` and `Object.notify()` methods. Other APIs in the package provide greater power and finer control for advanced cases that require fairness, concurrent access to shared data with read-write locks, timed or interruptible lock acquisition, or optimistic reading.
The flexibility of the `java.util.concurrent.locks` APIs comes at the expense of more awkward syntax. The APIs should generally be used with the `try-finally` construct in order to ensure that locks are released appropriately; this is, of course, not necessary with `synchronized`. The `java.util.concurrent.locks` APIs also have different performance characteristics than `synchronized` methods or statements.
We [previously recommended][jep444-pinning] solving frequent and long-lived pinning problems by migrating code from using `synchronized` to using `ReentrantLock`. Once the `synchronized` keyword no longer pins virtual threads, such migration will no longer be necessary. You need not revert code that has been migrated to use `ReentrantLock` back to using `synchronized`.
If you are writing new code, we agree with the recommendation in [_Java Concurrency in Practice_](https://jcip.net/) §13.4: Use `synchronized` where practical, since it is more convenient and less error prone, and use `ReentrantLock` and the other APIs in `java.util.concurrent.locks` when more flexibility is required. Either way, reduce the potential for contention by narrowing the scope of locks and avoid, where possible, doing I/O or other blocking operations while holding locks.
Future Work
------------
There are a few remaining cases, unrelated to the `synchronized` keyword, in which a virtual thread cannot unmount when blocking:
- When resolving a symbolic reference ([JVMS §5.4.3](https://docs.oracle.com/javase/specs/jvms/se23/html/jvms-5.html#jvms-5.4.3)) to a class or interface and the virtual thread blocks while loading a class. This is a case where the virtual thread pins the carrier due to a native frame on the stack.
- When blocking inside a class initializer. This is also a case where the virtual thread pins the carrier due to a native frame on the stack.
- When waiting for a class to be initialized by another thread ([JVMS §5.5](https://docs.oracle.com/javase/specs/jvms/se23/html/jvms-5.html#jvms-5.5)). This is a special case where the virtual thread blocks in the JVM, thus pinning the carrier.
These cases should rarely cause issues but we will revisit them if they prove to be problematic.
Alternatives
------------
- Compensate for pinning by temporarily expanding the parallelism of the virtual-thread scheduler. The scheduler already does this for `Object.wait()`, by ensuring that a spare platform thread is available while a virtual thread is waiting.
Increasing parallelism would help with some cases, but it does not scale. The maximum number of platform threads available to the scheduler is limited, with a default limit of 256 threads. If many virtual threads were to block inside a `synchronized` method then no value of parallelism would help.
- Rewrite the bytecode of each class, as the JVM loads it, to replace each use of `synchronized` with an equivalent use of `ReentrantLock`.
The `synchronized` statement can be used with any object, so this would require maintaining a mapping from objects to locks, a significant overhead.
There are cases where the transformation would not be fully transparent, in particular for `synchronized` methods, since [JVMS §2.11.10](https://docs.oracle.com/javase/specs/jvms/se23/html/jvms-2.html#jvms-2.11.10) requires acquiring the monitor before invoking the method.
There are also many challenges with this approach in areas such as JNI locking, several features of JVM TI, and the JVMS requiring that a monitor be automatically released in all cases. This approach would also require the re-implementation of many serviceability features.
Risks and Assumptions
------------
The performance of some code may be different when virtual threads are used in place of platform threads. When a thread exits a monitor it may have to queue a virtual thread to the scheduler. This is currently not as efficient as the case where exiting a monitor unparks a platform thread.
Dependencies
------------
The changes we propose here depend upon a [change to the specification](https://bugs.openjdk.org/browse/JDK-8331422) of the JVM TI function [`GetObjectMonitorUsage`](https://docs.oracle.com/en/java/javase/23/docs/specs/jvmti.html#GetObjectMonitorUsage) in [Java 23]. This function no longer supports returning information about monitors owned by virtual threads. Doing so would have required significant bookkeeping to find the monitors owned by unmounted virtual threads.
[JEP 444]: https://openjdk.org/jeps/444
[jep444-pinning]: https://openjdk.org/jeps/444#Pinning
[ffm]: https://openjdk.org/jeps/454
[Java 21]: https://openjdk.org/projects/jdk/21/
[Java 23]: https://openjdk.org/projects/jdk/23/
[jucl]: https://docs.oracle.com/en/java/javase/23/docs/api/java.base/java/util/concurrent/locks/package-summary.html
[`ReentrantLock`]: https://docs.oracle.com/en/java/javase/23/docs/api/java.base/java/util/concurrent/locks/ReentrantLock.html
[`Condition`]: https://docs.oracle.com/en/java/javase/23/docs/api/java.base/java/util/concurrent/locks/Condition.html
[platform threads]: https://docs.oracle.com/en/java/javase/23/docs/api/java.base/java/lang/Thread.html#platform-threads