JDK-8218403 : Deoptimizing top frame not always sufficient to reach interpreter-only mode
  • Type: Bug
  • Component: hotspot
  • Sub-Component: compiler
  • Affected Version: 10,11,12,13
  • Priority: P3
  • Status: Open
  • Resolution: Unresolved
  • Submitted: 2019-02-04
  • Updated: 2022-06-23
The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.
Related Reports
Blocks :  
Relates :  
Some JVMTI code needs to force interpreter-only mode.  If the top frame is compiled, it will force it to be deoptimized.  Unfortunately, there are situations where this does not work.  For example, resolve stubs jump to the callee rather than returning to the deoptimized caller.
It seems we already have all the right pieces to make this work. We already know how to deoptimize in the callee and reexecute the invoke in the caller using PopFrame. So the missing logic is being able to do the equivalent without actually entering the callee.

The call transition is the biggest problem. There is also a similar problem on the return transition. We can get suspended doing a poll-return safepoint, and then JVMTI can't deoptimize, so it will return JVMTI_ERROR_OPAQUE_FRAME. So there's precedence for using JVMTI_ERROR_OPAQUE_FRAME. I agree, the before state should be recoverable. Tobias reminded me about Valhalla c2i adapters, that now have oopmaps, which gave me an idea. Instead of rebuilding the before state from the register map and calling convention, what if we execute the c2i argument shuffle first and then rebuild the state from that? Then we could have a new deoptimization entry point that knows how to handle that situation.

Is this only a problem at call sites? Calls are particular in that it should be feasible to rebuild the state before the call at the call site from the state after and the call arguments that are at known locations (defined by the calling convention).

ILW = Deoptimization of top frame continues in callee, with jvmti methods PopFrame/ForceEarlyReturn, no workaround but can disable compilation or reduce likelihood by not using Graal/-Xcomp = HLM = P3

Here's a patch to popframe009 to demonstrate the problem with C1/C2 and -Xcomp: http://cr.openjdk.java.net/~dlong/popframe009/webrev/

3. return to deoptimized caller instead of callee This looks attractive, because it makes the code behave how JVMTI expects. However, we already have oopmap and safepoint information at the callsite, but it contains the state *after* the call has returned, not before. To implement [3], we would need both before and after oopmaps for the same PC, or we would need to replace the after oopmap with a before oopmap, which would break safepoint polls on return, which would need to be changed to safepoint poll on entry. We already have stackbang on entry barriers on entry, so doing safepoint checks here might fit nicely.

Summary of possible solutions from JDK-8195635: 1. block suspend 2. have PopFrame/ForceEarlyReturn return JVMTI_ERROR_OPAQUE_FRAME 3. return to deoptimized caller instead of callee

This problem shows up especially with Graal and -Xcomp, because we block in the resolve stub waiting for a Graal compile, and when we continue in the callee we could be executing JVMCI code like HotSpotJVMCIRuntime::adjustCompilationLevel(). The same problem with C2 and -Xcomp may go unnoticed, because for tests like popframe009, the caller and callee are both fibonacci().