JDK-8214584 : stackwalk returns null for scalar-replaced value
  • Type: Bug
  • Component: hotspot
  • Sub-Component: compiler
  • Affected Version: 9,10,11,12,13,14
  • Priority: P4
  • Status: Open
  • Resolution: Unresolved
  • Submitted: 2018-12-01
  • Updated: 2020-02-11
The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.
Other
tbdUnresolved
Related Reports
Relates :  
JDK-8214583 - AccessController.getContext may return wrong value after JDK-8212605
Relates :  
JDK-8232391 - consolidate various kinds of HotSpot return barriers
Relates :  
JDK-8214585 - LiveStackFrameInfo.getLocals does not materialize virtual objects
Relates :  
JDK-8214763 - add compiler intrinsic to materialize objects
Relates :  
JDK-8227745 - Enable Escape Analysis for Better Performance in the Presence of JVMTI Agents
Description
compiledVFrame::locals() will return unallocated (scalar-replaced) values if JIT escape analysis has eliminted the allocation.  This will cause stackwalk clients to see an unexpected null object when LiveFrameStream::create_primitive_slot_instance() loads the value.
Comments
In this case I would think that something similar to JVMCI's materializeVirtualObjects could be a solution for LiveStackFrame. IMHO objects should be relocked too before references escape. materializeVirtualObjects doesn't do it. It is done when the owning compiled frame is replaced with equivalent interpreted frames.
11-02-2020
That would be a good question for a libraries expert, but my guess is that a clone would not be allowed, because it does not preserve object identity. I am assuming that any object rematerialized using one method (stackwalk or deoptimization) should be == and not just equals() to that same object rematerialized using the other method.
06-02-2020
Hi Dean, I agree. Now I understand the relation to JDK-8232391 better. It could be used to dispose saved clones without deoptimization of the owner frame. What about LiveStackFrame? How is it meant to be used? As a private interface it might be ok to create and return clones of virtual objects without deoptimizing the owning frame.
27-12-2019
Hi Richard. It's not just the Java LiveStackFrame code that sees the problem, it's also the C++ compiledVFrame::locals(). Always deoptimizing will hurt the performance of security checks because AccessController.getContext() does a stack walk. That's why I was trying to figure out a way to rematerialize and memoize the locals without a full deoptimization.
23-12-2019
Hi Dean et al., LiveStackFrame* is a private interface. Are there limitations on how it can be used, e.g. is it allowed to store retrieved locals to static fields making them globally reachable? I couldn't find out too much about its purpose... The retrieval of references potentially changes the escape state of the referees, so in general (without special rules) optimizations based on it need to be reverted: Let F be a Java frame for which locals (expressions, monitors) are retrieved If F is a compiled java frame, then... - F needs to be marked for deoptimization, if not globally escaping objects (ArgEscape, NoEscape) are in scope - Virtual objects need to be rematerialized, if any are in scope - Objects with eliminated locking need to be relocked Let C be a (indirect) caller frame of F If C is a compiled java frame and not escaping objects are passed as arguments (ArgEscape) at the call site, then... - Objects with eliminated locking need to be relocked as they might be among the references retrieved from F. - C needs to be marked for deoptimization even if no objects were relocked, because code paths with eliminated locking could be reached. Rematerialization and relocking must be transactional and it must happen at most once per compiled frame. This implies that rematerialized objects must be kept for future references and for deoptimization of the owner frame. The implementation proposed for JDK-8227745 provides this functionality. I've prepared a small experimental patch based on JDK-8227745 that lets the StackWalkTests attached to JDK-8214585 succeed. Would be great if you could try and look at it. http://cr.openjdk.java.net/~rrich/webrevs/2019/8214584/experiment_v1.patch Apply also http://cr.openjdk.java.net/~rrich/webrevs/2019/8227745/webrev.most.recent/ EATests.java comes with the change for JDK-8227745. The tests there cover the claimed points above. They could be adapted to show that they are necessary for correctness of LiveStackFrame* accesses too. Thanks, Richard.
04-12-2019
Recently proposed general-purpose return barriers (JDK-8232391) could be useful for cleaning up "deferred locals".
10-11-2019
Thanks Dan!
20-06-2019
[~dlong] - I have moved the entries that I added from this bug to JDK-8192647. I have also updated the links in Mach5 that I added to refer to JDK-8192647.
20-06-2019
[~dcubed][~mikael] I believe the tests showing OutOfMemoryError: Java heap space: failed reallocation of scalar replaced objects are a different problem than the compiledVFrame::locals problem I intended when I filed this. This message just means that the OOM was thrown at deoptimization reallocation time, but I don't see anything related to locals from a stack walk. JDK-8192647 might be more appropriate for these failures.
19-06-2019
Thanks for the details, Dean. Sounds like there's not simple fix for this..
07-12-2018
> Just deoptimizing the compiled frame when we try to access a scalar-replaced local sounds reasonable. Why do you think a failing re-allocation is a problem? The same unexpected OOME ("Java heap space: failed reallocation of scalar replaced objects") can happen at all the places where a deoptimization is triggered for whatever reason. The problem is the fix for JDK-6898462, which blows away frames without running finally blocks. You're right, the justification for the JDK-6898462 fix was that OOME can happen it places where the code doesn't actually do an allocation. However, this behavior does make finally blocks and exception handlers a lot less useful than they could be. Deoptimizing is also not ideal if we want to keep the code compiled for performance reasons.
07-12-2018
> I don't think there is a solution for the missing object identity problem if we decide to materialize without a deoptimization. The caller could walk the stack frame of the same method multiple times and compare the same local which would then compare different materializations of the same scalar-replaced object and thus return false, right? We could make sure that once materialized, we always return the same materialized value, by storing it in the thread's "deferred locals" data, however without deoptimizing, compiled code may never see the materialized value. For example, if the code had a loop: while (true) { if (virtual_obj == static_volatile_obj) { // report match [...] } [...] } we could materialize "virtual_obj" and assign the value to static_volatile_obj. The interpreter would see the change if we deoptimized, but the compiled code could have optimized out the "if" because it considered a match impossible.
07-12-2018
I don't think there is a solution for the missing object identity problem if we decide to materialize without a deoptimization. The caller could walk the stack frame of the same method multiple times and compare the same local which would then compare different materializations of the same scalar-replaced object and thus return false, right? Just deoptimizing the compiled frame when we try to access a scalar-replaced local sounds reasonable. Why do you think a failing re-allocation is a problem? The same unexpected OOME ("Java heap space: failed reallocation of scalar replaced objects") can happen at all the places where a deoptimization is triggered for whatever reason. ILW = Stackwalk returns unexpected value for local, with scalar replacement of non-escaping object in C2 or Graal, disable EA (-XX:-DoEscapeAnalysis) or scalar replacement (-XX:-EliminateAllocations) = MLM = P4 Dean, I'm assigning this to you for now for tracking purposes.
06-12-2018
If we materialize and deoptimize, then at least any == after the deopt happens will work in the interpreter. That doesn't help any == compares that already failed, however.
05-12-2018
Materializing may not be enough if the caller expects object identity (==) to work, because the JIT compiler can assume that a non-escaping object != an escaping object.
04-12-2018
It might be nice to materialize virtual objects without deoptimizing, by storing the allocated values in the "deferred locals", but currently those only get cleaned up at deopt time. We could introduce a kind of "null deopt" that cleans up deferred locals and then returns to the original caller without deoptimizing.
01-12-2018
We could detect this situation and force a deoptimization, but that could throw an exception and blow away inlined frames if the allocation fails. Alternatively, we could throw an exception (caller might not be prepared for that) or force the caller to check for scalar-replaced objects. The LiveStackFrameInfo class would probably need to wrap these values instead of returning null.
01-12-2018