JDK-8233915 : JVMTI FollowReferences: Java Heap Leak not found because of C2 Scalar Replacement
  • Type: Bug
  • Component: hotspot
  • Sub-Component: compiler
  • Affected Version: 11,12,13,14
  • Priority: P3
  • Status: Resolved
  • Resolution: Fixed
  • Submitted: 2019-11-11
  • Updated: 2022-05-21
  • Resolved: 2020-10-20
The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.
JDK 16
16 b21Fixed
Related Reports
Duplicate :  
Duplicate :  
Relates :  
Relates :  
Description
The attached test (v4) shows that C2 scalar replacement (-XX:+EliminateAllocations) can prevent JVMTI
agents from finding a memory leak using JVMTI FollowReferences().

The required JVMTI capability can_tag_objects is an 'always' capability, meaning an agent can
request it during the live phase. This allows for loading an agent dynamically into the vm to do heap
diagnostics. Therefore it is not sufficient to disable scalar replacement when can_tag_objects is
taken by an agent. The test shows this also.

This bug report duplicates JDK-8230956, but it is supposed to be less theoretical giving an example
that is close to real world scenarios and it shows, that just disabling escape analysis is not
sufficient.

Steps performed by the test:

// Please use v4. Previous versions are obsolete.

- Warmup: call dontinline_leakHoldingMethodWithScalarReplacement(long leakSize) often with small
  leak size to get it compiled by C2.

- After warmup call dontinline_leakHoldingMethodWithScalarReplacement(long leakSize) again with
  Long.MAX_VALUE as target leak size.

- In that call create and inflate a new leak, linking new LeakObject instances into a list which is
  rooted at an object referenced by the local variable 'holder'.

- This 'holder' object is allocated at the entry of dontinline_leakHoldingMethodWithScalarReplacement(long leakSize).
  C2 eliminated this allocation based on proof by escape analysis that the created object is local
  to the compiled method. The scalar field values are used directely be the compiled code (scalar replacement).

- Continue inflating until until {@link OutOfMemoryError} is thrown.

- Catch the OOM in {@link #dontinline_allocateNewLeakObj()} and dynamically load the agent to create
  a class histogram of reachable heap objects.  This can be done, because the capability
  can_tag_objects required for JVMTI FollowReferences() can be added during the live phase.

- Call into the agent which uses JVMTI FollowReferences() to sum up used bytes per class of
  reachable objects. The implementation of FollowReferences() misses scalar replaced objects and
  consequently all leaked objects too.

- Control returns to dontinline_leakHoldingMethodWithScalarReplacement(long leakSize) and an uncommon trap is hit.

- The compiled frame is replaced with interpreter frames. Scalar replaced objects are reallocated on the heap.

- Capture 2nd class histogram. This one shows the leak, because the holder was reallocated on the heap.

- Store the root object 'holder' into a static variable. Because of this store the holder object was and is
  "definitively reachable" in the sense of JLS 12.6 Finalization of Class Instances.

- Capture 3nd class histogram. It contains the leak as well.

- Delete reference to the leak.

- Capture 4th class histogram. It shows that the leak became unreachable.

Comments
Changeset: 40f847e2 Author: Richard Reingruber <rrich@openjdk.org> Date: 2020-10-20 15:31:55 +0000 URL: https://git.openjdk.java.net/jdk/commit/40f847e2
20-10-2020

RFR thread - http://mail.openjdk.java.net/pipermail/hotspot-compiler-dev/2019-November/035851.html
26-11-2019

Uploaded v4 of the test. It contains small adjustments for ZGC.
13-11-2019

Attached v2 of the test. It fixes percentage output. Pls ignore the version before.
12-11-2019

Potential fix: JDK-8227745
11-11-2019