JDK-8260009 : InstanceKlass::has_as_permitted_subclass() fails if subclass was redefined
  • Type: Bug
  • Component: hotspot
  • Sub-Component: runtime
  • Affected Version: 16
  • Priority: P3
  • Status: Resolved
  • Resolution: Fixed
  • Submitted: 2021-01-19
  • Updated: 2021-01-28
  • Resolved: 2021-01-22
The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.
JDK 16 JDK 17
16.0.1Fixed 17 b07Fixed
Description
From: MRasmussen@perforce.com:

Retransforming or redefining a sealed class can lead to a JVMTI_ERROR_FAILS_VERIFICATION, with Exception <a 'java/lang/IncompatibleClassChangeError'{0x00000000feb820a0}: class autotest.TransparentRectangle cannot inherit from sealed class autotest.Rectangle> (0x00000000feb820a0), even though the class actually does that.

I've tried to investigate the reason, and from what I can tell, it boils down to the check in InstanceKlass::has_as_permitted_subclass.

In jdk-16+27, this method always seems to take the Symbol branch (_constants->tag_at(cp_index).is_klass() is false).
But for jdk-16+28, it takes the Klass branch after the classes are loaded, and from what I can tell the (k2 == k) comparison can then fail during the redefinition since the Klass stored in the _permitted_subclasses is not the same as the one being verified, as a new InstanceKlass is allocated for the stream parser.

Unfortunately, I haven't been able to produce a small self-contained test example, but I can reliably reproduce it while developing JRebel (which utilizes class retransformation), but hopefully the above is at least helpful in finding the issue and fixing it.

/Michael
Comments
Fix Request This bug should be back-ported to JDK-16u because it causes class redefinition of a permitted subclass to fail in certain situations. The fix is small and low risk and includes a test case. The patch applied cleanly.
22-01-2021

Changeset: f928265e Author: Harold Seigel <hseigel@openjdk.org> Date: 2021-01-22 13:52:05 +0000 URL: https://git.openjdk.java.net/jdk/commit/f928265e
22-01-2021

Moving from hotspot/runtime to hotspot/jvmti. Update: Harold thinks this belongs in hotspot/runtime so he's moved it back.
21-01-2021

Discussed with lfoltan and coleenp: The problem occurs if the subclass is resolved by the super class and the subclass is being redefined. During redefinition of the subclass, a scratch InstanceKlass gets created when reloading the redefined class. Checking if the scratch ik is in its super's permitted_subclass list fails, because the super's permitted_subclass contains the real InstanceKlass for the subclass, not the scratch ik.
20-01-2021