JDK-8181144 : JDI - VirtualMachine.allClasses() does not return loaded but uninitialized class
  • Type: Bug
  • Component: hotspot
  • Sub-Component: jvmti
  • Affected Version: 8,10,11
  • Priority: P3
  • Status: Resolved
  • Resolution: Not an Issue
  • Submitted: 2017-05-26
  • Updated: 2021-05-21
  • Resolved: 2021-05-21
The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.
JDK 17
17Resolved
Related Reports
Duplicate :  
Duplicate :  
Relates :  
Relates :  
Relates :  
Relates :  
Description
VirtualMachine.allClasses() does not return a loaded class when the class is
not initialized.  This prevents reloading of the uninitialized class in our
application.

Consider the test case:

public class Main {
 public static void foo(Foo f) {
 }

 public static void main(String[] args) throws ClassNotFoundException {
   //load class, but don't initialize it
   Class.forName("Foo", false, Main.class.getClassLoader());
   //alternative: call Main.class.getMethods();
   new Foo().bar(); // breakpoint here
 }

}

class Foo {
 static {
   System.out.println("Foo initialized");
 }

 void bar() {
   System.out.println("yy");
 }
}

When debugger stops at breakpoint, where Foo is loaded but not initialized,
VirtualMachine.allClasses() does not contain a class Foo as it should,
according to
https://docs.oracle.com/javase/8/docs/jdk/api/jpda/jdi/com/sun/jdi/VirtualMachine.html#allClasses()

jdk1.8.0_131\bin\jdb" Main
Initializing jdb ...
> stop at Main:9
Deferring breakpoint Main:9.
It will be set after the class is loaded.
> run
run Main
Set uncaught java.lang.Throwable
Set deferred uncaught java.lang.Throwable
>
VM Started: Set deferred breakpoint Main:9

Breakpoint hit: "thread=main", Main.main(), line=9 bci=12
9        new Foo().bar(); // breakpoint here

main[1] class Foo
"Foo" is not a valid id or class name.
main[1]

It's not until we initialize Foo that it is then returned by
VirtualMachine.allClasses():

main[1] step
>
Step completed: "thread=main", Foo.<clinit>(), line=16 bci=0
16        System.out.println("Foo initialized");

main[1] class Foo
Class: Foo
extends: java.lang.Object 
Comments
Closing this bug as NAI.
21-05-2021

[~dholmes] Thanks a lot for confirmation. I already have a jdb based test case which is exactly what you would like to have. I'll file a test bug on this.
18-05-2021

[~sspitsyn] Sorry for the delay. Yes I agree. I misspoke above when I stated "Now that Class.forName has reverted to not linking, this issue still needs to be fixed." If Class.forName returns a class that has not been prepared then allClasses() should not return it - which is what the original test case was observing. Hence the original test case is in fact (now) working as designed. It is worth ensuring that we have a test that loads and prepares classes (but doesn't initialize them) and checks that allClasses() does include them.
18-05-2021

(1) The JDI VirtualMachine.allClasses() spec has this: "Returns all loaded types in the target VM. The returned list includes all reference types, including hidden classes or interfaces, loaded at least to the point of preparation and types (like array) for which preparation is not defined." Please, note, this part: "loaded at least to the point of preparation". The JDI maintains a cache of loaded classes. There are no unprepared classes in the cache. The whole mechanism is based on it. The allClasses() is fist called at the JDI initialization time. The JDWP agent uses the JVMTI GetLoadedClasses and removes all unprepared classes from its result. Then the JDI classes cache is being regularly updated on the base of JVMTI ClassPrepare events. (2) The spec of Class.forName() starting from JDK 14 says: "this method attempts to locate and load the class or interface." In the JDK 13 it was: "this method attempts to locate, load, and link the class or interface." But it was changed by JDK-8233272 to remove mention about class linking to align spec with implementation. It means, the class Foo loaded with the Class.forName() is not linked, and so, is not prepared. As it is not prepared then it should not be included into the result of VirtualMachine.allClasses(). My conclusion is that the VirtualMachine.allClasses() is implemented according to its spec. So, this bug has to be closed as "Not an Issue", I think. [~dholmes] David, could you confirm, please, if you are agreed with my conclusion?
14-05-2021

David, thank you for the analysis. As you noted, the JDI classesByName can be used in this particular case instead of the allClasses. It is specified to return classes with the same state as the allClasses. It is translated to the classesForSignature (that calls JVMTI GetLoadedClasses anyway and filters status returned from JVMTI ClassStatus. It seems to me you are right that the Foo class is not returned because it has not been linked yet. But the context (code path) is not clear to me yet. Do you mean this? : java.lang.Class.forName() -> forName0() -> JVM_FindClassFromCaller() -> find_class_from_class_loader() Other calling contexts of the find_class_from_class_loader() are: JVM_FindClassFromClass() // used in java.base/share/native/libverify/check_code.c jni_FindClass() jni_GetDirectBufferAddress -> initializeDirectBufferSupport() -> lookupDirectBufferClasses() -> lookupOne() The impact on jni_FindClass() can be potentially the most problematic. Do you think, it make sense to temporarily move this issue to the core-libs for initial evaluation?
11-05-2021

Moved to 15.
26-11-2019

Okay, thanks. I misunderstood the intention of JDK-8233272.
19-11-2019

JDK-8233272 is not re-implementing anything, it is updating the spec to match the old, and now restored, behaviour of Class.forName. Now that Class.forName has reverted to not linking this issue still needs to be fixed.
14-11-2019

Then I'm closing this one as dup of new bug JDK-8233272 which has to re-implement the backed out JDK-8212117.
14-11-2019

The fix for JDK-8212117 introduced too much incompatibility, and had to be backed out (by JDK-8233091). This issue will need to be resolved in some other way.
04-11-2019

It seems the suggested fix for the bug JDK-8212117 fixes this bug as well. Below is the jdb log with the patch http://cr.openjdk.java.net/%7Ebchristi/8212117/webrev05-notWorking/ suggested for JDK-8212117: jdb Main Initializing jdb ... > stop at Main:9 Deferring breakpoint Main:9. It will be set after the class is loaded. > run run Main Set uncaught java.lang.Throwable Set deferred uncaught java.lang.Throwable > VM Started: Set deferred breakpoint Main:9 Breakpoint hit: "thread=main", Main.main(), line=9 bci=12 9 new Foo().bar(); // breakpoint here main[1] class Foo Class: Foo extends: java.lang.Object main[1] As we can see, this output is not printed anymore: "Foo" is not a valid id or class name.
17-07-2019

I don't mind if you close it. The only thing that I'm not 100% sure about is where JDI starts out with the list of loaded classes and then uses ClassPrepare events to keep that view up to date when classes are linked. I was just wondering if we have any other cases where classes could be loaded but not linked where it would get temporarily out of sync.
01-11-2018

Closing this issue as a dup of JDK-8212117.
01-11-2018

Alan, There is nothing to do to fix issue on the serviceability side. I'm not sure what to do with it now as it is a shadow bug from the bugdb bug data base. Is it Okay to close as a dup of the JDK-8212117 ?
01-11-2018

I've looked for any anomalies in JDI in this area for the second time to make sure nothing is overlooked. But I don't see any. This is the code path: - the breakpoint is hit at the following line in main method: 9 new Foo().bar(); // breakpoint here - user types in the jdb and gets the error message: main[1] class Foo "Foo" is not a valid id or class name. - jdb command "class" => commandClass => Env.getReferenceClassFromToken(idClass) => Env.vm().classesByName(idToken) - VirtualMachine.classesByName(className) => (retrievedAllTypes) ? findReferenceTypes(signature) : retrieveClassesBySignature(signature) - findReferenceTypes uses cached result from previous call to JDWP.VirtualMachine.AllClassesWithGeneric - retrieveClassesBySignature makes a call to JDWP.VirtualMachine.ClassesBySignature - both JDWP.VirtualMachine commands ClassesBySignature and AllClassesWithGeneric make a call to the JVMTI GetLoadedClasses() and GetClassStatus. The GetClassStatus () is used to filter out the classes without the JVMTI_CLASS_STATUS_PREPARED bit. The retrieveClassesBySignature does an extra filtering by the class signature. - JVMTI GetLoadedClasses() => JvmtiGetLoadedClasses::getLoadedClasses() => ClassLoaderDataGraph::loaded_classes_do() => ClassLoaderData::loaded_classes_do() The condition to return loaded classes in the ClassLoaderData::loaded_classes_do() is: k->is_array_klass() || (k->is_instance_klass() && InstanceKlass::cast(k)->is_loaded()) so this part can not cause any issues. So that the issue is that the JVMTI GetClassStatus() does not return JVMTI_CLASS_STATUS_PREPARED bit for the Foo class. It means that it was not prepared yet. My conclusion is that there are no issues in the JDI/JDWP/JVMTI implementation. Everything is implemented according to the specs.
30-10-2018

I'm not sure what kind of anomalies you are considering. If Class.forName is fixed then naturally that may change both the initial set of classes seen by GetLoadedClasses and the timing of subsequent ClassPrepare events, if there were loaded but unlinked classes present that are loaded by forName.
14-10-2018

Do we need to check for any anomalies in JDI in this area? JDI uses GetLoadedClasses initially to get a view of the loaded classes and then uses ClassPrepare events to keep that view up to date as classes are linked. I can't quite tell how it deals with classes that have been loaded but not linked. If Class.forName is fixed then maybe this scenario doesn't arise (except for the briefest of moments) and it shouldn't impact the debugging experience.
13-10-2018

I do not see that there are two issues here at all. If you fix Class.forName then the test case passes as expected. The relation to ClassPrepare events is unclear to me in its relevance. ?? If the Class was not linked by forName then it has not been "prepared". The JVMTI class status for PREPARED is predicated on is_linked(): jint InstanceKlass::jvmti_class_status() const { jint result = 0; if (is_linked()) { result |= JVMTI_CLASS_STATUS_VERIFIED | JVMTI_CLASS_STATUS_PREPARED; } if (is_initialized()) { assert(is_linked(), "Class status is not consistent"); result |= JVMTI_CLASS_STATUS_INITIALIZED; } if (is_in_error_state()) { result |= JVMTI_CLASS_STATUS_ERROR; } return result; } The final steps of link_class_impl set the linked state and post the prepare event: set_init_state(linked); if (JvmtiExport::should_post_class_prepare()) { Thread *thread = THREAD; assert(thread->is_Java_thread(), "thread->is_Java_thread()"); JvmtiExport::post_class_prepare((JavaThread *) thread, this); } So until the class is linked there is no prepare event for it and so nothing that might happen in response to that event will happen. I see only one bug here: Class.forName does not link a class before returning it.
12-10-2018

As JDK-8212117 tracks the Class.forName issue, I move JDK-8181144 back to hotspot/jvmti to investigate if GetLoadedClasses is called to refresh the list of loaded classes when ClassPrepare event is received.
12-10-2018

We have create JDK-8212117 to track the Class.forName issue. Should we move JDK-8181144 back to hotspot/jvmti for the JVM TI AllLoadedClasses issue?
12-10-2018

No, I'm trying to separate the issue of the spec vs. implementation issue with Class.forName and the JVM TI question. Class.forName is essentially JVM_FindClassFromCaller and I would be really nervous about changing that after all these years.
12-10-2018

[~alanb] Alan you seem to be missing the basic point that Class.forName is supposed to return classes that have been linked, but it does not ensure that. At the breakpoint the class is supposed to be linked.
12-10-2018

JDI maintains a set of ReferenceType objects that it updates when it gets ClassPrepare events. At the break point in the bug report, Foo has been loaded but has not been verified and linked so the JDI client hasn't got the ClassPrepare event to add the type to its set. I don't think this debugging session leads to another call to JVM TI GetLoadedClasses. We may need to dig into the history GetLoadedClasses (it goes back to JVMDI) to see if it ever includes loaded but not linked classes.
12-10-2018

Thanks, David! I'm moving this bug to core-libs to get some evaluation. We would like a core-libs opinion on this issue. Please, feel free move this issue back if necessary.
11-10-2018

Of course we can fix in forName/forName0 if other callers of JVM_FindClassFromCaller() or find_class_from_class_loader() don't need the class to be linked.
11-10-2018

> Do you mean this? : > java.lang.Class.forName() -> forName0() -> JVM_FindClassFromCaller() -> find_class_from_class_loader() Yes. forName() is supposed to return a Class that has been linked, and optionally initialized. It currently returns classes that have not been linked. Although this is a spec conformance issue it has likely been doing the wrong thing for so long that it might be an issue to fix it - though very few usecases can see that a class is unlinked. I would suggest moving to core-libs as a P2 conformance issue to at least get their evaluation. Thanks.
11-10-2018

I've confirmed that in jvm.cpp find_class_from_classloader returns a class that is not linked. Simple fix: diff -r d6aa9ea2405d src/hotspot/share/prims/jvm.cpp --- a/src/hotspot/share/prims/jvm.cpp +++ b/src/hotspot/share/prims/jvm.cpp @@ -3544,7 +3544,11 @@ // protection_domain. The protection_domain is passed as NULL by the java code // if there is no security manager in 3-arg Class.forName(). Klass* klass = SystemDictionary::resolve_or_fail(name, loader, protection_domain, throwError != 0, CHECK_NULL); - + if (klass->is_instance_klass()) { + InstanceKlass* ik = InstanceKlass::cast(klass); + ik->link_class(CHECK_NULL); + } + but we need to verify it is appropriate for all callers of this to get a linked class. That also makes this a core-libs bug. Arguably you might expect the VM to return a linked class, but I think that has to be handled explicitly as shown - we can't just link classes arbitrarily deep inside the systemDictionary code.
07-10-2018

ClassLoaderDataGraph::loaded_classes_do() is not restricted to initialized classes. It uses ClassLoaderData::loaded_classes_do which only filters on is_loaded: void ClassLoaderData::loaded_classes_do(KlassClosure* klass_closure) { // Lock-free access requires load_acquire for (Klass* k = OrderAccess::load_acquire(&_klasses); k != NULL; k = k->next_link()) { // Do not filter ArrayKlass oops here... if (k->is_array_klass() || (k->is_instance_klass() && InstanceKlass::cast(k)->is_loaded())) { #ifdef ASSERT oop m = k->java_mirror(); assert(m != NULL, "NULL mirror"); assert(m->is_a(SystemDictionary::Class_klass()), "invalid mirror"); #endif klass_closure->do_klass(k); } } } And is_loaded() covers classes that are not yet initialized. If uninitialized classes are not being found then they must be being filtered out by some other piece of code. classesForSignature filters on JVMTI_CLASS_STATUS_PREPARED But JVMTI_CLASS_STATUS_PREPARED is set as follows: jint InstanceKlass::jvmti_class_status() const { jint result = 0; if (is_linked()) { result |= JVMTI_CLASS_STATUS_VERIFIED | JVMTI_CLASS_STATUS_PREPARED; } but "linked" is a more advanced state than "loaded" in instanceKlass: // See "The Java Virtual Machine Specification" section 2.16.2-5 for a detailed description // of the class loading & initialization procedure, and the use of the states. enum ClassState { allocated, // allocated (but not yet linked) loaded, // loaded and inserted in class hierarchy (but not linked yet) linked, // successfully linked/verified (but not initialized yet) being_initialized, // currently running class initializer fully_initialized, // initialized (successfull final state) initialization_error // error happened during initialization }; So the issue seems not that the class is not initialized, but that the class is not linked. However Class.forName states the class is linked: "Given the fully qualified name for a class or interface (in the same format returned by getName) this method attempts to locate, load, and link the class or interface. " so perhaps the issue is that the class is not actually in the linked state as expected when forName returns it?
07-10-2018

The JDI allClasses() implementation (in the debugger agent) uses the JVMTI GetLoadedClasses(). The GetLoadedClasses() implementation uses the runtime ClassLoaderDataGraph::loaded_classes_do(). I���m not sure if there is another way to implement the JDI allClasses. This bug can be qualified as a JVMTI or even a Runtime bug. Can the JVMTI use a different CLDG iterator instead of loaded_classes_do?
05-10-2018

The specification for allClasses() states: "The returned list will include reference types loaded at least to the point of preparation " Where "preparation" is defined in the JVMS as: 5.4.2 Preparation Preparation involves creating the static fields for a class or interface and initializing such fields to their default values (��2.3, ��2.4). This does not require the execution of any Java Virtual Machine code; explicit initializers for static fields are executed as part of initialization (��5.5), not preparation. --- So it is a bug for allClasses() to not return uninitialized classes.
12-07-2018