JDK-8296936 : -Xcheck:jni creates WARNING: JNI local refs: xxx, exceeds capacity: 32 with jdwp agent active
  • Type: Bug
  • Component: hotspot
  • Sub-Component: jvmti
  • Affected Version: 20
  • Priority: P4
  • Status: New
  • Resolution: Unresolved
  • OS: generic
  • CPU: generic
  • Submitted: 2022-11-14
  • Updated: 2022-11-17
Related Reports
Relates :  
JDK-8297106 - Remove the -Xcheck:jni local reference capacity checking
Relates :  
JDK-8296709 - WARNING: JNI call made without checking exceptions
Relates :  
JDK-8193222 - EnsureLocalCapacity() should maintain capacity requests through multiple calls
Description
When the vm is started with jdwp agent active and -Xcheck:jni is specified in addition, the message 
WARNING: JNI local refs: xxx, exceeds capacity: 32
is issued. 'xxx' is a (varying) number significantly larger than the anticipated capacity (32). The warning message causes the newly introduced test 
test/jdk/tools/launcher/TestXcheckJNIWarnings.java
to fail. The test will be enhanced by JDK-8296706 to cover the jdwp agent case as well (two test ids). The jdwp agent case will be problem-listed until this bug is resolved.

To reproduce, simply issue the command
java -Xcheck:jni -agentlib:jdwp=transport=dt_socket,server=y,suspend=n -version

If the number of outstanding JNI local refs is as expected, the checks in 
src/hotspot/prims/jniCheck.cpp
should be adapted. Otherwise, it should be analysed why the number of local refs is excessive and if it can be brought down.
Comments
[~dholmes] Thank you for the comments with updates on this!
17-11-2022
I'm going to get rid of this functionality - see JDK-8297106 - as it cannot be implemented correctly. I will close this issue as a duplicate of that once approved.
17-11-2022
I think my fix in JDK-8193222 is flawed. The intent was to not decrease the ensured capacity, but the fix also fails to increase it in all circumstances
16-11-2022
I suppose for these big numbers it will likely work by chance if after the call to the JVMTI method, the native code calls EnsureLocalCapacity with the number of local-refs that JVMTI returned. IIUC the actual capacity check won't happen until the first JNI call after the JVMTI call. I'd need to dig deep into the history here to try and understand the exact rationale for how it works. I think this is one of those areas that always looks wrong when looked at with fresh eyes, and then as you dig things become a bit clearer.
16-11-2022
> EnsureLocalCapacity only increases the planned capacity if the requested capacity is greater than the current planned capacity - it doesn't take into account how much is already used! Oh! I wonder how often it is actually used properly in that case. To its credit, the debug agent does use PushLocalFrame (passing in the planned capacity) whenever it allocates localrefs (this current issue not withstanding), so I guess that is the preferred way to make sure the needed # of localrefs is available. But that still doesn't solve the JVMTI problem, which is now worse than I thought since EnsureLocalCapacity can't be relied on.
16-11-2022
The Xcheck:jni behaviour was intended to allow the programmer to detect if they have sufficient local ref capacity so the code can run correctly on any VM. That seems good in theory, but in practice ... does any other VM actually enforce a limit? The implementation of checking the local ref capacity has been quite buggy and I'm still not sure it is completely correct, In fact the more I look at it the more I think it is wrong and makes this current problem impossible to solve cleanly. At any given time there is an existing capacity limit (the planned capacity), and an actual usage of capacity. EnsureLocalCapacity only increases the planned capacity if the requested capacity is greater than the current planned capacity - it doesn't take into account how much is already used!
16-11-2022
Maybe it should be up to JVMTI to do the EnsureLocalCapacity() since it has knowledge of how many localrefs it needs to allocate (and it is after all the one doing the allocating). There is a minor issue with this solution. JVMTI has no way of knowing what the current capacity is, so it would probably just do an EnsureLocalCapacity() for the number it needs. That means when done, there may be no capacity left for the caller. So callers of these APIs would need to defend against this and call EnsureLocalCapacity() again. To be polite (and probably avoid most issues), JVMTI should probably do a EnsureLocalCapacity(32) before returning just to make sure the default number of localrefs is still available.
16-11-2022
The warning is happening during the debug agent's call to JVMTI GetLoadedClasses(). It returns an array of jclass references, which are local refs. The problem is the agent has no idea how many classes might be returned, so it can't use EnsureLocalCapacity() in a reasonable way. The best it could do is pass some arbitrary very high number in hopes that it is always enough. This is not just an issue for the debug agent. I noticed the JLI agent also calls GetLoadedClasses(). I added -Xcheck:jni to the following test: test/jdk/java/lang/instrument/GetAllLoadedClassesTest.java In the log it produced: WARNING: JNI local refs: 974, exceeds capacity: 32 at sun.instrument.InstrumentationImpl.getAllLoadedClasses0(java.instrument@20-internal/Native Method) at sun.instrument.InstrumentationImpl.getAllLoadedClasses(java.instrument@20-internal/InstrumentationImpl.java:201) at GetAllLoadedClassesTest.testGetAllLoadedClasses(GetAllLoadedClassesTest.java:70) at GetAllLoadedClassesTest.doRunTest(GetAllLoadedClassesTest.java:61) at ATestCaseScaffold.runTest(ATestCaseScaffold.java:60) at GetAllLoadedClassesTest.main(GetAllLoadedClassesTest.java:55) at java.lang.invoke.LambdaForm$DMH/0x0000000801006000.invokeStatic(java.base@20-internal/LambdaForm$DMH) at java.lang.invoke.LambdaForm$MH/0x0000000801007000.invoke(java.base@20-internal/LambdaForm$MH) at java.lang.invoke.Invokers$Holder.invokeExact_MT(java.base@20-internal/Invokers$Holder) at jdk.internal.reflect.DirectMethodHandleAccessor.invokeImpl(java.base@20-internal/DirectMethodHandleAccessor.java:155) at jdk.internal.reflect.DirectMethodHandleAccessor.invoke(java.base@20-internal/DirectMethodHandleAccessor.java:104) at java.lang.reflect.Method.invoke(java.base@20-internal/Method.java:578) at com.sun.javatest.regtest.agent.MainWrapper$MainThread.run(MainWrapper.java:125) at java.lang.Thread.run(java.base@20-internal/Thread.java:1591) And this issue extends beyond GetLoadedClasses(). The following JVMTI APIs all return an array of local refs, and could potentially result in exceeding the capacity. The debug agent and JLI agent use most of them: GetLoadedClasses GetClassLoaderClasses GetAllModules GetObjectsWithTags GetAllThreads GetOwnedMonitorInfo GetTopThreadGroups GetThreadGroupChildren
15-11-2022