JDK-8181419 : Race in jdwp invoker handling may lead to crashes or invalid results
  • Type: Bug
  • Component: core-svc
  • Sub-Component: debugger
  • Affected Version: 8u152,9,10
  • Priority: P3
  • Status: Resolved
  • Resolution: Fixed
  • Submitted: 2017-06-01
  • Updated: 2022-01-03
  • Resolved: 2017-06-19
The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.
JDK 10 JDK 8
10 b13Fixed 8u162Fixed
Related Reports
Relates :  
Relates :  
Relates :  
Relates :  
(Issue was first found and analyzed by Ralf Schmelter at SAP. Thanks to him for his work.)

At the end of JDWP invoke handling we want to release all global references the invoke may have created. This may also  include the return value object ref, if return value was an object, and the exception ref, if an exception was thrown. 

Both return value object ref and exception ref need to be kept alive during the sending of the JDWP answer packet, because during sending references are translated into objectIDs and both references have to be not yet collected to do that.

The sending of the answer packet must happen outside the invokerLock, to minimize the chance for deadlocks.

Once sending is done, we enter the invokerLock again and clean up return value object ref and exception object ref.


	enter invokerLock;
A	reset Thread's InvokeRequest structure: 
              request->pending = JNI_FALSE;
              request->started = JNI_FALSE;
              request->available = JNI_TRUE; /* For next time around */	
	retrieve exception ref and return value object ref from InvokeRequest structure
B       leave invokerLock();
C      send answer packet containing exception ref and return value object ref, both will be resolved to objectIDs.
	enter invokerLock;
D	release  exception ref and return value object ref *by accessing the thread request structure*
	leave invokerLock();

At (A) we clear the InvokeRequest structure of the thread for the next invoke request. 

At (B) we leave lock protection and from now on a new invoke request may be filled in immediately by the JDWP listener thread.

At (D) we release the references by taking them from the thread request structure. I think this is wrong, because the request structure may already have been filled by another completed invoke request, whose return values we now delete. If that request answer is sent after ours, it will encounter invalid object references.


IMHO there are several ways to fix this:

1) move the "request->available = JNI_TRUE;" down after the answer packet was sent. But this opens the possibility for deadlocks, because while this (potentially longish) IO happens, no other invoke request can be served.

2) We do not need to access the thread request structure to release the object references contained in it. Just use local variables to keep the object references to release.

3) We also could split outStream_writeObjectRef() into two parts, one which gets the objectID, one which writes the objectID. Then, we could get the objectIDs for exception ref and return value object ref before (B), release them also before (B), and just send back the objectIDs in C.

I favour (2) though, because it is a less invasive change.
Fix Request as per http://openjdk.java.net/projects/jdk9/fix-request-process This bug is an improvement to JDK-8153711 which is already in JDK 9. Thus, it would make sense to also get this bug into JDK 9 as it only makes the fix of JDK-8153711 more robust. Test coverage is provided by the test added with JDK-8153711, test/com/sun/jdi/OomDebugTest.java. I'm the author of the original fix JDK-8153711 and have reviewed this fix. Serguei Spitsyn reviewed it too. He was one of the reviewers of the original fix. Patch for JDK 10 applies as is for JDK 9, it builds fine for me and passes the test.

URL: http://hg.openjdk.java.net/jdk10/jdk10/jdk/rev/b0d1ada042b6 User: stuefe Date: 2017-06-19 10:58:07 +0000