JDK-8290482 : Update JNI Specification of DestroyJavaVM for better alignment with JLS, JVMS, and Java SE API Specifications
  • Type: Bug
  • Component: hotspot
  • Sub-Component: runtime
  • Affected Version: 20
  • Priority: P4
  • Status: Resolved
  • Resolution: Fixed
  • Submitted: 2022-07-19
  • Updated: 2022-10-03
  • Resolved: 2022-09-27
The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.
JDK 20
20 b17Fixed
Related Reports
CSR :  
JDK-8292897 - Update JNI Specification of DestroyJavaVM for better alignment with JLS, JVMS, and Java SE API Specifications
Relates :  
JDK-8290196 - 12.8: Clarify the definition of program exit
Relates :  
JDK-8291955 - Cleanup the JNI invocation API spec and code examples
Relates :  
JDK-8290036 - Define and specify Runtime shutdown sequence
Relates :  
JDK-8290388 - 5.7: Clarify the definition of JVM termination
Description
There are some updates pending to the JVMS, JLS, and the specification of java.lang.Runtime in the Java SE API that clarify the JDK shutdown sequence, JVM halt, and program exit. The JNI Specification will need some updates to align with these changes.

Specific points include:

* The Overview in Chapter 5 includes a small section "Terminating the VM" which refers to "JNI_DestroyJavaVM" (name isn't quite right) which mentions "user threads" and the possibility of "holding system resources" and how the VM cannot automatically free them. It seems like this discussion isn't very helpful. Probably all that's necessary here is a forward reference to the DestroyJavaVM function later in this chapter.

* The specification of the DestroyJavaVM function mentions that it "terminates" the VM and makes a "best-effort attempt to release resources". It should mention something about initiating the _shutdown sequence_ (to be defined in the java.lang.Runtime class specification). Adding a link here is probably sufficient here.

* I don't think this function actually halts the VM itself; I believe it waits for something on the JDK side to halt the VM. (The JDK calls Runtime.halt at the end of the shutdown sequence, or any thread can call Runtime.halt directly.) But what this function actually does should be specified.

* The discussion about DestroyJavaVM waiting for the current thread to be the only non-daemon thread should be clarified to cover the case where the current thread is a daemon thread.

* There probably should be some statement about the state of the VM after this call returns. Presumably on success it means that the JVM has halted (no Java code is being executed) or similar.

* The main table of contents has an entry in Chapter 5, Overview, "Unloading the VM" which points to nowhere (but probably is intended to point to the "Terminating the VM" section.
Comments
Changeset: e5b65c40 Author: David Holmes <dholmes@openjdk.org> Date: 2022-09-27 23:57:32 +0000 URL: https://git.openjdk.org/jdk/commit/e5b65c40ea032c6955311593e02ed44f14dfe80a
27-09-2022
A pull request was submitted for review. URL: https://git.openjdk.org/jdk/pull/10352 Date: 2022-09-20 00:55:32 +0000
20-09-2022
I've tweaked the overview section but still retained it as it summarises the use of the invocation API. The spec for DestroyJavaVM should now be much clearer with regards to exactly how it operates. I've added the change that the thread must not have any Java frames on the call stack (and also fixed that in DetachCurrentThread).
17-08-2022
I will also make some minor clarifications (not changes) to JNI_CreateJavaVM and the AttachCurrentThread* functions as needed e.g. to clearly state that the thread that calls JNI_CreateJavaVM and becomes the "main thread" is attached to the VM.
16-08-2022
> It seems like it ought to be illegal if DestroyJavaVM is called from native code while Java frames are on the stack. (For example, from a native > method called from Java.) At least, when I tried it, it went through the shutdown sequence, and then the JVM crashed. I investigated the crash to see whether it was something easily avoided, such that the thread would just hang as expected (blocking at the termination safepoint as it tries to transition from the native method back to Java code). But it is not easily avoided at all. The current logic treats things as a termination of the thread, as well as termination of the VM. Logically this is incorrect when there are Java frames on the stack as the thread has not terminated by JLS definition. If we try to proceed with thread termination then we crash when freeing the VM Thread/JavaThread resources as these are not in a valid state when Java frames still exist - we don't try to terminate live threads. If we try to skip that aspect of the termination then other parts of the VM termination code will encounter problems as it can only be executed by a terminated JavaThread. We would have to actually introduce a new (third) termination process to make this work. The second process used for Runtime.exit() can take a lot of short-cuts (that don't require the thread to be terminated) because we know we are blowing away the entire process. Given we have to tread carefully in mandating specific behaviours in the JNI spec, we should probably just adjust the spec to allow an implementation to constrain calls to DestroyJavaVM to be made from threads with no active Java frames, or else report JNI_ERR and do nothing. Update: Actually there is a simpler path here. DestroyJavaVM is already specified such that trying to detach a thread with active Java frames is not allowed and returns JNI_ERR. So if we clarify that DestroyJavaVM would also detach the current thread, then implicitly it too cannot have any active Java frames. ... except that is only stated in the overview section NOT the actual method specification, so that needs to be fixed too.
16-08-2022
The broken link in the top-level TOC is fixed under JDK-8291585
31-07-2022
[calling DestroyJavaVM from a native method with Java frames on the stack] > Though it shouldn't crash it should just hang if you try to return to the VM. If the caller is a non-daemon thread it indeed just hangs. If the caller is a daemon thread then it crashes in malloc() with an invalid pointer, in the JavaThread destructor, I think in the "delete thread" statement of Threads::destroy_vm(). This is right after a comment, "Deleting the shutdown thread here is safe." :-) This is anyway of little import since it's clear that calling DestroyJavaVM this way is nonsensical. I've filed bug JDK-8290732 regarding the behavior when DestroyJavaVM is called on a daemon thread. I'll defer to your judgment as to how the specification should handle this and other issues.
20-07-2022
> I did some more poking at DestroyJavaVM. Don't poke too hard, this is more tissue paper than keflar. ;-) > * It seems like it ought to be illegal if DestroyJavaVM is called from native code while Java frames are on the stack. True, this makes no sense. Though it shouldn't crash it should just hang if you try to return to the VM. There is not much resilience here e.g. if two threads call DestroyJavaVM then things would also get "interesting". We could add more guidance on usage and perhaps some additional checks - though JNI typically requires the user to use it correctly. > The assumption seems to be that DestroyJavaVM is called on a non-daemon thread. I'd call this a bug. The real assumption is that DestroyJavaVM is called from an unattached thread in the first place, so it will attach as a non-daemon. But we don't require that it be an unattached thread so it should work. > It would be good to clarify the circumstances under which DestroyJavaVM returns At a high level yes. It initiates the shutdown sequence and logically waits for that to complete, so if it doesn't complete then neither does DestroyJavaVM.
20-07-2022
I did some more poking at DestroyJavaVM. Here are some observations that I think ought to be addressed in the spec and maybe in the implementation, depending on how the spec changes. * It seems like it ought to be illegal if DestroyJavaVM is called from native code while Java frames are on the stack. (For example, from a native method called from Java.) At least, when I tried it, it went through the shutdown sequence, and then the JVM crashed. The JVM is shut down but the caller attempts to return to Java code, which of course doesn't make sense. The spec might need a precondition that this function be called only when no Java frames are on the stack. Or the behavior could be unspecified. And perhaps some checking could be done to ensure this case won't happen. * The code in Threads::destroy_vm() waits until the non-daemon thread count drops to one before proceeding with the shutdown sequence and then exiting the VM. The assumption seems to be that DestroyJavaVM is called on a non-daemon thread. The launcher does this; this provides the expected behavior of awaiting termination of all the application's non-daemon threads before shutdown. However, if DestroyJavaVM is called from a daemon thread, it starts the shutdown sequence while one of the application's non-daemon threads is still executing. This is easily demonstrated by having the main native thread detach and then reattach as a daemon thread. I'm not sure whether this is a bug somewhere or whether there should be a requirement that DestroyJavaVM be called on a non-daemon thread. * It would be good to clarify the circumstances under which DestroyJavaVM returns. I think that at the end of the shutdown sequence, it will stop (or block, or whatever) all the Java threads, do all the stuff in the comment at Threads::destroy_vm(), and then return. If something prevents the shutdown sequence from completing, DestroyJavaVM will never return. This differs from Runtime::halt which doesn't start the shutdown sequence, but which does minimal cleanup and then exits the entire process. (From native code this can be done by JVM_Halt(), but that's not part of JNI AFAICS, so yes discussion of it probably doesn't belong here.) (I may have said in the past that the JVM halts at the end of the shutdown sequence. From Java threads' point of view this might be the indistinguishable from calling Runtime::halt, but it's quite different from the JNI perspective.)
20-07-2022
> - The section "Terminating the VM" ought to give _some_ motivation for the wait-until-no-nondaemon-thread-running behavior. That is the termination/exit condition defined by the JLS for how/when a program normally exits, so the motivation can/should come from there. > - Re: "halt() does not happen with DestroyJavaVM, nor should it. But by definition the VM has terminated by that stage as per latest JLS 12.8 text." Although it is not stated, there is an understanding that Runtime.halt(), or in general "halting the VM" implies termination of the process hosting the VM. If we start talking about "halt" with DestroyJavaVM it will cause a lot of confusion.
20-07-2022
- The section "Terminating the VM" ought to give _some_ motivation for the wait-until-no-nondaemon-thread-running behavior. - Re: "halt() does not happen with DestroyJavaVM, nor should it. But by definition the VM has terminated by that stage as per latest JLS 12.8 text." -- the latest 12.8 text in JDK-8290196 _avoids_ saying what happens to the VM, and speaks _only_ of what happens to the (threads of the) program: "Upon program exit, any daemon or non-daemon thread that has not yet exited will execute no further Java code." It is incumbent on the JNI Spec to explain that the reason why threads are "blocked" from executing further Java code is because the VM has terminated. It would also be helpful for the JNI Spec to note that the VM terminates and DestroyJavaVM returns, versus when the VM halts and Runtime.halt does not return.
19-07-2022
> * I don't think this function actually halts the VM itself; I believe it waits for something on the JDK side to halt the VM. (The JDK calls Runtime.halt at the end of the shutdown sequence, or any thread can call Runtime.halt directly.) But what this function actually does should be specified. halt() is only called after the shutdown sequence via Runtime.exit(). halt() does not happen with DestroyJavaVM, nor should it. But by definition the VM has terminated by that stage as per latest JLS 12.8 text. We basically have: Runtime.exit -> shutdown sequence (ie run hooks and wait for them, tell VM to "exit") -> halt() DestroyJavaVM -> wait (if necessary) until no non-daemon thread running -> shutdown sequence (ie run hooks and wait for them, tell VM to "exit") > The main table of contents has an entry in Chapter 5, Overview, "Unloading the VM" which points to nowhere (but probably is intended to point to the "Terminating the VM" section. Yes oversight on my part when I re-did the wording to not talk about "unloading the VM" - which we never did and would be technically extremely challenging. The mention of thread resources etc goes back to this old text. The new text about best-effort allows a complaint implementation to fully unload the JVM but doesn't place any actual requirements on an implementation in that regard.
19-07-2022
[~abuckley] [~dholmes] please take note.
19-07-2022