JDK-7154963 : crash in JvmtiEnvBase::get_current_contended_monitor()
  • Type: Bug
  • Component: hotspot
  • Sub-Component: jvmti
  • Affected Version: hs19.1,hs24,hs25
  • Priority: P3
  • Status: Closed
  • Resolution: Duplicate
  • OS: linux
  • CPU: x86
  • Submitted: 2012-03-19
  • Updated: 2013-09-24
  • Resolved: 2013-09-24
The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.
Other
tbd_minorResolved
Related Reports
Relates :  
Relates :  
Description
Ashok reported the following crash in JVM/TI GetCurrentContendedMonitor():

Hi Daniel,

 

I am working on a JVM crash in our environment in a JVMTI call.

I have looked at the problem and analyzed the core dump, I don���t see a possibility of the JVMD agent causing this crash. I am suspecting a bug in JVM code.

 

Can you please have a look at this crash and let us know your view on it or can you suggest others from the JVM team who could have a look at it ( I could raise a bug for this).

 

The details of the crash and the related files and its location are in the attached file. I have attached the hs_err file for reference.

 

Thanks and Regards,

Ashok
Here's the HSX-19 version of the code that is crashing. I don't have
a copy of HSX-19.1 at my fingertips...

src/share/vm/prims/jvmtiEnvBase.cpp:

   642  jvmtiError
   643  JvmtiEnvBase::get_current_contended_monitor(JavaThread *calling_thread,
JavaThread *java_thread, jobject *monitor_ptr) {
   644  #ifdef ASSERT
   645    uint32_t debug_bits = 0;
   646  #endif
   647    assert((SafepointSynchronize::is_at_safepoint() ||
   648            is_thread_fully_suspended(java_thread, false, &debug_bits)),
   649           "at safepoint or target thread is suspended");
   650    oop obj = NULL;
   651    ObjectMonitor *mon = java_thread->current_waiting_monitor();
   652    if (mon == NULL) {
   653      // thread is not doing an Object.wait() call
   654      mon = java_thread->current_pending_monitor();
   655      if (mon != NULL) {
   656        // The thread is trying to enter() or raw_enter() an ObjectMonitor
.
   657        obj = (oop)mon->object();
   658        // If obj == NULL, then ObjectMonitor is raw which doesn't count
   659        // as contended for this API
   660      }
   661      // implied else: no contended ObjectMonitor
   662    } else {
   663      // thread is doing an Object.wait() call
   664      obj = (oop)mon->object();
   665      assert(obj != NULL, "Object.wait() should have an object");
   666    }
   667
   668    if (obj == NULL) {
   669      *monitor_ptr = NULL;
   670    } else {
   671      HandleMark hm;
   672      Handle     hobj(obj);
   673      *monitor_ptr = jni_reference(calling_thread, hobj);
   674    }
   675    return JVMTI_ERROR_NONE;
   676  }

The above function matches the current HSX-24-B04 source.


Based a quick look at the disassembly that Ashok provided
(I stripped off some of the name mangling stuff for clarity):

(gdb) disass 0x00002af8d716b911
Dump of assembler code for function JvmtiEnvBase29get_current_contended_monitor:
0x00002af8d716b8e0 <JvmtiEnvBase29get_current_contended_monitor+0>:    push   %rbp
0x00002af8d716b8e1 <JvmtiEnvBase29get_current_contended_monitor+1>:    mov    %rsp,%rbp
0x00002af8d716b8e4 <JvmtiEnvBase29get_current_contended_monitor+4>:    sub    $0x60,%rsp
0x00002af8d716b8e8 <JvmtiEnvBase29get_current_contended_monitor+8>:    mov    0xc8(%rdx),%rax
0x00002af8d716b8ef <JvmtiEnvBase29get_current_contended_monitor+15>:   mov    %r13,-0x18(%rbp)
0x00002af8d716b8f3 <JvmtiEnvBase29get_current_contended_monitor+19>:   mov    %r14,-0x10(%rbp)
0x00002af8d716b8f7 <JvmtiEnvBase29get_current_contended_monitor+23>:   mov    %r15,-0x8(%rbp)
0x00002af8d716b8fb <JvmtiEnvBase29get_current_contended_monitor+27>:   xor    %r13d,%r13d
0x00002af8d716b8fe <JvmtiEnvBase29get_current_contended_monitor+30>:   mov    %rbx,-0x28(%rbp)
0x00002af8d716b902 <JvmtiEnvBase29get_current_contended_monitor+34>:   mov    %r12,-0x20(%rbp)
0x00002af8d716b906 <JvmtiEnvBase29get_current_contended_monitor+38>:   test   %rax,%rax
0x00002af8d716b909 <JvmtiEnvBase29get_current_contended_monitor+41>:   mov    %rsi,%r15
0x00002af8d716b90c <JvmtiEnvBase29get_current_contended_monitor+44>:   mov    %rcx,%r14
0x00002af8d716b90f <JvmtiEnvBase29get_current_contended_monitor+47>:   je     0x2af8d716b990 <JvmtiEnvBase29get_current_contended_monitor+176>
0x00002af8d716b911 <JvmtiEnvBase29get_current_contended_monitor+49>:   mov    0x8(%rax),%r13
0x00002af8d716b915 <JvmtiEnvBase29get_current_contended_monitor+53>:   test   %r13,%r13
0x00002af8d716b918 <JvmtiEnvBase29get_current_contended_monitor+56>:   je     0x2af8d716b986 <JvmtiEnvBase29get_current_contended_monitor=+166>


I think the crash is happening at line 664:

   662    } else {
   663      // thread is doing an Object.wait() call
   664      obj = (oop)mon->object();

on the attempt to dereference 'mon' to get the associated object.

Comments
Closing this issue as dup of JDK-8022836. Please, see previous comment from Dan.
24-09-2013

This bug was fixed by the following changeset: Changeset: c1d7040a1183 Author: sgabdura Date: 2013-09-18 16:48 +0400 URL: http://hg.openjdk.java.net/hsx/hotspot-rt/hotspot/rev/c1d7040a1183 8022836: JVM crashes in JVMTIENVBASE::GET_CURRENT_CONTENDED_MONITOR and GET_OWNED_MONITOR Summary: Check that the _java_thread parameter is valid when it is possible that the JavaThread has exited after the initial checks were made in generated/jvmtifiles/jvmtiEnter.cpp: jvmti_GetCurrentContendedMonitor() Reviewed-by: dcubed, dsamersoff ! src/share/vm/prims/jvmtiEnvBase.hpp The HSX-25 code review was done using this bug id (7154963), but the fix was pushed using the JBS shadow (8022836) of the bugDB bug. Why? I have no idea. Probably some process thing that I don't know about. I suspect that this bug (7154963) should be closed as "duplicate-by" 8022836, but I'm not sure.
19-09-2013

SQE OK to defer.
09-07-2013

7u40-defer-request justification: Requesting deferral to 7u60 as the fix for this will not be ready in time for 7u40.
09-07-2013

I think this has fallen off the radar because it didn't mention hs24 or hs25. As a P3 this should have been targetted to hs24 and deferred if needed.
09-07-2013

Are there any plans to fix this race Condition? I'm asking as this was now reported by an external Weblogic customer. Thomas
09-07-2013

EVALUATION This line: 1056 if (is_thread_fully_suspended(java_thread, true, &debug_bits)) { can also return 'false' if suspension has not been requested for the target thread which is permitted by JVM/TI GetCurrentContendedMonitor(). I keep forgetting that "feature". In that usage model for the function, the VM is brought to a safepoint to ensure that access of the target java_thread is "safe". However, since JvmtiEnvBase::get_current_contended_monitor() doesn't revalidate the target 'java_thread', things aren't "safe". Put another way, the outer wrapper code in JvmtiEnter.cpp sanity checks the target 'java_thread', but there is nothing to keep that JavaThread from exiting by the time the VM operation tries to do the work.
19-03-2012

EVALUATION I just noticed that this call: 1056 if (is_thread_fully_suspended(java_thread, true, &debug_bits)) { passes 'true' for the 'wait_for_suspend' parameter which means that is_thread_fully_suspended() will only return 'false' for a couple of reasons: - the suspend operation took too many retries (very rare) - the target thread has exited Line 1056 calls src/share/vm/prims/jvmtiEnvBase.cpp: 461 JvmtiEnvBase::is_thread_fully_suspended(JavaThread* thr, bool wait_for_s uspend, uint32_t *bits) { 462 // "other" threads require special handling 463 if (thr != JavaThread::current()) { 464 if (wait_for_suspend) { 465 // We are allowed to wait for the external suspend to complete 466 // so give the other thread a chance to get suspended. 467 if (!thr->wait_for_ext_suspend_completion(SuspendRetryCount, 468 SuspendRetryDelay, bits)) { 469 // didn't make it so let the caller know 470 return false; 471 } 472 } which results in a call to: src/share/vm/runtime/thread.cpp 540 // Wait for an external suspend request to complete (or be cancelled). 541 // Returns true if the thread is externally suspended and false otherwis e. 542 // 543 bool JavaThread::wait_for_ext_suspend_completion(int retries, int delay, 544 uint32_t *bits) { which calls: 561 { 562 MutexLockerEx ml(SR_lock(), Mutex::_no_safepoint_check_flag); 563 is_suspended = is_ext_suspend_completed(true /* called_by_wait */, 564 delay, bits); 565 pending = is_external_suspend(); 566 } src/share/vm/runtime/thread.cpp: 418 bool JavaThread::is_ext_suspend_completed(bool called_by_wait, int delay , uint32_t *bits) { 419 TraceSuspendDebugBits tsdb(this, false /* !is_wait */, called_by_wait, bits); 420 421 bool did_trans_retry = false; // only do thread_in_native_trans retry once 422 bool do_trans_retry; // flag to force the retry 423 424 *bits |= 0x00000001; 425 426 do { 427 do_trans_retry = false; 428 429 if (is_exiting()) { 430 // Thread is in the process of exiting. This is always checked 431 // first to reduce the risk of dereferencing a freed JavaThread. 432 *bits |= 0x00000100; 433 return false; 434 } and that first check in the loop is where I think the 'false' return value is coming from. The target java_thread has left the building...
19-03-2012

EVALUATION Here's the generated/jvmtifiles/jvmtiEnter.cpp code: 885 static jvmtiError JNICALL 886 jvmti_GetCurrentContendedMonitor(jvmtiEnv* env, 887 jthread thread, 888 jobject* monitor_ptr) { 889 890 #ifdef JVMTI_KERNEL 891 return JVMTI_ERROR_NOT_AVAILABLE; 892 #else 893 if(!JvmtiEnv::is_vm_live()) { 894 return JVMTI_ERROR_WRONG_PHASE; 895 } 896 Thread* this_thread = (Thread*)ThreadLocalStorage::thread(); 897 if (this_thread == NULL || !this_thread->is_Java_thread()) { 898 return JVMTI_ERROR_UNATTACHED_THREAD; 899 } 900 JavaThread* current_thread = (JavaThread*)this_thread; 901 ThreadInVMfromNative __tiv(current_thread); 902 __ENTRY(jvmtiError, jvmti_GetCurrentContendedMonitor , current_thread) 903 debug_only(VMNativeEntryWrapper __vew;) 904 CautiouslyPreserveExceptionMark __em(this_thread); 905 JvmtiEnv* jvmti_env = JvmtiEnv::JvmtiEnv_from_jvmti_env(env); 906 if (!jvmti_env->is_valid()) { 907 return JVMTI_ERROR_INVALID_ENVIRONMENT; 908 } 909 910 if (jvmti_env->get_capabilities()->can_get_current_contended_monitor = = 0) { 911 return JVMTI_ERROR_MUST_POSSESS_CAPABILITY; 912 } 913 jvmtiError err; 914 JavaThread* java_thread; 915 if (thread == NULL) { 916 java_thread = current_thread; 917 } else { 918 oop thread_oop = JNIHandles::resolve_external_guard(thread); 919 if (thread_oop == NULL) { 920 return JVMTI_ERROR_INVALID_THREAD; 921 } 922 if (!thread_oop->is_a(SystemDictionary::Thread_klass())) { 923 return JVMTI_ERROR_INVALID_THREAD; 924 } 925 java_thread = java_lang_Thread::thread(thread_oop); 926 if (java_thread == NULL) { 927 return JVMTI_ERROR_THREAD_NOT_ALIVE; 928 } 929 } 930 if (monitor_ptr == NULL) { 931 return JVMTI_ERROR_NULL_POINTER; 932 } 933 err = jvmti_env->GetCurrentContendedMonitor(java_thread, monitor_ptr); 934 return err; 935 #endif // JVMTI_KERNEL 936 } checks for jthread parameter are on lines 915, 919, 922, and 925. If the jthread makes it through all those checks, then the following code is called: src/share/vm/prims/jvmtiEnv.cpp: 1048 // Threads_lock NOT held, java_thread not protected by lock 1049 // java_thread - pre-checked 1050 // monitor_ptr - pre-checked for NULL 1051 jvmtiError 1052 JvmtiEnv::GetCurrentContendedMonitor(JavaThread* java_thread, jobject* m onitor_ptr) { 1053 jvmtiError err = JVMTI_ERROR_NONE; 1054 uint32_t debug_bits = 0; 1055 JavaThread* calling_thread = JavaThread::current(); 1056 if (is_thread_fully_suspended(java_thread, true, &debug_bits)) { 1057 err = get_current_contended_monitor(calling_thread, java_thread, mon itor_ptr); 1058 } else { 1059 // get contended monitor information at safepoint. 1060 VM_GetCurrentContendedMonitor op(this, calling_thread, java_thread, monitor_ptr); 1061 VMThread::execute(&op); 1062 err = op.result(); 1063 } 1064 return err; 1065 } /* end GetCurrentContendedMonitor */ If the target 'java_thread' is fully suspended, then we call get_current_contended_monitor() directory. Otherwise, we do the work via a safepoint. Since Ashok's stack trace shows the VM operation code path, we know that is_thread_fully_suspended() returned false. Two header comments for the above code are important here: 1048 // Threads_lock NOT held, java_thread not protected by lock 1049 // java_thread - pre-checked The java_thread passed to JvmtiEnv::GetCurrentContendedMonitor() was checked before being passed, but there is nothing to prevent that target java_thread from exiting...
19-03-2012