JDK-6250286 : AsynGetCallTrace can deadlock
  • Type: Bug
  • Component: hotspot
  • Sub-Component: jvmti
  • Affected Version: 5.0u4,6
  • Priority: P3
  • Status: Resolved
  • Resolution: Fixed
  • OS: generic
  • CPU: generic
  • Submitted: 2005-04-04
  • Updated: 2010-04-02
  • Resolved: 2005-05-13
The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.
Other JDK 6
5.0u4Fixed 6 b35Fixed
Related Reports
Relates :  
Description
I was running Dan's forte stress tests and the PepTest hung. The stack
trace is here:

=>[1] ___lwp_mutex_lock(0xdfa1a4d8), at 0xdfadb349
  [2] _lwp_mutex_lock(0xdfa1a4d8), at 0xdfb073c6
  [3] ThreadCritical::ThreadCritical(this = ???) (optimized), at 0xdf01b0b5 (line ~33) in "threadCritical_solaris.cpp"
  [4] Chunk::operator new(requested_size = ???, length = ???) (optimized), at 0xde989b52 (line ~179) in "allocation.cpp"
  [5] Arena::grow(this = ???, x = ???) (optimized), at 0xde98a388 (line ~300) in "allocation.cpp"
  [6] HandleArea::allocate_handle(this = ???, obj = ???) (optimized), at 0xdeb563f0 (line ~17) in "handles.cpp"
  [7] Handle::Handle(this = ???, thread = ???, obj = ???) (optimized), at 0xdeb564f9 (line ~25) in "handles.cpp"
  [8] jniIdSupport::to_jmethod_id_or_null(method_oop = ???) (optimized), at 0xdec45254 (line ~873) in "jniId.cpp"
  [9] methodOopDesc::find_jmethod_id_or_null(this = ???) (optimized), at 0xdeecbcb7 (line ~820) in "methodOop.cpp"
  [10] forte_fill_call_trace_given_top(thd = ???, trace = ???, depth = ???, top_frame = CLASS) (optimized), at 0xdeb11cff (line ~665) in "forte.cpp"
  [11] AsyncGetCallTrace(trace = ???, depth = ???, ucontext = ???) (optimized), at 0xdeb121ac (line ~837) in "forte.cpp"
  [12] profhandler(sig = 29, siginfo = 0x80453c4, ucontext = 0x80451c4), line 171 in "b4757672.c"
  [13] __sighndlr(0x1d, 0x80453c4, 0x80451c4, 0xde6112d0), at 0xdfb8269f
  ---- called from signal handler with signal 29 (SIGPROF) ------
  [14] ThreadCritical::~ThreadCritical( this = ???) (optimized), at 0xdf01b2d2 (line ~52) in "threadCritical_solaris.cpp"
  [15] Chunk::operator new(requested_size = ???, length = ???) (optimized), at 0xde989b84 (line ~179) in "allocation.cpp"
  [16] Arena::grow(this = ???, x = ???) (optimized), at 0xde98a388 (line ~300) in "allocation.cpp"
  [17] HandleArea::allocate_handle(this = ???, obj = ???) (optimized), at 0xdeb563f0 (line ~17) in "handles.cpp"
  [18] Handle::Handle(this = ???, thread = ???, obj = ???) (optimized), at 0xdeb564f9 (line ~25) in "handles.cpp"
  [19] SymbolTable::basic_add(this = ???, cp = CLASS, names_count = ???, names = ???, lengths = ???, cp_indices = ???, hashValues = ???, __the_thread__ = ???) (optimized), at 0xdefe9a8f (line ~131) in "symbolTable.cpp"
  [20] SymbolTable::add(cp = CLASS, names_count = ???, names = ???, lengths = ???, cp_indices = ???, hashValues = ???, __the_thread__ = ???) (optimized), at 0xdefe960a (line ~64) in "symbolTable.cpp"
  [21] ClassFileParser::parse_constant_pool_entries(this = ???, cp = CLASS, length = ???, __the_thread__ = ???) (optimized), at 0xdea37094 (line ~160) in "classFileParser.cpp"
  [22] ClassFileParser::parse_constant_pool(this = ???, __the_thread__ = ???) (optimized), at 0xdea37f11 (line ~212) in "classFileParser.cpp" 
...



This seems to be a very obvious deadlock in ThreadCritical. The PROF signal must
have hit right after this line in ~ThreadCritical

      global_mut_owner = -1;

and before this line:

if (os::Solaris::mutex_unlock(&global_mut)) 


and we deadlock.
###@###.### 2005-04-04 20:54:43 GMT

Comments
SUGGESTED FIX The general idea to fix this bug will be to refactor jniIDCreator into: jniIdCreatorCommon - common parts of the helper class jniIdCreator - helper class for the regular system jniIDCreatorWithNoHandles - helper class for things like AsyncGetCallTrace() which cannot allocate handles AsyncGetCallTrace() will also be modified to create a NoHandleMark() object in order to enforce the "never allocate handles" invariant. ###@###.### 2005-04-20 00:53:08 GMT See attached 6250286-cr1-full-webrev.tar.Z file for proposed Tiger-Update4 version of the fix. ###@###.### 2005-04-21 17:50:06 GMT
20-04-2005

EVALUATION AsyncGetCallTrace() should never, ever try to allocate memory since it is called from a signal handler. I suspect that recent changes to either: jniIdSupport::to_jmethod_id_or_null() methodOopDesc::find_jmethod_id_or_null() has broken AsyncGetCallTrace(). Robert recently rewrite JNI ID support and I reviewed so I'm not sure how this slipped through the cracks. ###@###.### 2005-04-04 21:32:44 GMT The problem is in the new version of to_jmethod_id_or_null(). The original version was very simple: jmethodID jniIdSupport::to_jmethod_id_or_null(methodOop method_oop) { int index = method_oop->method_index(); klassOop k = method_oop->method_holder(); return (jmethodID)jniIdPrivate::id_for_or_null(k, index); } and id_for_or_null() was equally simple. // return an id if it is already allocated, otherwise return NULL. // Must be async-safe. No allocation should be done and // so handles are not used to avoid deadlock. static inline intptr_t id_for_or_null(klassOop k, int index) { jniIdMap* map = (jniIdMap*)(instanceKlass::cast(k)->jni_id_map()); if (map == NULL) { return NULL; } return map->id_for(index); } The new version of to_jmethod_id_or_null() looks equally simple: // Return an existing jmethodID for a methodOop, or NULL if none exists. jmethodID jniIdSupport::to_jmethod_id_or_null(methodOop method_oop) { jniIdCreator crt(method_oop); return crt.jmid_or_null(); } However, the jniIdCreator object has both methodHandle and instanceKlassHandle fields. The constructor allocates handles for both of those fields. I believe just the existence of the Handle fields will cause Handle allocation even if the fields are not initialized in the constructor. Unless the NULL Handle is special and the same one is always returned during implied initialization. ###@###.### 2005-04-05 23:51:31 GMT One correction: src/share/vm/runtime/handles.hpp says that Handle h; just declares the Handle and does not cause any allocation. So the existance of the methodHandle and instanceKlassHandle fields should not be an issue. Just the initialization of those fields in the constructor. That should simplify things a bit. ###@###.### 2005-04-06 00:18:18 GMT The JNI ID rewrite was done using bug ID 5109602. ###@###.### 2005-04-19 17:50:39 GMT
04-04-2005