JDK-6950537 : G1+COOPS: gcbasher crashes during evacuation failure
  • Type: Bug
  • Component: hotspot
  • Sub-Component: gc
  • Affected Version: hs18,7
  • Priority: P3
  • Status: Closed
  • Resolution: Not an Issue
  • OS: generic,solaris
  • CPU: x86
  • Submitted: 2010-05-07
  • Updated: 2013-09-18
  • Resolved: 2010-12-01
Related Reports
Duplicate :  
JDK-6957403 - G1: Bigapp crashes with SEGSEGV when COOPs are used.
Relates :  
JDK-6932480 - Crash in CompilerThread/Parser. Unloaded array klass?
Description
On Solaris/x64:


java  -d64 -server -XX:+UseG1GC -XX:+UseCompressedOops -Xmx128m -XX:+PrintGCDetails -X
X:+PrintGCTimeStamps -XX:+ShowMessageBoxOnError -jar GCBasher.jar -time:300000

Crash happens reliably in product build but not in a fastdebug build (at least
not with the set of options above).
Comments
EVALUATION This is a bug in SS12, not G1.
01-12-2010
SUGGESTED FIX For the record, another "fix" is to inject a dummy statement that uses *p in the macro: #define InstanceKlass_SPECIALIZED_OOP_REVERSE_ITERATE( \ T, start_p, count, do_oop, \ assert_fn) \ { \ T* const start = (T*)(start_p); \ T* p = start + (count); \ while (start < p) { \ --p; \ (assert_fn)(p); \ do { volatile T foo = *p; } while (0); \ do_oop; \ } \ } SS12 seems to generate the right code for the above macro. But, this might also have a performance impact.
10-06-2010
EVALUATION The bug does not seem to happen with SS11 and SS12.1, only with SS12.
10-06-2010
SUGGESTED FIX John Coomes is pointing out that dropping the optimization level for instanceKlass.cpp would have a non-trivial impact on GC performance (around 10%).
10-06-2010
SUGGESTED FIX Drop the optimization level for instanceKlass.cpp to -xO2 until the C++ compiler issue is resolved: In amd64.make: OPT_CFLAGS/instanceKlass.o = -xO2
09-06-2010
EVALUATION This is a very strange bug. I added some instrumentation to a JVM to try to figure out where it was failing and why. While looking at the generated traces I spotted something quite strange: while itearting over the HashMap$Entry entries, the oop iterator (in which we'd eventually crash) called from here: void G1CollectedHeap::drain_evac_failure_scan_stack() { assert(_evac_failure_scan_stack != NULL, "precondition"); while (_evac_failure_scan_stack->length() > 0) { oop obj = _evac_failure_scan_stack->pop(); klassOop k = obj->klass(); const char* str = k->klass_part()->internal_name(); const markOop mark = obj->mark(); _evac_failure_closure->set_region(heap_region_containing(obj)); obj->oop_iterate_backwards(_evac_failure_closure); } } was discovering 4 ref fields (excluding the klass word). But instances of HashMap$Entry only have 3 (key, value, next). As I said in the Comments section, the fourth field that the iteration would come across is the second 32-bit word on the second header word (i.e., the word next to the compressed klass pointer). I thought we'd stuff fields there, when we can. However, it doesn't seem to be the case. So, why are we scanning that field that seems to contain junk? I eventually worked out something curious: the oop_iterate_backwards() method would scan the extra field, but the oop_iterate() method would not. Having said that, the macros that do the iterations (InstanceKlass_OOP_MAP_REVERSE_ITERATE() / InstanceKlass_SPECIALIZED_OOP_REVERSE_ITERATE() and InstanceKlass_OOP_MAP_ITERATE() / InstanceKlass_SPECIALIZED_OOP_ITERATE() respectively) really looked equivalent with no obvious differences / bugs. Replacing the oop_iterate_backwards() method in the above code with oop_iterate() actually seemed to eliminate the failure. I then added some instrumentation in the InstanceKlass_SPECIALIZED_OOP_REVERSE_ITERATE() macro itself (just a conditional print statement) and the failure disappeared (!!!). After discussing this with John Cuthbertson he thought it might be a C++ optimzer issue and recommended to lower the optimization level for this file (instanceKlass.cpp). This also seems to eliminate the failure. Here's some output from my instrumentation with the original code (I print out the contents of the fields in the OopClosure passed to the oop iterator): => obj = 0x00000000fa33c8a8 java.util.HashMap$Entry mark = 0x00000000fa33c8ab *0x00000000fa33c8b0 = 0x00000000facba920 *0x00000000fa33c8c0 = 0x0000000000000000 *0x00000000fa33c8bc = 0x00000000fa6025b0 *0x00000000fa33c8b8 = 0x00000000fa33c890 *0x00000000fa33c8b4 = 0x00000000033b0eca <= 0x00000000fa33c8a8 (the last field, 0x00000000fa33c8b4, is actually part of the header and not a reference field; and it does contain junk) Here's the equivalent from a JVM that was compiled with the optimization level for instanceKlass.cpp dropped to -xO2 (the code was unchanged): => obj = 0x00000000f71bdba0 java.util.HashMap$Entry mark = 0x00000000f71bdba3 *0x00000000f71bdba8 = 0x00000000facba920 *0x00000000f71bdbb8 = 0x0000000000000000 *0x00000000f71bdbb4 = 0x00000000fa606430 *0x00000000f71bdbb0 = 0x00000000f71bdb88 <= 0x00000000f71bdba0 The iteration seems correct here. Yikes.
09-06-2010