Bug ID: JDK-8075215 SATB buffer processing found reclaimed humongous object

JDK-8075215 : SATB buffer processing found reclaimed humongous object

Type: Bug
Component: hotspot
Sub-Component: gc
Affected Version: 8

Priority: P1
Status: Closed
Resolution: Fixed

Submitted: 2015-03-16
Updated: 2017-06-22
Resolved: 2015-05-18

Versions (Unresolved/Resolved/Fixed)

The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.

JDK 8	JDK 9
8u60 b17Fixed	9Fixed

Related Reports

Blocks :	JDK-8078021 - SATB apply_closure_to_completed_buffer should have closure argument
Blocks :	JDK-8077383 - Re-enable eager reclaim by default
Blocks :	JDK-8078848 - Early reclaim of large objects that are referenced by a few objects
Blocks :	JDK-8078853 - SATB apply_closure_to_completed_buffer should have closure argument
Blocks :	JDK-8079167 - G1SATBBufferEnqueueingThresholdPercent == 0 should disable filtering
Relates :	JDK-8073463 - G1 does not mangle freed heap regions
Relates :	JDK-8077314 - Assertion fails on reference in free region in SATB buffer
Relates :	JDK-8027959 - Early reclamation of large objects in G1
Relates :	JDK-8075466 - SATB queue pre-filter verify found reclaimed humongous object
Relates :	JDK-8069367 - Eagerly reclaimed humongous objects left on mark stack

Description

#  Internal Error (/hotspot/src/share/vm/gc_implementation/g1/concurrentMark.inline.hpp:280), pid=17184, tid=139891857815296
#  assert(obj->is_oop_or_null(true )) failed: Expected an oop or NULL at 0x000000048c400000
#
# JRE version: Java(TM) SE Runtime Environment (9.0) (build 1.9.0-internal-fastdebug-20150313150524.iignatye.8026047-b00)
# Java VM: Java HotSpot(TM) 64-Bit Server VM (1.9.0-internal-fastdebug-20150313150524.iignatye.8026047-b00 mixed mode linux-amd64 compressed oops)

Stack trace:
V  [libjvm.so+0x108e361]  VMError::report_and_die()+0x151;;  VMError::report_and_die()+0x151
V  [libjvm.so+0x73af4b]  report_vm_error(char const*, int, char const*, char const*)+0x7b;;  report_vm_error(char const*, int, char const*, char const*)+0x7b
V  [libjvm.so+0x6eb3f8]  CMTask::deal_with_reference(oop)+0xf8;;  CMTask::deal_with_reference(oop)+0xf8
V  [libjvm.so+0x6ebe53]  CMObjectClosure::do_object(oop)+0x33;;  CMObjectClosure::do_object(oop)+0x33
V  [libjvm.so+0xf00d78]  ObjPtrQueue::apply_closure_to_buffer(ObjectClosure*, void**, unsigned long, unsigned long)+0x78;;  ObjPtrQueue::apply_closure_to_buffer(ObjectClosure*, void**, unsigned long, unsigned long)+0x78
V  [libjvm.so+0xf016f0]  SATBMarkQueueSet::apply_closure_to_completed_buffer(unsigned int)+0x80;;  SATBMarkQueueSet::apply_closure_to_completed_buffer(unsigned int)+0x80
V  [libjvm.so+0x6d873b]  CMTask::drain_satb_buffers() [clone .part.188]+0x9b;;  _ZN6CMTask18drain_satb_buffersEv.part.188+0x9b
V  [libjvm.so+0x6e2ed5]  CMTask::do_marking_step(double, bool, bool)+0x275;;  CMTask::do_marking_step(double, bool, bool)+0x275
V  [libjvm.so+0x6ed469]  CMRemarkTask::work(unsigned int)+0x2b9;;  CMRemarkTask::work(unsigned int)+0x2b9
V  [libjvm.so+0x10df200]  GangWorker::loop()+0x2b0;;  GangWorker::loop()+0x2b0
V  [libjvm.so+0xddee42]  java_start(Thread*)+0xf2;;  java_start(Thread*)+0xf2

Comments

Updated summary to better describe the problem.
28-04-2015
ILW = High (crash), High (happens frequently), Medium (run without eager reclamation) = P1
01-04-2015
Possible workaround: disable eager reclaim using -XX:-G1EagerReclaimHumongousObjects
01-04-2015
I agree with that, and that the improved empty space mangling is not the cause of the problem. I was merely trying to explain the reason why the assert fails, and the potential impact on the product as far as known at this point.
01-04-2015
I would argue that the assert is correct and that it's incorrect to have reclaimed objects in the SATB queues, but that's a discussion for another bug.
01-04-2015
The cause for this issue surfacing is the fix for empty space mangling in JDK-8073463. Previously these assertions just found a reclaimed but still valid humongous object (look at the address of the oop) at the location, passing the assert. Which also means that this is an (annoying) debug-only problem.
01-04-2015
Happened twice while running an adhoc test run. The ILW was motivated by this "happened once". Can we revise the ILW now that this seems to be happening quite often?
01-04-2015
I think Thomas is mostly correct, except that I don't think it is necessary for the memory to be overwritten by a new object. Space mangling when the reclaimed object's region was freed would be sufficient.
18-03-2015
ILW = High (crash), Low (happened once), Medium (run without eager reclamation) = P3
18-03-2015
Potentially caused by eager reclaim (JDK-8027959): the SATB buffers may contain a reference to a formerly reclaimed object which memory has been overwritten by a new object, making it point into a humongous continues region. In this particular case, it seems that that object also has been removed already, i.e. is a free region. So that referenced memory does not contain a valid klass that causes the check in oopDesc::is:oop(), line 566 to fail (i.e. if (Universe::heap()->is_in_reserved(obj->klass_or_null())) return false; ). If that is the case, I think this is a benign problem for product: this reference will be filtered out later in the TAMS check. Both free and humongous continues regions have a TAMS==bottom (iirc).
16-03-2015