JDK-8249217 : Unexpected StackOverflowError in "process reaper" thread still happens
  • Type: Bug
  • Component: core-libs
  • Sub-Component: java.lang
  • Affected Version: 15,16
  • Priority: P4
  • Status: Resolved
  • Resolution: Fixed
  • OS: linux
  • CPU: x86_64
  • Submitted: 2020-07-11
  • Updated: 2020-09-24
  • Resolved: 2020-07-24
The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.
JDK 15 JDK 16
15.0.2Fixed 16 b08Fixed
Related Reports
Relates :  
JDK-8217475 - Unexpected StackOverflowError in "process reaper" thread
Sub Tasks
JDK-8250523 :  
Remove abortOnException diagnostic option from TestHumongousNonArrayAllocation.java - Resolved
Description
The following test failed in the JDK16 CI:

runtime/cds/appcds/dynamicArchive/LambdaProxyCallerIsHidden.java

Here's a snippet from the log file:

#section:main
----------messages:(7/481)----------
command: main -XX:+UnlockDiagnosticVMOptions -XX:+WhiteBoxAPI -Xbootclasspath/a:. LambdaProxyCallerIsHidden
reason: User specified action: run main/othervm -XX:+UnlockDiagnosticVMOptions -XX:+WhiteBoxAPI -Xbootclasspath/a:. LambdaProxyCallerIsHidden 
Mode: othervm [/othervm specified]
Additional options from @modules: --add-modules java.base --add-exports java.base/jdk.internal.misc=ALL-UNNAMED
Timeout information:
--- Timeout information end.
elapsed time (seconds): 1225.829
----------configuration:(4/111)----------
Boot Layer
  add modules: java.base                   
  add exports: java.base/jdk.internal.misc ALL-UNNAMED

----------System.out:(9/3762)----------
Command line: [/opt/mach5/mesos/work_dir/jib-master/install/jdk-16+6-180/linux-x64-debug.jdk/jdk-16/fastdebug/bin/java -cp /opt/mach5/mesos/work_dir/slaves/4728e7c1-7e67-490e-be0f-6bbf2a2f33db-S312/frameworks/1735e8a2-a1db-478c-8104-60c8b0af87dd-0196/executors/233971cd-2d9e-4b70-862f-ea2f2e53d608/runs/0af541cb-a880-4872-a77e-203af1fb1a78/testoutput/test-support/jtreg_open_test_hotspot_jtreg_hotspot_runtime/classes/0/runtime/cds/appcds/dynamicArchive/LambdaProxyCallerIsHidden.d:/opt/mach5/mesos/work_dir/jib-master/install/jdk-16+6-180/src.full/open/test/hotspot/jtreg/runtime/cds/appcds/dynamicArchive:/opt/mach5/mesos/work_dir/slaves/4728e7c1-7e67-490e-be0f-6bbf2a2f33db-S312/frameworks/1735e8a2-a1db-478c-8104-60c8b0af87dd-0196/executors/233971cd-2d9e-4b70-862f-ea2f2e53d608/runs/0af541cb-a880-4872-a77e-203af1fb1a78/testoutput/test-support/jtreg_open_test_hotspot_jtreg_hotspot_runtime/classes/0/test/lib:/opt/mach5/mesos/work_dir/jib-master/install/jdk-16+6-180/src.full/open/test/lib:/opt/mach5/mesos/work_dir/slaves/4728e7c1-7e67-490e-be0f-6bbf2a2f33db-S312/frameworks/1735e8a2-a1db-478c-8104-60c8b0af87dd-0196/executors/233971cd-2d9e-4b70-862f-ea2f2e53d608/runs/0af541cb-a880-4872-a77e-203af1fb1a78/testoutput/test-support/jtreg_open_test_hotspot_jtreg_hotspot_runtime/classes/0/test/hotspot/jtreg/runtime/cds/appcds:/opt/mach5/mesos/work_dir/jib-master/install/jdk-16+6-180/src.full/open/test/hotspot/jtreg/runtime/cds/appcds:/opt/mach5/mesos/work_dir/slaves/4728e7c1-7e67-490e-be0f-6bbf2a2f33db-S312/frameworks/1735e8a2-a1db-478c-8104-60c8b0af87dd-0196/executors/233971cd-2d9e-4b70-862f-ea2f2e53d608/runs/0af541cb-a880-4872-a77e-203af1fb1a78/testoutput/test-support/jtreg_open_test_hotspot_jtreg_hotspot_runtime/classes/0/test/hotspot/jtreg/runtime/cds/appcds/dynamicArchive/test-classes:/opt/mach5/mesos/work_dir/jib-master/install/jdk-16+6-180/src.full/open/test/hotspot/jtreg/runtime/cds/appcds/dynamicArchive/test-classes:/opt/mach5/mesos/work_dir/jib-master/install/jtreg/5.1/b01/bundles/jtreg_bin-5.1.zip/jtreg/lib/javatest.jar:/opt/mach5/mesos/work_dir/jib-master/install/jtreg/5.1/b01/bundles/jtreg_bin-5.1.zip/jtreg/lib/jtreg.jar -XX:MaxRAMPercentage=6 -Djava.io.tmpdir=/opt/mach5/mesos/work_dir/slaves/4728e7c1-7e67-490e-be0f-6bbf2a2f33db-S312/frameworks/1735e8a2-a1db-478c-8104-60c8b0af87dd-0196/executors/233971cd-2d9e-4b70-862f-ea2f2e53d608/runs/0af541cb-a880-4872-a77e-203af1fb1a78/testoutput/test-support/jtreg_open_test_hotspot_jtreg_hotspot_runtime/tmp -Xcomp -XX:+CreateCoredumpOnCrash -ea -esa -XX:CompileThreshold=100 -XX:+UnlockExperimentalVMOptions -server -XX:+TieredCompilation -XX:ArchiveClassesAtExit=/opt/mach5/mesos/work_dir/slaves/4728e7c1-7e67-490e-be0f-6bbf2a2f33db-S312/frameworks/1735e8a2-a1db-478c-8104-60c8b0af87dd-0196/executors/233971cd-2d9e-4b70-862f-ea2f2e53d608/runs/0af541cb-a880-4872-a77e-203af1fb1a78/testoutput/test-support/jtreg_open_test_hotspot_jtreg_hotspot_runtime/scratch/3/appcds-10h14m04s696.jsa -Xshare:auto -Xlog:class+load,cds+dynamic,cds -cp /opt/mach5/mesos/work_dir/slaves/4728e7c1-7e67-490e-be0f-6bbf2a2f33db-S312/frameworks/1735e8a2-a1db-478c-8104-60c8b0af87dd-0196/executors/233971cd-2d9e-4b70-862f-ea2f2e53d608/runs/0af541cb-a880-4872-a77e-203af1fb1a78/testoutput/test-support/jtreg_open_test_hotspot_jtreg_hotspot_runtime/scratch/3/callerishidden.jar LambdaProxyCallerIsHiddenApp ]
[2020-07-11T10:14:07.961808666Z] Gathering output for process 11841
[ELAPSED: 51730 ms]
[logging stdout to runtime.cds.appcds.dynamicArchive.LambdaProxyCallerIsHidden.java-0000-dump.stdout]
[logging stderr to runtime.cds.appcds.dynamicArchive.LambdaProxyCallerIsHidden.java-0000-dump.stderr]
[STDERR]

[2020-07-11T10:14:58.082054979Z] Waiting for completion for process 11841
Timeout refired 1200 times
----------System.err:(15/1348)----------
Exception in thread "process reaper" java.lang.StackOverflowError
	at java.base/java.util.concurrent.ConcurrentHashMap.fullAddCount(ConcurrentHashMap.java:2584)
	at java.base/java.util.concurrent.ConcurrentHashMap.addCount(ConcurrentHashMap.java:2334)
	at java.base/java.util.concurrent.ConcurrentHashMap.putVal(ConcurrentHashMap.java:1075)
	at java.base/java.util.concurrent.ConcurrentHashMap.putIfAbsent(ConcurrentHashMap.java:1541)
	at java.base/java.lang.invoke.MethodType$ConcurrentWeakInternSet.add(MethodType.java:1400)
	at java.base/java.lang.invoke.MethodType.makeImpl(MethodType.java:359)
	at java.base/java.lang.invoke.MethodHandleNatives.findMethodHandleType(MethodHandleNatives.java:389)
	at java.base/java.util.concurrent.CompletableFuture.completeValue(CompletableFuture.java:305)
	at java.base/java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:2136)
	at java.base/java.lang.ProcessHandleImpl$1.run(ProcessHandleImpl.java:166)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630)
	at java.base/java.lang.Thread.run(Thread.java:832)
Java HotSpot(TM) 64-Bit Server VM warning: outputStream::do_vsnprintf output truncated -- buffer length is 2000 bytes but 2058 bytes are needed.
----------rerun:(49/10273)*----------

Here's the description of the test task:

Run test open/test/hotspot/jtreg/:hotspot_runtime with linux-x64-debug with -Xcomp -XX:+CreateCoredumpOnCrash -ea -esa -XX:CompileThreshold=100 -XX:+UnlockExperimentalVMOptions -server -XX:+TieredCompilation #tier4-comp

I don't know if any of the options affect the reproducibility of
this latest failure.

Update: This failure occurred in jdk-16+6-180-tier4 which includes
the fix for:

 JDK-8217475 Unexpected StackOverflowError in "process reaper" thread
Comments
URL: https://hg.openjdk.java.net/jdk/jdk/rev/1f37a5cd6afc User: rriggs Date: 2020-07-24 15:56:17 +0000
24-07-2020
The class initialization of java.util.concurrent.ThreadLocalRandom is consistently on the native stack when the StackOverflowError occurs. java.util.ConcurrentHashMap class initialization is also implicated. With a preemptive class initialization of ThreadLocalRandom and ConcurrentHashMap the stackoverflow exceptions have not been seen in overnight testing using the test hotspot/jtreg/gc/g1/humongousObjects/TestHumongousNonArrayAllocation.
22-07-2020
Another failing test is: runtime/cds/appcds/dynamicArchive/UnsafeAnonymous.java I have to say I'm no longer confident that I know which JVM the exception is coming from. is it the JVM under test or one of jtreg's internally launched JVMs?
16-07-2020
Currently there are 7 sightings linked to this bug. There are no duplicates so there isn't a clear indication about which test would be best to try and "trap and trace".
14-07-2020
It would be useful to get a hs_err file when this happens. Would it be reasonable to add " -XX:+UnlockDiagnosticVMOptions -XX:AbortVMOnException=java.lang.StackOverflowError" to TestHumongousNonArrayAllocation which also fails intermittently?
14-07-2020