JDK-8223537 : testlibrary_tests/ctw/ClassesListTest.java fails with Agent timeout frequently
  • Type: Bug
  • Component: hotspot
  • Sub-Component: compiler
  • Affected Version: 11,12,13
  • Priority: P2
  • Status: Closed
  • Resolution: Fixed
  • Submitted: 2019-05-08
  • Updated: 2020-04-08
  • Resolved: 2019-05-10
The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.
JDK 11 JDK 13 Other
11.0.5-oracleFixed 13 b21Fixed openjdk8u222Fixed
Related Reports
Duplicate :  
JDK-8223549 - Some test cases in vmTestbase timeout
Relates :  
JDK-8223549 - Some test cases in vmTestbase timeout
Relates :  
JDK-8222670 - pathological case of JIT recompilation and code cache bloat
Description
testlibrary_tests/ctw/ClassesListTest.java

Log: 
#-----testresult-----
description=file\:/scratch/opt/mach5/mesos/work_dir/jib-master/install/jdk-13-969/src.full/open/test/hotspot/jtreg/testlibrary_tests/ctw/ClassesListTest.java
elapsed=6040614 1\:40\:40.614
end=Tue May 07 21\:53\:55 UTC 2019
environment=regtest
execStatus=Error. Agent error\: java.lang.Exception\: Agent 5 timed out with a timeout of 6000 seconds; check console log for any additional details

----------System.err:(3/35)----------

JavaTest Message: Test complete.

result: Error. Agent error: java.lang.Exception: Agent 5 timed out with a timeout of 6000 seconds; check console log for any additional details
Comments
backport to jdk11u. only adjust line number. https://cr.openjdk.java.net/~xliu/8223537/webrev.11u-dev/
07-06-2019
Testing tier1-4 passed clean with 2 known failures. One is JDK-8223363 and other some testing environment issue: "The process cannot access the file because it is being used by another process."
10-05-2019
Thanks. I will run these 2 combinations if I touch compiler code.
09-05-2019
https://cr.openjdk.java.net/~xliu/8223537/webrev.01/
09-05-2019
Tier6 run tests with next 2 combinations of flags (Tiered on/off): -Xcomp -ea -esa -XX:CompileThreshold=100 -XX:+TieredCompilation -Xcomp -ea -esa -XX:CompileThreshold=100 -XX:-TieredCompilation You can run jtreg tests with -javaoptions:'<flags>' to have the same testing.
09-05-2019
I tested locally the patch and result is good.
09-05-2019
hi, Tobias, Vladimir, Thanks. I sent out webrev. What's tier6? Do you have any suggest how I can prevent this next time? To be honest, I don't know -Xcomp before. Can I manually trigger -Xcomp test on my side.
09-05-2019
So the thread that issued the compile task is blocking in CompileBroker::wait_for_completion() until task->is_complete(). With your fix for JDK-8222670, such a blocking compile task could be marked as stale and is removed from the queue while the issuing thread is still waiting for its completion. Now calling CompileQueue::purge_stale_tasks() will set the task to complete through CompileTaskWrapper and notify the waiting thread. Your fix looks good to me, please send it out for review.
09-05-2019
this is the draft webrev. could you take a look at it? https://cr.openjdk.java.net/~xliu/8223537/webrev/
09-05-2019
The following hotfix should solve -Xcomp issue. The basic idea purge stale tasks after select_task. it could wake up java threads who are waiting for result. diff -r 6fbbf466fc6d src/hotspot/share/compiler/compileBroker.cpp --- a/src/hotspot/share/compiler/compileBroker.cpp Thu May 09 03:11:56 2019 +0200 +++ b/src/hotspot/share/compiler/compileBroker.cpp Thu May 09 00:25:35 2019 -0700 @@ -449,8 +449,8 @@ save_hot_method = methodHandle(task->hot_method()); remove(task); - purge_stale_tasks(); // may temporarily release MCQ lock } + purge_stale_tasks(); // may temporarily release MCQ lock return task; } $jtreg -testjdk:./images/jdk -javaoptions:'-Xcomp -ea -esa -XX:CompileThreshold=100 -XX:+TieredCompilation' ~/Devel/openjdk/jdk/test/hotspot/jtreg/testlibrary_tests/ctw/ClassesListTest.java Test results: passed: 1 Report written to /Users/xxinliu/Devel/openjdk/jdk/build/macosx-x86_64-server-fastdebug/JTreport/html/report.html Results written to /Users/xxinliu/Devel/openjdk/jdk/build/macosx-x86_64-server-fastdebug/JTwork $jtreg -testjdk:./images/jdk -javaoptions:'-Xcomp -ea -esa -XX:CompileThreshold=100 -XX:+TieredCompilation' ~/Devel/openjdk/jdk/test/hotspot/jtreg/compiler/tiered/Level2RecompilationTest.java Test results: passed: 1
09-05-2019
I knew the reason. -Xcomp is blocking mode. it will wait for completion in CompileBroker::wait_for_completion. in JDK-8222670, I invoked remove_and_mark_stale for the task. compile_queue->remove_and_mark_stale(max_task); even the task is stale, it is not completed. it stuck there.
09-05-2019
sorry for the regression. it seems to relate to '-Xcomp'. with it, hotspot hang there. please backout it.
09-05-2019
taking a look.
09-05-2019
To run test: jtreg -testjdk:<your jdk> -va -javaoptions:'-Xcomp -ea -esa -XX:CompileThreshold=100 -XX:+TieredCompilation' testlibrary_tests/ctw/ClassesListTest.java
09-05-2019
Xin, please look. We may backout your JDK-8222670 changes.
09-05-2019
And results are ... JDK-8222670 caused it.
09-05-2019
Failed first time it was in jdk-13-966. There were 5 changesets but suspicious ones: 8222670 pathological case of JIT recompilation and code cache bloat 8223332 Update JVMCI To find which caused it I submitted 3 test jobs with build 964, 965, 966.
09-05-2019
Looking on failures and it fails when run with Tiered -Xcomp -ea -esa -XX:CompileThreshold=100 -XX:+TieredCompilation. And it passed with C2 only: -XX:-TieredCompilation
09-05-2019
Biggest change around then was: JDK-8223332 Update JVMCI
08-05-2019