JDK-8212637 : C2 Stuck in C2Compiler::compile_method with 100%CPU usage for one week
  • Type: Bug
  • Component: hotspot
  • Sub-Component: compiler
  • Affected Version: 8u172,openjdk8u242,openjdk8u272
  • Priority: P4
  • Status: Open
  • Resolution: Unresolved
  • OS: linux
  • CPU: generic
  • Submitted: 2018-10-17
  • Updated: 2024-07-03
Related Reports
Relates :  
JDK-8214939 - C2 Compiler process is stuck in MemNode::can_see_stored_value()
Description
ADDITIONAL SYSTEM INFORMATION :
RedHat 7.3 / jre1.8.0_172-amd64 / Spring stack

A DESCRIPTION OF THE PROBLEM :
For one week now we have the C2 compiler thread stuck on C2Compiler::compile_method for org.springframework.core.ResolvableType#forMethodParameter.

We had analyze this problem like here https://medium.com/@vladimirsitniko/analyzing-a-stuck-hotspot-c2-compilation-85e0ca230744 and had the same behavior, same thread stack, same method with is been compiled, ....

Maybe it's resolved on earlier version of the jre but we can't find any references in release notes.


FREQUENCY : rarely
Comments
Hi [~dongbohe], I tried but didn't get it. This problem is only in production environment, there is no necessary debuginfo, and a related coredump was deleted by mistake. The problem seems to be that Proj's input is actually a MergeMem, while this MergeMem is also a use of Proj, which causes an infinite loop in MemNode::can_see_stored_value. I uploaded some pictures to describe the ill-formed IR. We have a trick workaround internally but didn't really fix it.
07-03-2022
[~yyang] Hi, Yangyi. Do you have any new discoveries there? and do you have the C2 compile log?
19-02-2022
We have rarely encountered it several times in the past. Today it appears in production env again (1.8.0_242). We will keep this machine as long-living as possible. I am able to investigate it, if you are also interested in this problem/need more information about it, please comment on this. Thanks! --- get stuck in the following stack: Thread 1907 (Thread 0xffff62bff190 (LWP 1941)): #0 0x0000ffff9aa72e48 in MemNode::can_see_stored_value(Node*, PhaseTransform*) const () from /opt/taobao/install/ajdk-8_10_15_fp11-b50-dep/jre/lib/aarch64/server/libjvm.so #1 0x0000ffff9aa73b54 in LoadNode::Value(PhaseTransform*) const () from /opt/taobao/install/ajdk-8_10_15_fp11-b50-dep/jre/lib/aarch64/server/libjvm.so #2 0x0000ffff9ab564b8 in PhaseIterGVN::transform_old(Node*) () from /opt/taobao/install/ajdk-8_10_15_fp11-b50-dep/jre/lib/aarch64/server/libjvm.so #3 0x0000ffff9ab545a0 in PhaseIterGVN::optimize() () from /opt/taobao/install/ajdk-8_10_15_fp11-b50-dep/jre/lib/aarch64/server/libjvm.so #4 0x0000ffff9a6b8f58 in Compile::Optimize() () from /opt/taobao/install/ajdk-8_10_15_fp11-b50-dep/jre/lib/aarch64/server/libjvm.so #5 0x0000ffff9a6ba75c in Compile::Compile(ciEnv*, C2Compiler*, ciMethod*, int, bool, bool, bool) () from /opt/taobao/install/ajdk-8_10_15_fp11-b50-dep/jre/lib/aarch64/server/libjvm.so #6 0x0000ffff9a60f1dc in C2Compiler::compile_method(ciEnv*, ciMethod*, int) () from /opt/taobao/install/ajdk-8_10_15_fp11-b50-dep/jre/lib/aarch64/server/libjvm.so #7 0x0000ffff9a6c242c in CompileBroker::invoke_compiler_on_method(CompileTask*) () from /opt/taobao/install/ajdk-8_10_15_fp11-b50-dep/jre/lib/aarch64/server/libjvm.so #8 0x0000ffff9a6c31e8 in CompileBroker::compiler_thread_loop() () from /opt/taobao/install/ajdk-8_10_15_fp11-b50-dep/jre/lib/aarch64/server/libjvm.so #9 0x0000ffff9ac53d54 in JavaThread::thread_main_inner() () from /opt/taobao/install/ajdk-8_10_15_fp11-b50-dep/jre/lib/aarch64/server/libjvm.so #10 0x0000ffff9ac53eb8 in JavaThread::run() () from /opt/taobao/install/ajdk-8_10_15_fp11-b50-dep/jre/lib/aarch64/server/libjvm.so #11 0x0000ffff9aafe918 in java_start(Thread*) () from /opt/taobao/install/ajdk-8_10_15_fp11-b50-dep/jre/lib/aarch64/server/libjvm.so #12 0x0000ffff9b848d38 in start_thread () from /lib64/libpthread.so.0 #13 0x0000ffff9b727680 in thread_start () from /lib64/libc.so.6 Thread 1906 (Thread 0xffff627fe190 (LWP 1942)): #0 0x0000ffff9a6846c8 in MemBarCPUOrderNode::Opcode() const () from /opt/taobao/install/ajdk-8_10_15_fp11-b50-dep/jre/lib/aarch64/server/libjvm.so #1 0x0000ffff9aa72e24 in MemNode::can_see_stored_value(Node*, PhaseTransform*) const () from /opt/taobao/install/ajdk-8_10_15_fp11-b50-dep/jre/lib/aarch64/server/libjvm.so #2 0x0000ffff9aa73b54 in LoadNode::Value(PhaseTransform*) const () from /opt/taobao/install/ajdk-8_10_15_fp11-b50-dep/jre/lib/aarch64/server/libjvm.so #3 0x0000ffff9ab564b8 in PhaseIterGVN::transform_old(Node*) () from /opt/taobao/install/ajdk-8_10_15_fp11-b50-dep/jre/lib/aarch64/server/libjvm.so #4 0x0000ffff9ab545a0 in PhaseIterGVN::optimize() () from /opt/taobao/install/ajdk-8_10_15_fp11-b50-dep/jre/lib/aarch64/server/libjvm.so #5 0x0000ffff9a6b8f58 in Compile::Optimize() () from /opt/taobao/install/ajdk-8_10_15_fp11-b50-dep/jre/lib/aarch64/server/libjvm.so #6 0x0000ffff9a6ba75c in Compile::Compile(ciEnv*, C2Compiler*, ciMethod*, int, bool, bool, bool) () from /opt/taobao/install/ajdk-8_10_15_fp11-b50-dep/jre/lib/aarch64/server/libjvm.so #7 0x0000ffff9a60f1dc in C2Compiler::compile_method(ciEnv*, ciMethod*, int) () from /opt/taobao/install/ajdk-8_10_15_fp11-b50-dep/jre/lib/aarch64/server/libjvm.so #8 0x0000ffff9a6c242c in CompileBroker::invoke_compiler_on_method(CompileTask*) () from /opt/taobao/install/ajdk-8_10_15_fp11-b50-dep/jre/lib/aarch64/server/libjvm.so #9 0x0000ffff9a6c31e8 in CompileBroker::compiler_thread_loop() () from /opt/taobao/install/ajdk-8_10_15_fp11-b50-dep/jre/lib/aarch64/server/libjvm.so #10 0x0000ffff9ac53d54 in JavaThread::thread_main_inner() () from /opt/taobao/install/ajdk-8_10_15_fp11-b50-dep/jre/lib/aarch64/server/libjvm.so #11 0x0000ffff9ac53eb8 in JavaThread::run() () from /opt/taobao/install/ajdk-8_10_15_fp11-b50-dep/jre/lib/aarch64/server/libjvm.so #12 0x0000ffff9aafe918 in java_start(Thread*) () from /opt/taobao/install/ajdk-8_10_15_fp11-b50-dep/jre/lib/aarch64/server/libjvm.so #13 0x0000ffff9b848d38 in start_thread () from /lib64/libpthread.so.0 #14 0x0000ffff9b727680 in thread_start () from /lib64/libc.so.6 Thread 1905 (Thread 0xffff61fff190 (LWP 1943)): #0 0x0000ffff9aa72e60 in MemNode::can_see_stored_value(Node*, PhaseTransform*) const () from /opt/taobao/install/ajdk-8_10_15_fp11-b50-dep/jre/lib/aarch64/server/libjvm.so #1 0x0000ffff9aa73b54 in LoadNode::Value(PhaseTransform*) const () from /opt/taobao/install/ajdk-8_10_15_fp11-b50-dep/jre/lib/aarch64/server/libjvm.so #2 0x0000ffff9ab564b8 in PhaseIterGVN::transform_old(Node*) () from /opt/taobao/install/ajdk-8_10_15_fp11-b50-dep/jre/lib/aarch64/server/libjvm.so #3 0x0000ffff9ab545a0 in PhaseIterGVN::optimize() () from /opt/taobao/install/ajdk-8_10_15_fp11-b50-dep/jre/lib/aarch64/server/libjvm.so #4 0x0000ffff9a6b8f58 in Compile::Optimize() () from /opt/taobao/install/ajdk-8_10_15_fp11-b50-dep/jre/lib/aarch64/server/libjvm.so #5 0x0000ffff9a6ba75c in Compile::Compile(ciEnv*, C2Compiler*, ciMethod*, int, bool, bool, bool) () from /opt/taobao/install/ajdk-8_10_15_fp11-b50-dep/jre/lib/aarch64/server/libjvm.so #6 0x0000ffff9a60f1dc in C2Compiler::compile_method(ciEnv*, ciMethod*, int) () from /opt/taobao/install/ajdk-8_10_15_fp11-b50-dep/jre/lib/aarch64/server/libjvm.so #7 0x0000ffff9a6c242c in CompileBroker::invoke_compiler_on_method(CompileTask*) () from /opt/taobao/install/ajdk-8_10_15_fp11-b50-dep/jre/lib/aarch64/server/libjvm.so #8 0x0000ffff9a6c31e8 in CompileBroker::compiler_thread_loop() () from /opt/taobao/install/ajdk-8_10_15_fp11-b50-dep/jre/lib/aarch64/server/libjvm.so #9 0x0000ffff9ac53d54 in JavaThread::thread_main_inner() () from /opt/taobao/install/ajdk-8_10_15_fp11-b50-dep/jre/lib/aarch64/server/libjvm.so #10 0x0000ffff9ac53eb8 in JavaThread::run() () from /opt/taobao/install/ajdk-8_10_15_fp11-b50-dep/jre/lib/aarch64/server/libjvm.so #11 0x0000ffff9aafe918 in java_start(Thread*) () from /opt/taobao/install/ajdk-8_10_15_fp11-b50-dep/jre/lib/aarch64/server/libjvm.so #12 0x0000ffff9b848d38 in start_thread () from /lib64/libpthread.so.0 #13 0x0000ffff9b727680 in thread_start () from /lib64/libc.so.6
22-11-2021
Re-open the issue because the problem has reappeared
30-09-2021
In our production environment, this problem has appeared twice recently, once with 8u252 and once with 8u272 (about half a year later). The compiled methods are `org/springframework/core/ResolvableType.forMethodParameter(Lorg/springframework/core/MethodParameter;)Lorg/springframework/core/ResolvableType;` and `org/springframework/core/ResolvableType.forMethodParameter(Lorg/springframework/core/MethodParameter;Ljava/lang/reflect/Type:I)LLorg/springframework/core/ResolvableType;` respectively. Stack trace: #0 0x00007f77d5731f7f in MemNode::can_see_stored_value (this=this@entry=0x7f77a90abed8, st=st@entry=0x7f77a904ceb0, phase=phase@entry=0x7f77c85d4e90) at /usr1/jenkins/CloudCompilerJDK/open_source/openjdk/aarch64-shenandoah-jdk8u272-b10/hotspot/src/share/vm/opto/memnode.cpp:817 #1 0x00007f77d5732b86 in LoadNode::Value (this=0x7f77a90abed8, phase=0x7f77c85d4e90) at /usr1/jenkins/CloudCompilerJDK/open_source/openjdk/aarch64-shenandoah-jdk8u272-b10/hotspot/src/share/vm/opto/memnode.cpp:1752 #2 0x00007f77d5813b01 in PhaseIterGVN::transform_old (this=0x7f77c85d4e90, n=<optimized out>) at /usr1/jenkins/CloudCompilerJDK/open_source/openjdk/aarch64-shenandoah-jdk8u272-b10/hotspot/src/share/vm/opto/phaseX.cpp:1182 #3 0x00007f77d5811bfc in PhaseIterGVN::optimize (this=this@entry=0x7f77c85d4e90) at /usr1/jenkins/CloudCompilerJDK/open_source/openjdk/aarch64-shenandoah-jdk8u272-b10/hotspot/src/share/vm/opto/phaseX.cpp:1003 #4 0x00007f77d52fc112 in Compile::Optimize (this=this@entry=0x7f77c85d6f80) at /usr1/jenkins/CloudCompilerJDK/open_source/openjdk/aarch64-shenandoah-jdk8u272-b10/hotspot/src/share/vm/opto/compile.cpp:2142 #5 0x00007f77d52fe704 in Compile::Compile (this=0x7f77c85d6f80, ci_env=<optimized out>, compiler=0x7f77d01867e0, target=<optimized out>, osr_bci=<optimized out>, subsume_loads=<optimized out>, do_escape_analysis=true, eliminate_boxing=true) at /usr1/jenkins/CloudCompilerJDK/open_source/openjdk/aarch64-shenandoah-jdk8u272-b10/hotspot/src/share/vm/opto/compile.cpp:878 #6 0x00007f77d524bdd4 in C2Compiler::compile_method (this=0x7f77d01867e0, env=0x7f77c85d7b10, target=0x7f77a8076ff0, entry_bci=-1) at /usr1/jenkins/CloudCompilerJDK/open_source/openjdk/aarch64-shenandoah-jdk8u272-b10/hotspot/src/share/vm/opto/c2compiler.cpp:118 #7 0x00007f77d5307ec2 in CompileBroker::invoke_compiler_on_method ( task=task@entry=0x7f7794003e80) at /usr1/jenkins/CloudCompilerJDK/open_source/openjdk/aarch64-shenandoah-jdk8u272-b10/hotspot/src/share/vm/compiler/compileBroker.cpp:2012 #8 0x00007f77d5309a88 in CompileBroker::compiler_thread_loop () at /usr1/jenkins/CloudCompilerJDK/open_source/openjdk/aarch64-shenandoah-jdk8u272-b10/hotspot/src/share/vm/compiler/compileBroker.cpp:1817 #9 0x00007f77d5984c51 in JavaThread::thread_main_inner (this=0x7f77d0187000) at /usr1/jenkins/CloudCompilerJDK/open_source/openjdk/aarch64-shenandoah-jdk8u272-b10/hotspot/src/share/vm/runtime/thread.cpp:1735 #10 0x00007f77d57c52d8 in java_start (thread=0x7f77d0187000) at /usr1/jenkins/CloudCompilerJDK/open_source/openjdk/aarch64-shenandoah-jdk8u272-b10/hotspot/src/os/linux/vm/os_linux.cpp:845
30-09-2021
Closing as Incomplete for now as there is no reproducer to make any progress on this.
17-12-2018
Looks similar to the one JDK-8214939, as we can see both stuck at "MemNode::can_see_stored_value(Node*, PhaseTransform*)"
06-12-2018
More details about the issue has been discussed here - https://medium.com/@vladimirsitniko/analyzing-a-stuck-hotspot-c2-compilation-85e0ca230744 It is not very clear from the discription that, if the same issue has been fixed in higher versions? Submitter is trying on 8u172. Please verify the issue against JDK 11 latest build [1] or early access build of JDK12 [2] Kindly share your results with us [1] https://www.oracle.com/technetwork/java/javase/downloads/jdk11-downloads-5066655.html [2] http://jdk.java.net/12/
18-10-2018