JDK-8054224 : Recursive method that was compiled by C1 is unable to catch StackOverflowError
  • Type: Bug
  • Component: hotspot
  • Sub-Component: compiler
  • Affected Version: 8,9
  • Priority: P3
  • Status: Resolved
  • Resolution: Fixed
  • OS: generic
  • CPU: generic
  • Submitted: 2014-08-04
  • Updated: 2015-01-21
  • Resolved: 2014-08-13
The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.
JDK 8 JDK 9
8u40Fixed 9 b29Fixed
Description
Starting from JDK8-FCS following method will throw StackOverflowError when it compiled by C1:

void run() {
    try {
        run();
    } catch (Throwable t) {
        // StackOverflowError expected to be caught here
    }
}

Issue happens when StackOverflowError is thrown during constraint class resolution (in this case j.l.Throwable and it is unresolved for some reason) in Method::fast_exception_handler_bci_for.
In that case SharedRuntime::compute_compiled_exc_handler will return address of unwind handler and C1's Runtime1::exception_handler_for_pc_helper will store in ExceptionCache as handler for StackOverflowError @ run() bci.
As a result, at every further attempt to find exception handler during unwinding Runtime1::exception_handler_for_pc_helper will return unwind handler from cache and method will never catch StackOverflowError.

For C2 StackOverflowError is also occurring during Throwable resolution, but in that case stack will be unwound until there will be enough stack space to resolve Throwable and actual exception handler will be returned.

Issue could be reproduced starting from 8 FCS. It is also reproducible with latest JDK9 build (b25).
Issue could be reproduced only with C1.
Comments
Issue could be reproduced with existing regression test: compiler/6865265/StackOverflowBug.java jtreg -jdk:$JAVA_HOME -dir:$HS/test -client -Xcomp -XX:-Inline compiler/6865265/StackOverflowBug.java
04-08-2014

Issue could be reproduced with following test case: public class Test { public static void main(String args[]) { new Test().run(); } public void run() { try { run(); } catch (Throwable t) { } } } Command line to reproduce: $JAVA_HOME/bin/java -client -Xcomp -XX:CompileOnly=Test.run Test or $JAVA_HOME/bin/java -Xcomp -XX:CompileOnly=Test.run -XX:+TieredCompilation -XX:TieredStopAtLevel=2 Test TraceExceptions output: Exception <a 'java/lang/StackOverflowError'> (0xd130dc78) thrown [/HUDSON/workspace/9-2-build-linux-i586/jdk9/909/hotspot/src/share/vm/runtime/sharedRuntime.cpp, line 596] for thread 0xf5b08c00 Exception <a 'java/lang/StackOverflowError'> (0xd130dc78) thrown in compiled method <{method} {0xa38da210} 'run' '()V' in 'Test'> at PC 0xe6c70774 for thread 0xf5b08c00 Exception <a 'java/lang/StackOverflowError'> (0xd1310f28) thrown [/HUDSON/workspace/9-2-build-linux-i586/jdk9/909/hotspot/src/share/vm/runtime/javaCalls.cpp, line 382] for thread 0xf5b08c00 Exception <a 'java/lang/StackOverflowError'> (0xd1314270) thrown [/HUDSON/workspace/9-2-build-linux-i586/jdk9/909/hotspot/src/share/vm/runtime/javaCalls.cpp, line 382] for thread 0xf5b08c00 Thread 0xf5b08c00 continuing at PC 0xe6c707f8 for exception thrown at PC 0xe6c70774 Exception <a 'java/lang/StackOverflowError'> (0xd1314270) thrown in interpreter method <{method} {0xa38da198} 'main' '([Ljava/lang/String;)V' in 'Test'> at bci 15 for thread 0xf5b08c00
04-08-2014