JDK-8192853 : [Graal] org.graalvm.compiler tests fail with Graal as JIT compiler and -Xcomp flag
  • Type: Bug
  • Component: hotspot
  • Sub-Component: compiler
  • Affected Version: 10
  • Priority: P3
  • Status: Resolved
  • Resolution: Won't Fix
  • Submitted: 2017-11-30
  • Updated: 2019-09-13
  • Resolved: 2019-06-03
Related Reports
Duplicate :  
JDK-8222551 - [Graal] org.graalvm.compiler.hotspot.test.ArrayNewInstanceTest fails with java.lang.AssertionError
Duplicate :  
JDK-8222554 - [Graal] org.graalvm.compiler.jtt.except.BC_checkcast3 fails with org.junit.ComparisonFailure
Duplicate :  
JDK-8222748 - [Graal] org.graalvm.compiler.core.test.inlining.PolymorphicInliningTest fails with AssertionError
Relates :  
JDK-8223924 - [Graal] org.graalvm.compiler.api.directives.test.ProbabilityDirectiveTest fails with -Xcomp
Relates :  
JDK-8222552 - [Graal] org.graalvm.compiler.jtt.jdk.UnsafeAllocateInstance01 fails with "AssertionError: expected null but got an exception"
Relates :  
JDK-8189828 - [Graal] Graal unit tests fail in -Xcomp mode
Relates :  
JDK-8222748 - [Graal] org.graalvm.compiler.core.test.inlining.PolymorphicInliningTest fails with AssertionError
Relates :  
JDK-8222551 - [Graal] org.graalvm.compiler.hotspot.test.ArrayNewInstanceTest fails with java.lang.AssertionError
Relates :  
JDK-8222554 - [Graal] org.graalvm.compiler.jtt.except.BC_checkcast3 fails with org.junit.ComparisonFailure
Description
Following graal unit tests fail with 
 "-XX:+UnlockExperimentalVMOptions -XX:+EnableJVMCI -XX:+TieredCompilation -XX:+UseJVMCICompiler -Djvmci.Compiler=graal -Xcomp" flags:

org.graalvm.compiler.replacements.test.ReplacementsParseTest
org.graalvm.compiler.replacements.test.DerivedOopTest
org.graalvm.compiler.api.directives.test.ProbabilityDirectiveTest
org.graalvm.compiler.core.test.ProfilingInfoTest
org.graalvm.compiler.hotspot.test.HotSpotUnsafeSubstitutionTest
org.graalvm.compiler.hotspot.test.AheadOfTimeCompilationTest
org.graalvm.compiler.hotspot.test.CheckGraalIntrinsics
org.graalvm.compiler.loop.test.LoopPartialUnrollTest
org.graalvm.compiler.replacements.test.classfile.RedefineIntrinsicTest

These tests also fail without -Xcomp flag:
org.graalvm.compiler.hotspot.test.AheadOfTimeCompilationTest
org.graalvm.compiler.hotspot.test.HotSpotUnsafeSubstitutionTest
org.graalvm.compiler.core.test.ProfilingInfoTest
Comments
Closing this bug because graal unit tests in -Xcomp mode have been disabled as part of JDK-8224683. We might return to Graal and -Xcomp questions/issues once we have libgraal.
03-06-2019
Even using server -Xcomp is problematic (instead of Graal JIT mode + -Xcomp), because the -Xcomp causes problems with the profiling infomation.
30-05-2019
This bug was originally filed against Graal JIT mode + -Xcomp, we don't run graal unit tests in this configuration any more. We run them just in -server -Xcomp. Perhaps only small set of tests is not suitable for "-server -Xcomp" and perhaps such tests should be patched similar way as ProfilingInfoTest or added into problem list. But majority of tests perhaps should continue to be tested.
18-04-2019
[~kvn][~dnsimon][~epavlova][~thartmann] Should we go ahead and not run Graal unit tests with -Xcomp?
18-04-2019
I agree that we may want to exclude these tests when -Xcomp is specified because they heavily rely on profile information. I'm marking this issue as test bug, the test harness should be fixed to reliably detect -Xcomp and skip execution (for example, via the getVMFlag Whitebox API method). However, even with the now available Whitebox interface to explicitly trigger C1/C2 compilations, we still need -Xcomp testing. There are cases where we explicitly want immature profiling, for example, to trigger compilation-deoptimization cycles. With -Xcomp, many classes are not yet loaded, leading to different inlining and optimization decisions during compilation. In general, -Xcomp is similar to CTW in that it is very useful for stress testing by compiling all executed methods of the test and the test harness (also those that would normally not be compiled). So I think it's fine to skip -Xcomp execution in those tests that heavily rely on profiling but in general, Graal tests should work with -Xcomp. [~kvn], what do you think?
04-12-2017
Thinking about this a bit more, I'm not sure we want to invest effort in making these tests work with -Xcomp. As far as I understand, -Xcomp exists to assist with testing/stressing compilers like C1 and C2 that historically did not have an explicit Java API available to write white box tests. Graal has had such an API from the beginning and these tests are written with that mechanism in mind. They do not take into account the way -Xcomp fundamentally changes execution of bytecodes (e.g., causing profiles to be missing/immature). I attempted earlier to "fix" this by skipping tests (e.g. [1]) based on whether -Xcomp is on the command line. This turns out to be unreliable given that the utility for accessing the VM command line relies on the testing harness to pass it in an environment variable[2]. Unless I'm missing something with this line of thinking, I propose that Graal unit tests are not to be run with -Xcomp. [1] https://github.com/graalvm/graal/blob/ae813716ed6525dc4db5c51e7ef3cc904f73c18d/compiler/src/org.graalvm.compiler.core.test/src/org/graalvm/compiler/core/test/ProfilingInfoTest.java#L188-L192 [2] https://github.com/graalvm/graal/blob/a02149e930fe9fba3085c51e3ff91270019b3fd7/compiler/src/org.graalvm.compiler.test/src/org/graalvm/compiler/test/SubprocessUtil.java#L55
01-12-2017
Doug, could you please have a look or re-assign? Thanks.
01-12-2017
ILW = Graal unit tests fail (probably test bugs), several tests with -Xcomp, no workaround = MMH = P3
01-12-2017
This looks similar to JDK-8189828 (the HotSpotUnsafeSubstitutionTest failure was mentioned there as well). Those failures without -Xcomp do not fit though.
01-12-2017