JDK-8157306 : Random infrequent null pointer exceptions in javac
  • Type: Bug
  • Component: hotspot
  • Sub-Component: compiler
  • Affected Version: 8u92,9
  • Priority: P2
  • Status: Closed
  • Resolution: Fixed
  • OS: linux
  • CPU: generic
  • Submitted: 2016-05-19
  • Updated: 2017-11-29
  • Resolved: 2016-07-25
The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.
8u152Fixed 9 b131Fixed
Related Reports
Relates :  
Relates :  
For jdk8u test case: hotspot/test/runtime/6925573/SortMethodsTest.java, I witnessed random infrequent null pointer exceptions in javac with aarch64 jdk built from the latest jdk9-hs-comp branch.

Note: This issue is only reproducable under -XX:+UseParallelGC, it does not happens with the default G1GC for jdk9.  Also reproducable with the latest aarch64-jdk8u branch.  I have also found that turning off instruction scheduling (-XX:-OptoScheduling) or turning off narrow class pointers (-XX:-UseCompressedClassPointers) makes it go away.  Also it's easier to reproduce the bug when the memory subsystem is under high pressure.  For example, running specjbb2005 with 32 warehouses in parallel.  Somehow I suspect this issue is related to the aarch64 weak memory model. 
To reproduce the bug, one need to search the jtreg log for each run of the test, something like this:  
  jtreg -va -nr hotspot/test/runtime/6925573/SortMethodsTest.java &> log
  msg=`egrep NullPointerException log`
  if [ "$msg" != "" ]; then
  cat log;

JTreg error message:
10 x more methods requires 0 x more time
10 x more methods requires 2 x more time
An exception has occurred in the compiler (9-internal). Please file a bug against the Java compiler via the Java bug reporting page (http://bugreport.java.com) after checking the Bug Database (http://bugs.java.com) for duplicates. Include your program and the following diagnostic in your report. Thank you.
        at com.sun.tools.javac.comp.Lower.visitReturn(jdk.compiler@9-internal/Lower.java:3635)
        at com.sun.tools.javac.tree.JCTree$JCReturn.accept(jdk.compiler@9-internal/JCTree.java:1526)
        at com.sun.tools.javac.tree.TreeTranslator.translate(jdk.compiler@9-internal/TreeTranslator.java:58)
        at com.sun.tools.javac.comp.Lower.translate(jdk.compiler@9-internal/Lower.java:2392)
        at com.sun.tools.javac.tree.TreeTranslator.translate(jdk.compiler@9-internal/TreeTranslator.java:70)
        at com.sun.tools.javac.tree.TreeTranslator.visitBlock(jdk.compiler@9-internal/TreeTranslator.java:167)
        at com.sun.tools.javac.comp.Lower.visitBlock(jdk.compiler@9-internal/Lower.java:3608)
        at com.sun.tools.javac.tree.JCTree$JCBlock.accept(jdk.compiler@9-internal/JCTree.java:1000)
        at com.sun.tools.javac.tree.TreeTranslator.translate(jdk.compiler@9-internal/TreeTranslator.java:58)
        at com.sun.tools.javac.comp.Lower.translate(jdk.compiler@9-internal/Lower.java:2392)
        at com.sun.tools.javac.tree.TreeTranslator.visitMethodDef(jdk.compiler@9-internal/TreeTranslator.java:150)
        at com.sun.tools.javac.comp.Lower.visitMethodDefInternal(jdk.compiler@9-internal/Lower.java:2853)
        at com.sun.tools.javac.comp.Lower.visitMethodDef(jdk.compiler@9-internal/Lower.java:2765)
        at com.sun.tools.javac.tree.JCTree$JCMethodDecl.accept(jdk.compiler@9-internal/JCTree.java:852)
        at com.sun.tools.javac.tree.TreeTranslator.translate(jdk.compiler@9-internal/TreeTranslator.java:58)
        at com.sun.tools.javac.comp.Lower.translate(jdk.compiler@9-internal/Lower.java:2392)
        at com.sun.tools.javac.comp.Lower.visitClassDef(jdk.compiler@9-internal/Lower.java:2538)
        at com.sun.tools.javac.tree.JCTree$JCClassDecl.accept(jdk.compiler@9-internal/JCTree.java:760)
        at com.sun.tools.javac.tree.TreeTranslator.translate(jdk.compiler@9-internal/TreeTranslator.java:58)
        at com.sun.tools.javac.comp.Lower.translate(jdk.compiler@9-internal/Lower.java:2392)
        at com.sun.tools.javac.comp.Lower.translate(jdk.compiler@9-internal/Lower.java:2411)
        at com.sun.tools.javac.comp.Lower.translateTopLevelClass(jdk.compiler@9-internal/Lower.java:3977)
        at com.sun.tools.javac.main.JavaCompiler.desugar(jdk.compiler@9-internal/JavaCompiler.java:1543)
        at com.sun.tools.javac.main.JavaCompiler.desugar(jdk.compiler@9-internal/JavaCompiler.java:1401)
        at com.sun.tools.javac.main.JavaCompiler.compile(jdk.compiler@9-internal/JavaCompiler.java:940)
        at com.sun.tools.javac.api.JavacTaskImpl$1.call(jdk.compiler@9-internal/JavacTaskImpl.java:104)
        at com.sun.tools.javac.api.JavacTaskImpl$1.call(jdk.compiler@9-internal/JavacTaskImpl.java:100)
        at com.sun.tools.javac.api.JavacTaskImpl.handleExceptions(jdk.compiler@9-internal/JavacTaskImpl.java:135)
        at com.sun.tools.javac.api.JavacTaskImpl.doCall(jdk.compiler@9-internal/JavacTaskImpl.java:100)
        at com.sun.tools.javac.api.JavacTaskImpl.call(jdk.compiler@9-internal/JavacTaskImpl.java:94)
        at SortMethodsTest.main(SortMethodsTest.java:90)
        at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(java.base@9-internal/Native Method)
        at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(java.base@9-internal/NativeMethodAccessorImpl.java:62)
        at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(java.base@9-internal/DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(java.base@9-internal/Method.java:531)
        at com.sun.javatest.regtest.agent.MainWrapper$MainThread.run(MainWrapper.java:92)
        at java.lang.Thread.run(java.base@9-internal/Thread.java:843)
No new tests were added since the problem showed with existing Hotspot jtreg tests.

webrev: http://cr.openjdk.java.net/~aph/8157306.1/

ILW = javac doesn't work; rarely; none = HLH = P2

Yes, I'm sure that's true. I'll be looking at this some more to find the root cause. Whatever it is, it is very rare.

For hotspot/test/runtime/6925573/SortMethodsTest.java, the patch from Ed also masks the NPE issue. I have run the test case for more than 600 times, and I witnessed no occurrence of NPE in javac.

FWIW, the following patch makes the problem go away. I understand that this is just masking the problem, but perhaps this could be included as part of the final fix as it really shouldn't be doing mov x11, xzr str w11, [...] --- cut here --- diff -r 2b047a72357b src/cpu/aarch64/vm/aarch64.ad --- a/src/cpu/aarch64/vm/aarch64.ad Wed Jun 01 02:00:02 2016 +0100 +++ b/src/cpu/aarch64/vm/aarch64.ad Wed Jun 08 08:45:19 2016 +0000 @@ -8442,17 +8442,15 @@ ins_pipe(istore_reg_mem); %} -instruct storeImmN0(iRegIHeapbase heapbase, immN0 zero, memory mem) +instruct storeImmN0(immN0 zero, memory mem) %{ match(Set mem (StoreN mem zero)); - predicate(Universe::narrow_oop_base() == NULL && - Universe::narrow_klass_base() == NULL && - (!needs_releasing_store(n))); - - ins_cost(INSN_COST); - format %{ "strw rheapbase, $mem\t# compressed ptr (rheapbase==0)" %} - - ins_encode(aarch64_enc_strw(heapbase, mem)); + predicate(!needs_releasing_store(n)); + + ins_cost(INSN_COST); + format %{ "strw zr, $mem\t# compressed ptr (NULL)" %} + + ins_encode(aarch64_enc_strw0(mem)); ins_pipe(istore_reg_mem); %} --- cut here ---

Hi, I have reproduced this with jdk 9. Download the file http://openjdk.linaro.org/npetest9.tar.xz untar it and cd to npetest9 edit the file 'doit' and change all occurrences of /home/ed/images/jdk9-server-release-1605/bin/java to point to your local java The doit script runs 40 copies of javac in parallel. I find typicall 1 in 10 of these fails. The test is just another extract from the jdk build.

It seems like a store and a load are being switched around. So, JCTree tmpResult = this.result; this.result = null; return (T)tmpResult; // XXX cast Turns into something like this.result = null; JCTree tmpResult = this.result; return (T)tmpResult; // XXX cast Here's the bad code: 0x0000007f8c8d5268: mov x11, xzr 0x0000007f8c8d526c: ldr x13, [sp,#32] 0x0000007f8c8d5270: lsr x12, x13, #9 0x0000007f8c8d5274: str w11, [x13,#12] here <======================= 0x0000007f8c8d5278: adrp x14, word_map_base ; {external_word} 0x0000007f8c8d527c: ldr w10, [x13,#12] here <======================= 0x0000007f8c8d5280: ldr w11, [sp,#16] 0x0000007f8c8d5284: strb wzr, [x14,x12,lsl #0] 0x0000007f8c8d52cc: lsl x1, x12, #3 And the good code: 0x0000007f7cc960f0: ldr x13, [sp,#32] 0x0000007f7cc960f4: ldr w10, [sp,#16] 0x0000007f7cc960f8: str w10, [x13,#60] ;*putfield pt ; - com.sun.tools.javac.comp.TransTypes::translate@19 (line 492) ; - com.sun.tools.javac.comp.TransTypes::visitSelect@162 (line 825) 0x0000007f7cc960fc: ldr w10, [x13,#12] ;*getfield result ; - com.sun.tools.javac.tree.TreeTranslator::translate@12 (line 59) ; - com.sun.tools.javac.comp.TransTypes::translate@12 (line 490) ; - com.sun.tools.javac.comp.TransTypes::visitSelect@162 (line 825) 0x0000007f7cc96100: lsr x11, x13, #9 0x0000007f7cc96104: adrp x12, word_map_base ; {external_word} 0x0000007f7cc96108: strb wzr, [x12,x11,lsl #0] ;*putfield pt ; - com.sun.tools.javac.comp.TransTypes::translate@19 (line 492) ; - com.sun.tools.javac.comp.TransTypes::visitSelect@162 (line 825) 0x0000007f7cc9610c: lsl x10, x10, #3 ;*getfield result ; - com.sun.tools.javac.tree.TreeTranslator::translate@12 (line 59) ; - com.sun.tools.javac.comp.TransTypes::translate@12 (line 490) ; - com.sun.tools.javac.comp.TransTypes::visitSelect@162 (line 825) 0x0000007f7cc96110: mov x12, xzr 0x0000007f7cc96114: str w12, [x13,#12] ;*putfield result ; - com.sun.tools.javac.tree.TreeTranslator::translate@18 (line 60) ; - com.sun.tools.javac.comp.TransTypes::translate@12 (line 490) ; - com.sun.tools.javac.comp.TransTypes::visitSelect@162 (line 825) You can see that in the first (bad) case 0x0000007f8c8d5274: str w11, [x13,#12] has been moved ahead of 0x0000007f8c8d527c: ldr w10, [x13,#12] which is obviously wrong.

Please try running the javac test with this option: dontinline com.sun.tools.javac.tree.TreeTranslator::translate

No, I haven't reproduced it with jdk9, only jdk8. But if Felix says he sees it with jdk9 I have no reason to doubt it! I'll try to double check it with jdk9.

OK, don't know if this helps, but I have just done many (>10) compiles with -XX:+PrintCompilation and each time it gets the NPE almost immediately after compiling 10822 4519 4 com.sun.tools.javac.comp.TransTypes::visitSelect (218 bytes) but the NPE occurs in java.lang.NullPointerException at com.sun.tools.javac.comp.Lower.visitSelect(Lower.java:3837) (I have also verified that it is still present with -XX:-TieredCompilation, but it is a lot less frequent - I cannot provoke it with single runs - I need to do 100 runs in parallel).

I can't get this script to work with JDK9's javac. Have you any idea how to do it?

OK, so now I *did* reproduce the problem with aph@arm64:~/npetest$ /home/aph/jdk8u/build/linux-aarch64-normal-server-release/jdk/bin/javac -J-version openjdk version "1.8.0-internal" OpenJDK Runtime Environment (build 1.8.0-internal-aph_2016_06_01_14_14-b00) OpenJDK 64-Bit Server VM (build 25.71-b00, mixed mode) So I guess we're looking at a recent regression.

aph@arm64:~/npetest$ ./doit aph@arm64:~/npetest$ ./doit aph@arm64:~/npetest$ ./doit aph@arm64:~/npetest$ ./doit aph@arm64:~/npetest$ /home/aph/jdk8-images/j2sdk-image/bin/javac -J-version openjdk version "1.8.0-internal" OpenJDK Runtime Environment (build 1.8.0-internal-aph_2016_03_31_16_49-b00) OpenJDK 64-Bit Server VM (build 25.71-b00, mixed mode)

It can also be reproduced more reliably by downloading the following tarball http://openjdk.linaro.org/npetest.tar.xz then do tar xf npetest.tar.xz cd npetest (edit the file doit and change '/home/ed/images/jdk8-orig/bin/javac' to the location of your javac) source doit the 'doit' script just compiles about 700 odd files in a single javac command. I tried this 10 times in a row on a Cavium board and it failed every time, although I have seen it succeed. As Felix says using -XX:-OptoScheduling, -XX:-UseCompressedClassPointers, or -XX:+UseG1GC makes it go away.

My test has been running for two days now. There have been no NullPointerExceptions. I'm tempted to close this bug report, but you could instead provide a script which reproduces this problem and tell me exactly what hardware you ran it on.

FWIW: This was triggered 11 times by last nights JTreg test run. See - http://openjdk.linaro.org/openjdk8-jtreg-nightly-tests/builds/server-release/2016/153/JTwork-hotspot/compiler/stable/TestStableShort.jtr - http://openjdk.linaro.org/openjdk8-jtreg-nightly-tests/builds/server-release/2016/153/JTwork-hotspot/compiler/rtm/method_options/TestNoRTMLockElidingOption.jtr - http://openjdk.linaro.org/openjdk8-jtreg-nightly-tests/builds/server-release/2016/153/JTwork-hotspot/gc/concurrentMarkSweep/GuardShrinkWarning.jtr - http://openjdk.linaro.org/openjdk8-jtreg-nightly-tests/builds/server-release/2016/153/JTwork-hotspot/gc/g1/TestPrintGCDetails.jtr - http://openjdk.linaro.org/openjdk8-jtreg-nightly-tests/builds/server-release/2016/153/JTwork-hotspot/gc/g1/Test2GbHeap.jtr - http://openjdk.linaro.org/openjdk8-jtreg-nightly-tests/builds/server-release/2016/153/JTwork-langtools/tools/javac/processing/options/testPrintProcessorInfo/Test.jtr - http://openjdk.linaro.org/openjdk8-jtreg-nightly-tests/builds/server-release/2016/153/JTwork-jdk/java/lang/instrument/DaemonThread/TestDaemonThread.jtr - http://openjdk.linaro.org/openjdk8-jtreg-nightly-tests/builds/server-release/2016/153/JTwork-jdk/java/lang/invoke/accessProtectedSuper/Test.jtr - http://openjdk.linaro.org/openjdk8-jtreg-nightly-tests/builds/server-release/2016/153/JTwork-jdk/java/io/Serializable/unshared/Write.jtr - http://openjdk.linaro.org/openjdk8-jtreg-nightly-tests/builds/server-release/2016/153/JTwork-jdk/java/math/BigInteger/OperatorNpeTests.jtr - http://openjdk.linaro.org/openjdk8-jtreg-nightly-tests/builds/server-release/2016/153/JTwork-jdk/java/nio/file/attribute/UserDefinedFileAttributeView/Basic.jtr In every case it gets the NullPointer at the same place at com.sun.tools.javac.comp.Lower.visitSelect(Lower.java:3837)

Sorry for the confusion caused. First of all, the bug cannot be reproduced with SPECjbb. Currently, I can reproduce the bug with hotspot/test/runtime/6925573/SortMethodsTest.java of jdk8u. The reason why I mention SPECjbb here is that I want to make the memory subsystem busy by running SPECjbb. When the the memory subsystem is busy, it is more likely that test case SortMethodsTest.java fails. I am not sure if it is a bug in javac though.

You say that this bug is easier to reproduce with SPECjbb. The trouble with using javac as a test program is that this may well be a bug in javac, and nothing at all to do with the VM. Please supply me with a script to reproduce this. I have SPECjbb.

I double checked and I am sure that I got this patch: http://cr.openjdk.java.net/~aph/8154739 The bug is still there, can be reproduced with the latest aarch64 jdk built from jdk9 hs-comp repo. I suspect this issue is caused by the C2 compiler, as -XX:-OptoScheduling option makes it go away.

This looks exactly like 8154739. Are you absolutely sure that you've got the patch http://cr.openjdk.java.net/~aph/8154739 ?