Bug ID: JDK-8047331 Assertion in CompiledFunction when running earley-boyer after Merge

JDK-8047331 : Assertion in CompiledFunction when running earley-boyer after Merge

Type: Bug
Component: core-libs
Sub-Component: jdk.nashorn
Affected Version: 8u40

Priority: P2
Status: Closed
Resolution: Fixed
OS: generic
CPU: generic

Submitted: 2014-06-19
Updated: 2015-01-23
Resolved: 2014-08-05

Versions (Unresolved/Resolved/Fixed)

The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.

JDK 8	JDK 9
8u40 b04Fixed	9Fixed

Description

The culprit is 

changeset:   868:aeccdbb8d366
parent:      867:f44ec6545b9a
parent:      816:fed8c83dfba4
user:        attila
date:        Mon Jun 02 17:36:43 2014 +0200
summary:     Merge

which apparently broke earley boyer -  this is a merge from 9. 
It has been broken since, and it is extremely strange that no harness has picked it up.

To reproduce, from the make directory:

alhazred:make marcus$ sh ../bin/runopt.sh -scripting ../test/script/basic/run-octane.js -- earley-boyer --verbose --iterations 2
[earley-boyer] loading 'earley-boyer' [earley-boyer.js]...
[earley-boyer] running 'earley-boyer' for 2 iterations of no less than 5 seconds
[earley-boyer] *** Aborted and setting score to zero. Reason: java.lang.AssertionError
java.lang.AssertionError
	at jdk.nashorn.internal.runtime.CompiledFunction.handleRewriteException(CompiledFunction.java:635)
	at jdk.nashorn.internal.runtime.CompiledFunction.handleRewriteException(CompiledFunction.java:553)
	at java.lang.invoke.MethodHandle.invokeWithArguments(MethodHandle.java:636)
	at jdk.nashorn.internal.scripts.Script$Recompilation$75$earley_boyer.BgL_makezd2parserzd2(/Users/marcus/src/nashorn-jdk9/make/../test/script/basic/../external/octane/earley-boyer.js:3877)
	at java.lang.invoke.MethodHandle.invokeWithArguments(MethodHandle.java:636)
	at jdk.nashorn.internal.scripts.Script$Recompilation$74$earley_boyer.test(/Users/marcus/src/nashorn-jdk9/make/../test/script/basic/../external/octane/earley-boyer.js:4621)
	at java.lang.invoke.MethodHandle.invokeWithArguments(MethodHandle.java:636)
	at jdk.nashorn.internal.scripts.Script$Recompilation$73$earley_boyer.BgL_earleyzd2benchmarkzd2$L:4647(/Users/marcus/src/nashorn-jdk9/make/../test/script/basic/../external/octane/earley-boyer.js:4648)
	at java.lang.invoke.MethodHandle.invokeWithArguments(MethodHandle.java:636)
	at jdk.nashorn.internal.scripts.Script$Recompilation$72$earley_boyer.RunBenchmark(/Users/marcus/src/nashorn-jdk9/make/../test/script/basic/../external/octane/earley-boyer.js:4677)
	at java.lang.invoke.MethodHandle.invokeWithArguments(MethodHandle.java:636)
	at jdk.nashorn.internal.scripts.Script$Recompilation$70$earley_boyer.BgL_earleyzd2benchmarkzd2(/Users/marcus/src/nashorn-jdk9/make/../test/script/basic/../external/octane/earley-boyer.js:4647)
	at java.lang.invoke.MethodHandle.invokeWithArguments(MethodHandle.java:636)
	at jdk.nashorn.internal.scripts.Script$Recompilation$69$earley_boyer.EarleyBoyer(/Users/marcus/src/nashorn-jdk9/make/../test/script/basic/../external/octane/earley-boyer.js:5)
	at jdk.nashorn.internal.scripts.Script$Recompilation$68$run_octane$cu2$restOf.run_one_benchmark(../test/script/basic/run-octane.js:143)
	at java.lang.invoke.MethodHandle.invokeWithArguments(MethodHandle.java:636)
	at java.lang.invoke.MethodHandle.invokeWithArguments(MethodHandle.java:636)
	at jdk.nashorn.internal.scripts.Script$Recompilation$28$run_octane$cu2$restOf.run_suite(../test/script/basic/run-octane.js:200)
	at java.lang.invoke.MethodHandle.invokeWithArguments(MethodHandle.java:636)
	at java.lang.invoke.MethodHandle.invokeWithArguments(MethodHandle.java:636)
	at jdk.nashorn.internal.scripts.Script$Recompilation$26$run_octane$cu2$restOf.:program(../test/script/basic/run-octane.js:305)
	at java.lang.invoke.MethodHandle.invokeWithArguments(MethodHandle.java:636)
	at java.lang.invoke.MethodHandle.invokeWithArguments(MethodHandle.java:636)
	at java.lang.invoke.MethodHandle.invokeWithArguments(MethodHandle.java:636)
	at java.lang.invoke.MethodHandle.invokeWithArguments(MethodHandle.java:636)
	at java.lang.invoke.MethodHandle.invokeWithArguments(MethodHandle.java:636)
	at java.lang.invoke.MethodHandle.invokeWithArguments(MethodHandle.java:636)
	at java.lang.invoke.MethodHandle.invokeWithArguments(MethodHandle.java:636)
	at java.lang.invoke.MethodHandle.invokeWithArguments(MethodHandle.java:636)
	at java.lang.invoke.MethodHandle.invokeWithArguments(MethodHandle.java:636)
	at java.lang.invoke.MethodHandle.invokeWithArguments(MethodHandle.java:636)
	at java.lang.invoke.MethodHandle.invokeWithArguments(MethodHandle.java:636)
	at java.lang.invoke.MethodHandle.invokeWithArguments(MethodHandle.java:636)
	at java.lang.invoke.MethodHandle.invokeWithArguments(MethodHandle.java:636)
	at java.lang.invoke.MethodHandle.invokeWithArguments(MethodHandle.java:636)
	at java.lang.invoke.MethodHandle.invokeWithArguments(MethodHandle.java:636)
	at java.lang.invoke.MethodHandle.invokeWithArguments(MethodHandle.java:636)
	at java.lang.invoke.MethodHandle.invokeWithArguments(MethodHandle.java:636)
	at java.lang.invoke.MethodHandle.invokeWithArguments(MethodHandle.java:636)
	at java.lang.invoke.MethodHandle.invokeWithArguments(MethodHandle.java:636)
	at java.lang.invoke.MethodHandle.invokeWithArguments(MethodHandle.java:636)
	at java.lang.invoke.MethodHandle.invokeWithArguments(MethodHandle.java:636)
	at jdk.nashorn.internal.runtime.ScriptFunctionData.invoke(ScriptFunctionData.java:567)
	at jdk.nashorn.internal.runtime.ScriptFunction.invoke(ScriptFunction.java:221)
	at jdk.nashorn.internal.runtime.ScriptRuntime.apply(ScriptRuntime.java:374)
	at jdk.nashorn.tools.Shell.apply(Shell.java:391)
	at jdk.nashorn.tools.Shell.runScripts(Shell.java:320)
	at jdk.nashorn.tools.Shell.run(Shell.java:169)
	at jdk.nashorn.tools.Shell.main(Shell.java:133)
	at jdk.nashorn.tools.Shell.main(Shell.java:112)
[earley-boyer] 0 ops/minute (0-0), warmup=0


The assertion is

        assert optimismInfo == oldOptimismInfo;


Not sure if it's a valid assert or not.

Comments

Verfied 8u40 b22 with regtests
23-01-2015
Resolving as fixed - the changes will be backported to 8u40 from 9 in a single merged changeset. This issue shouldn't have been marked as targeting 8u40 in the first place, as that created a synthetic "9 backport" issue when the fix was committed to 9; it was this issue that should've had 9 as its "Affects Version/s" field instead.
05-08-2014
Explanation of the check: optimismInfo == oldOptimismInfo can only be false in case of a recursive invocation (including indirectly through one or more mutually recursive functions). In this case, when the control returns to this invocation, we might already have deoptimized the method further, but we still have this old version on stack. If this old version itself runs into a need to deoptimize, we'll notice that the program point has already been invalidated (oldOptimismInfo.requestRecompile() will return false), we'll just generate a new rest of, and return from the method. In this case, the assert is never reached. So, the invariant here is that either !shouldRecompile, or optimismInfo == oldOptimismInfo. Unfortunately, with static type calculations, we can end up with a program point being emitted as non-optimistic by CodeGenerator, but without this being recorded in the OptimismInfo.invalidatedProgramPoints map. Such case can arise (naturally) in ternary operators, and that's exactly what happened here. Earley-boyer.js has a function that returns the value of a ternary on line 3876. The falseExpression of the ternary (with pp=30) gets deoptimized to Object. This'll trigger the trueExpression (pp=25) to be emitted as non-optimistic too (as the type of the ternary subexpressions needs to be the wider of the two). However, we never explicitly invalidated pp=25, and it got no entry in the invalidatedProgramPoints for the function. OTOH, CodeGenerator noticed that it has not emitted a single optimistic call site when it invalidated pp=30, so it reset the FunctionNode.IS_DEOPTIMIZABLE flag, making the CompiledFunction discard its OptimisticInfo (replace it with null), making the assert fail. The problem arises from the fact that OptimisticInfo.requestRecompile has less precise information on whether the program point in the RewriteException has already been invalidated in a recursive recompile. CodeGenerator's signaling through FunctionNode.IS_DEOPTIMIZABLE is (leading to optimismInfo = null setting), on the other hand, completely precise but it's a boolean signal ("still some program points deoptimizable"/"no program points remain deoptimizable"). There are several ways to fix this. One is to indeed forget this assert. Upon inspection, it is harmless. Replace final boolean shouldRecompile = oldOptimismInfo.requestRecompile(re); with final boolean shouldRecompile = optimismInfo != null && oldOptimismInfo.requestRecompile(re); instead, and be done with it. This'll avoid pointless recompilation of already full deoptimized methods. Another way to fix this is to have CodeGenerator.OptimisticOperation constructor explicitly update Compiler.invalidatedProgramPoints with the lower bound of the TypeBound used for that program point evaluation. This way, we can keep the assert and also avoid recompilation of recursive functions not only when they are fully deoptimized, but in general. I'd actually go with the first way to fix it. Recursive functions are quite rare, so we can tolerate an unnecessary recompilation of them here and there.
02-07-2014
Removing the assert works fine. No validation errors or anything, but it would be great if you could provide me with an explanation about the optimistic info inequality here, Attila. I don't fully understand the reason for this check.
19-06-2014