FULL PRODUCT VERSION :
java version "1.8.0_60-ea"
Java(TM) SE Runtime Environment (build 1.8.0_60-ea-b25)
Java HotSpot(TM) 64-Bit Server VM (build 25.60-b23, mixed mode)


FULL OS VERSION :
Linux ringil 3.13.0-61-generic #100-Ubuntu SMP Wed Jul 29 11:21:34 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux

A DESCRIPTION OF THE PROBLEM :
A rather involved data analysis which used to succeed fails with an ArrayIndexOutOfBoundsError.  Inspection of indices reveals that they are corrupted.  The precise corruption is stochastic, suggesting a failure of initialization; occasionally, the indices are correct and the run works.

-Xint allows the analysis to succeed normally.

-XX:-Inline also allows the analysis to succeed normally.

-XX:-DoEscapeAnalysis also allows the analysis to succeed normally.

I cannot find any combination of CompilerCommand=dontinline that allows success.

-XX:CompilerCommand=exclude,ichi/maths/package\$.rankFractionF also succeeds.

The code in question involves short-lived mutable objects capturing on-stack mutable variables, tail-call optimization, and recursion.

THE PROBLEM WAS REPRODUCIBLE WITH -Xint FLAG: No

THE PROBLEM WAS REPRODUCIBLE WITH -server FLAG: Yes

STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
Under Linux, the following will reproduce the bug.  Some details may differ slightly on other OSes.

1. Download the files needed from https://sl7.org/bugs/jdk-8u40-inline-escape-kerr.tgz (50 MB)
2. Unpack, enter jdk-8u40-inline-escape-kerr directory
3. Run with bash run-java.sh, or take the command-line out of the file (it is rather long)

A successful run will produce a directory called 20150618_150829 with two files in it that match those in expected-output.  However, in affected versions about 95% of the time an ArrayIndexOutOfBoundsError will be thrown instead.

The error has been verified on 8u40, 8u45, 8u51, and 8u60 on Linux, and on 8u51 on Mac OS.  It is not present in 8u25 or 8u31.

EXPECTED VERSUS ACTUAL BEHAVIOR :
Expected behavior: command-line output is (total runtime ~15s):

Reading files: 1 summary, 0 reference, 3 object.
  Summary file has 10442 data points.
  261 out of 3493 objects meet criteria.
Took 7.991 seconds to load data
Writing summary file 20150618_150829/e1368_SpoIIE_d02_15m30xt10s1m_B1.dat
  Write successful.

and the stated file is written (and the directory created), along with a second file with extension .rpt.ms.

Actual behavior with a buggy version:

Command-line output is

Reading files: 1 summary, 0 reference, 3 object.
  Summary file has 10442 data points.
  261 out of 3493 objects meet criteria.
Took 7.386 seconds to load data

and then the exception given below is thrown.  The output directory is created (if missing) but no files are output.

ERROR MESSAGES/STACK TRACES THAT OCCUR :
Exception in thread "main" java.lang.ArrayIndexOutOfBoundsException: -692
	at ichi.maths.package$.nearlyMaxF(package.scala:1923)
	at ichi.maths.package$.rankIndexF(package.scala:2056)
	at ichi.maths.package$.rankFractionF(package.scala:2229)
	at MultiSensed$Track$$anonfun$omegize$3.apply(MultiSensed.scala:616)
	at MultiSensed$Track$$anonfun$omegize$3.apply(MultiSensed.scala:613)
	at scala.Option.map(Option.scala:146)
	at MultiSensed$Track.omegize(MultiSensed.scala:613)
	at MultiSensed$$anonfun$prepareDance$3.apply(MultiSensed.scala:1229)
	at MultiSensed$$anonfun$prepareDance$3.apply(MultiSensed.scala:1229)
	at ichi.core.package$Soft.<init>(package.scala:150)
	at ichi.core.package$Soft$.apply(package.scala:161)
	at MultiSensed.prepareDance(MultiSensed.scala:1229)
	at MultiSensed$$anonfun$38$$anonfun$apply$24.apply(MultiSensed.scala:1242)
	at MultiSensed$$anonfun$38$$anonfun$apply$24.apply(MultiSensed.scala:1242)
	at scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.scala:33)
	at scala.collection.mutable.ArrayOps$ofRef.foreach(ArrayOps.scala:186)
	at MultiSensed$$anonfun$38.apply(MultiSensed.scala:1242)
	at MultiSensed$$anonfun$38.apply(MultiSensed.scala:1242)
	at ichi.core.package$UtilityWrapper$.tap$extension(package.scala:172)
	at MultiSensed.computeAll(MultiSensed.scala:1242)
	at Choreography.doEverything(Choreography.java:3494)
	at Choreography.main(Choreography.java:3561)

Note: the number isn't always -692.  It is always a negative number vaguely in the range of the data set size (-5914, -5702, etc.).

REPRODUCIBILITY :
This bug can be reproduced often.

---------- BEGIN SOURCE ----------
It is not possible to include the complete source code, as there is way too much of it.  The routines in question are written in Scala and, though the bytecode appears valid inasmuch as I can check by eye, it cannot be generated with javac as it uses features that the Java language does not support (tail call optimization in particular, which can be implemented as a goto once the stack is properly arranged).

Finding a minimization has thus far proven difficult.


---------- END SOURCE ----------

CUSTOMER SUBMITTED WORKAROUND :
Do not use short-lived boxing objects with a primitive mutable field in place of local mutable primitive variables to enable another (private) method to alter those fields. Rewrite to pass immutable primitives and pass back an instance of a boxing class if there are any updates.

It might only be a problem if the mutable boxing pattern is used in two methods, one of which calls the other (not sure).

Avoiding this is somewhat easy when writing Java, but not so easy when writing Scala which does it all for you, and the rewritten pattern is less efficient.

In addition, code patterns needed to avoid this bug are awkward in general, but since it seems only to be triggered rarely it is feasible to only fix the known problem.  (This does not give much confidence in the final result, however.)