JDK-8178047 : Aliasing problem with raw memory accesses
  • Type: Bug
  • Component: hotspot
  • Sub-Component: compiler
  • Affected Version: 8,9
  • Priority: P2
  • Status: Closed
  • Resolution: Fixed
  • OS: generic
  • CPU: generic
  • Submitted: 2017-04-04
  • Updated: 2019-09-13
  • Resolved: 2017-04-20
The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.
JDK 10 JDK 8 JDK 9
10Fixed 8u162Fixed 9 b167Fixed
Related Reports
Duplicate :  
Duplicate :  
Description
FULL PRODUCT VERSION :
$ java -version
java version "1.8.0_121"
Java(TM) SE Runtime Environment (build 1.8.0_121-b13)
Java HotSpot(TM) Server VM (build 25.121-b13, mixed mode)


FULL OS VERSION :
Linux localhost.localdomain 3.10.0-327.36.2.el7.x86_64 #1 SMP Mon Oct 10 23:08:37 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux

A DESCRIPTION OF THE PROBLEM :
The attached testcase fails when run in compiled mode on 32 bit Intel Linux.

It does NOT fail in interpreted mode (-Xint) or in compiled mode with a 64 bit VM. 

It does NOT fail on 32 bit Intel Windows.

The testcase has taken significant effort to extract. We have a COBOL product (NTT DATA Enterprise COBOL, formerly Dell Enterprise COBOL) that translates COBOL to Java for execution and the problem originates from a customer COBOL program.

The slightly unusual coding patterns reflect our implementation of COBOL "unsigned usage display" numerics, in which a decimal number is represented in memory using ASCII digits, i.e that characters '0', '1', ... (This also necessitates our use of 'Unsafe')

THE PROBLEM WAS REPRODUCIBLE WITH -Xint FLAG: No

THE PROBLEM WAS REPRODUCIBLE WITH -server FLAG: Yes

STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
Compile the code:

$ javac Sim.java Chunk.java

Execute 'Sim'

$ java Sim



EXPECTED VERSUS ACTUAL BEHAVIOR :
$ java Sim

Output is as follows:

main() >
[0000000000]
[0000000001]
[0000000002]
[0000000003]
[0000000004]
[0000000005]
[0000000006]
[0000000007]
[0000000008]
[0000000009]
...
[0000110638]
[0000110639]
[0000110640]
[0000110641]
[0000000000]
[0000000000]
[0000000000]
...
[0000000000]
[0000000000]
[0000000000]
main() <

The output should just be an increasing integer, as can be seen by using -Xint:

java -Xint Sim:

main() >
[0000000000]
[0000000001]
[0000000002]
[0000000003]
[0000000004]
[0000000005]
[0000000006]
[0000000007]
[0000000008]
[0000000009]
...
[0000198760]
[0000198761]
[0000198762]
[0000198763]
[0000198764]
main() <

The exact point in the failing case where the error occurs (output turns to zero) seems to vary, and I assume corresponds to when HotSpot compilation occurs

REPRODUCIBILITY :
This bug can be reproduced always.

---------- BEGIN SOURCE ----------
Sim.java:


public class Sim {
   private final Chunk c_A1;
   private final Chunk c_A2;
   private final Chunk c_R;

   public static void main(String[] args) {
      System.out.println("main() >");

      Sim s = new Sim();
      s.go();

      System.out.println("main() <");
   }

   public Sim() {
      Chunk r_0 = new Chunk(64);
      c_A1 = r_0.slice(0, 11);
      c_A2 = r_0.slice(11, 11);
      c_R = r_0.slice(22, 19);
   }

   private void go() {
      c_A1.fillSmall(0, 11, (byte)0x30);
      for (long i = 0; i < 198765L; i++) {
         c_A2.put_Long_DU(0, 11, i );
         c_R.fillSmall(9, 10, (byte)0x30);
         c_R.put_Long_DU(9, 10, c_R.get_DU_Long(9, 10) + c_A2.get_DU_Long(0, 11));
         c_R.put_Long_DU(9, 10, c_R.get_DU_Long(9, 10) + c_A1.get_DU_Long(0, 11));

         printR();
      }
   }

   private void printR() {
      final byte[] ba = c_R.getAsByteArray(9, 10);
      final String s = new String(ba);
      System.out.println("[" + s + "]");
   }
}



Chunk.java:


import java.lang.reflect.Field;
import sun.misc.Unsafe;

public class Chunk {

   private static final Unsafe UNSAFE;

   private final long baseAddress;
   private final int length;

    static {
       Unsafe u = null;
       try {
          final Class<Unsafe> uc = Unsafe.class;
          final Field field = uc.getDeclaredField("theUnsafe");
          field.setAccessible(true);
          u = (Unsafe) field.get(uc);
       } catch (NoSuchFieldException | IllegalAccessException | RuntimeException ex) {
          ex.printStackTrace();
       }
       UNSAFE = u;
    }

   public Chunk(int length) {
      baseAddress = UNSAFE.allocateMemory(length);
      this.length = length;
   }

   public final long getBaseAddress() {
      return baseAddress;
   }

   public final int getLength() {
      return length;
   }

   protected Chunk(Chunk oc, int op, int ol) {
      baseAddress = oc.getBaseAddress() + op;

      if (oc.getLength() == -1) {
         length = -1;
      } else {
         length = Math.min(ol, oc.getLength() - op);
      }
   }

   public Chunk slice(int p, int l) {
      Chunk o = new Chunk(this, p, l);
      return o;
   }

   public final void fillSmall(int tp, int tl, byte b) {
      long p = baseAddress + tp;
      for (int i = 0; i < tl; i++) {
         UNSAFE.putByte(p++, b);
      }
   }

   public void put_Long_DU(int p, int d, long v) {

      long a = baseAddress + p;

      for (int i = d - 1; i >= 0; i--) {
         long b = (v % 10) + 0x30;
         UNSAFE.putByte(a + i, (byte) b);
         v /= 10;
      }
   }

   public long get_DU_Long(int p, int l) {
      long a = baseAddress + p;

      long ret = 0;
      for (int i = 0; i < l; i++) {
         final int b = UNSAFE.getByte(a++);
         ret *= 10;
         ret += b & 0x0f;
      }

      return ret;
   }

   public byte[] getAsByteArray(int p, int l) {
      byte[] ret = new byte[l];
      long a = baseAddress + p;
      for (int i = 0; i < l; i++) {
         ret[i] = UNSAFE.getByte(a++);
      }
      return ret;
   }

}






---------- END SOURCE ----------

CUSTOMER SUBMITTED WORKAROUND :
End customer has had to change their original COBOL program to avoid generating the pattern that causes this issue. They have had to do this in multiple programs.


Comments
I'll backport this to 8.
21-07-2017

There is 8-bp label added and no back port created yet. We got another incident JDK-8183398 for backport of this to 8.
03-07-2017

We should consider to backport it.
19-04-2017

Approved for JDK 9.
19-04-2017

Fix Request This one is important to fix. The bug silently causes invalid results. The fix is conservative - it narrows the domain of the problematic graph transformation. Webrev: http://cr.openjdk.java.net/~iveresov/8178047/webrev.01 RBT results: https://jdash.se.oracle.com/rbt/rbt-igor.veresov-jdk9-dev-20170419-0328-38475
19-04-2017

Updated webrev: http://cr.openjdk.java.net/~iveresov/8178047/webrev.01 RBT results: https://jdash.se.oracle.com/rbt/rbt-igor.veresov-jdk9-dev-20170419-0328-38475
19-04-2017

This seems like a long-standing bug. Analysis in MemNode::find_previous_store() tries to relax memory dependencies by proving that memory accesses don���t alias. The code is fine for oops: if [offset, offset+length) intervals don���t overlap it proves that accesses don���t alias because bases always point to the start of an object. For raw accesses that���s not true. Offset analysis doesn���t mean much without proving that bases are the same. Suggested fix: http://cr.openjdk.java.net/~iveresov/8178047/webrev.00 RBT: https://jdash.se.oracle.com/rbt/rbt-igor.veresov-jdk9-dev-20170413-0345-38049
14-04-2017

Actually, no, the deopt happens because the return value is already 0. long l = c_R.get_DU_Long(9, 10); if (l == 0) { deopt; }
11-04-2017

This is the minimum amount of inlining to make it reproduce: $JAVA_HOME/bin/java -XX:-TieredCompilation -XX:OSROnlyBCI=2 -XX:+PrintInlining -XX:+PrintCompilation -XX:CompileCommand=compileonly,Sim2::go -XX:CompileCommand=compileonly,Chunk2::get_DU_Long -XX:CompileCommand=compileonly,sun.misc.Unsafe::* -XX:CompileCommand=compileonly,jdk.internal.misc.Unsafe::* Sim2 Haven't looked at the graph yet. But it seems like the problem is with the deopt. Incorrect results follow after: Uncommon trap bci=276 pc=0xf2b8dc7c, relative_pc=0x0000033c, method=Sim2.go()V Uncommon trap occurred in Sim2::go compiler=c2 compile_id=29 (@0xf2b8dc7c) thread=58214 reason=unstable_if action=reinterpret unloaded_class_index=-1 2831 29 % Sim2::go @ 2 (362 bytes) made not entrant DEOPT UNPACKING thread 0xf5b12400 vframeArray 0xadd3a318 mode 2 {method} {0xae88b740} 'go' '()V' in 'Sim2' - ifne @ bci 276 sp = 0xf5c51f00
11-04-2017

I attached the simplified test Sim2/Chunk2.java. The problem can be reproduced by running: java -Xbatch Sim2
07-04-2017

Yes, I have the same concerns. I'll check that.
05-04-2017

I have to be very suspicious about the use of Unsafe. There could easily be a memory stomp error that is only evident when JIting on 32-bit linux. I don't see anything that ensures accesses are not out of range.
05-04-2017

Well, there are OS specific flag values that affect C2 code (for example, compressed oops) but in this case it's probably just bad luck. I removed "Linux" from the ILW.
05-04-2017

I'm able to reproduce this with the latest JDK 9 build. The unsafe accesses are all in bounds and they are not reading from uninitialized memory. I was able to simplify the test and it looks like as if C2 optimized out some unsafe reads and therefore the result is always zero. Unfortunately, the generated code is still quite complex. Investigating. ILW = C2 compiled code returns incorrect result, rare but reproducible (x86_32 only), disable compilation of the affected method = HMM = P2
05-04-2017

Why would this be linux specific - Does C2 have OS specific code?
05-04-2017

Observation after 0000110641 it reset to 000000000 always... [0000110641] [0000000000] [0000000000]
04-04-2017

Issue is reproducible only on x86/32 bit linux machine on both 8 and 9 Below is the result, == 9 ea b163 - linux-x64 - Pass 9 ea b163 - linux-x86 -Xint - Pass 9 ea b163 - linux-x86 - Fail
04-04-2017