JDK-8153980 : Hotspot JIT compiler omits a field write after optimisation
  • Type: Bug
  • Component: hotspot
  • Sub-Component: compiler
  • Affected Version: 8,8u101,9
  • Priority: P2
  • Status: Closed
  • Resolution: Duplicate
  • OS: linux
  • CPU: x86_64
  • Submitted: 2016-04-11
  • Updated: 2016-04-11
  • Resolved: 2016-04-11
The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.
JDK 9
9Resolved
Related Reports
Duplicate :  
Description
FULL PRODUCT VERSION :
java version "1.8.0_60"
Java(TM) SE Runtime Environment (build 1.8.0_60-b27)
Java HotSpot(TM) 64-Bit Server VM (build 25.60-b23, mixed mode)

Also testing on java version "1.8.0_77"


FULL OS VERSION :
Linux stem 4.4.6-201.fc22.x86_64 #1 SMP Wed Mar 30 18:30:16 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux

A DESCRIPTION OF THE PROBLEM :
When using the Disruptor library (https://github.com/LMAX-Exchange/disruptor), the test code in the executable test case throws a null pointer exception (NPE) in a location where it should not.  In an early pass of the compiler it is possible to see the field being updated.

  # {method} {0x00007ffa86c70d58} 'lambda$test$0' '(JLjava/lang/Integer;Ljava/lang/String;Ldisruptorbug/DisruptorTest$Event;J)V' in 'disruptorbug/DisruptorTest'
... lines omitted
  0x00007ffa8d192688: mov    %rsi,0x10(%r8)     ;*putfield id
                                                ; - disruptorbug.DisruptorTest$Event::publish@2 (line 93)
                                                ; - disruptorbug.DisruptorTest::lambda$test$0@5 (line 59)

  0x00007ffa8d19268c: mov    %rdx,%r10
  0x00007ffa8d19268f: shr    $0x3,%r10
  0x00007ffa8d192693: mov    %r10d,0xc(%r8)
  0x00007ffa8d192697: mov    %r8,%rsi
  0x00007ffa8d19269a: shr    $0x9,%rsi
  0x00007ffa8d19269e: mov    $0x7ffa9d7e4000,%rdi
  0x00007ffa8d1926a8: movb   $0x0,(%rsi,%rdi,1)  ;*putfield obj
                                                ; - disruptorbug.DisruptorTest$Event::publish@7 (line 94)
                                                ; - disruptorbug.DisruptorTest::lambda$test$0@5 (line 59)

  0x00007ffa8d1926ac: mov    %rcx,%r10
  0x00007ffa8d1926af: shr    $0x3,%r10
  0x00007ffa8d1926b3: mov    %r10d,0x18(%r8)
  0x00007ffa8d1926b7: shr    $0x9,%r8
  0x00007ffa8d1926bb: movb   $0x0,(%r8,%rdi,1)  ;*putfield str
                                                ; - disruptorbug.D
...lines omitted.

However in a later compilation pass the putfield of 'obj' field is dropped.

  # {method} {0x00007ffa86c70ba8} 'test' '()V' in 'disruptorbug/DisruptorTest'
... lines omitted
  0x00007ffa8d1b7f03: mov    0x50(%rsp),%r8
  0x00007ffa8d1b7f08: mov    %r8,0x10(%r11)     ;*putfield id
                                                ; - disruptorbug.DisruptorTest$Event::publish@2 (line 93)
                                                ; - disruptorbug.DisruptorTest::lambda$test$0@5 (line 59)
                                                ; - disruptorbug.DisruptorTest$$Lambda$3/2094548358::translateTo@17
                                                ; - com.lmax.disruptor.RingBuffer::translateAndPublish@7 (line 931)
                                                ; - com.lmax.disruptor.RingBuffer::publishEvent@13 (line 445)
                                                ; - com.lmax.disruptor.dsl.Disruptor::publishEvent@5 (line 256)
                                                ; - disruptorbug.DisruptorTest::test@36 (line 58)

**** The putfield obj is not here!!!!! ****

  0x00007ffa8d1b7f0c: movl   $0xe3443067,0x18(%r11)  ;   {oop("str")}
  0x00007ffa8d1b7f14: shr    $0x9,%r11
  0x00007ffa8d1b7f18: mov    $0x7ffa9d7e4000,%r8
  0x00007ffa8d1b7f22: mov    %r12b,(%r8,%r11,1)  ;*putfield str
                                                ; - disruptorbug.DisruptorTest$Event::publish@13 (line 95)
                                                ; - disruptorbug.DisruptorTest::lambda$test$0@5 (line 59)
                                                ; - disruptorbug.DisruptorTest$$Lambda$3/2094548358::translateTo@17
                                                ; - com.lmax.disruptor.RingBuffer::translateAndPublish@7 (line 931)
                                                ; - com.lmax.disruptor.RingBuffer::publishEvent@13 (line 445)
                                                ; - com.lmax.disruptor.dsl.Disruptor::publishEvent@5 (line 256)
                                                ; - disruptorbug.DisruptorTest::test@36 (line 58)


THE PROBLEM WAS REPRODUCIBLE WITH -Xint FLAG: No

THE PROBLEM WAS REPRODUCIBLE WITH -server FLAG: Yes

STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
1) Download the LMAX disruptor library from http://repo1.maven.org/maven2/com/lmax/disruptor/3.3.2/disruptor-3.3.2.jar
2) Compile the executable test case with the disruptor library in the class path
3) Execute the executable test case (it has a main method) with the disruptor library in the class path

EXPECTED VERSUS ACTUAL BEHAVIOR :
Expected: The test should run indefinitely without error.
Actual: The code fails part way through with an NPE.
ERROR MESSAGES/STACK TRACES THAT OCCUR :
The exception trace will look like:

id = 135222 Event{id=22965387954243, obj=null, str='str'}
java.lang.NullPointerException
	at disruptorbug.DisruptorTest$Event.testNPE(DisruptorTest.java:100)
	at disruptorbug.DisruptorTest.handler1(DisruptorTest.java:37)
	at disruptorbug.DisruptorTest$$Lambda$2/1329552164.onEvent(Unknown Source)
	at com.lmax.disruptor.BatchEventProcessor.run(BatchEventProcessor.java:128)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)


REPRODUCIBILITY :
This bug can be reproduced always.

---------- BEGIN SOURCE ----------
import com.lmax.disruptor.BusySpinWaitStrategy;
import com.lmax.disruptor.dsl.Disruptor;
import com.lmax.disruptor.dsl.ProducerType;
import org.junit.Before;

import java.util.Objects;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;


public class DisruptorTest
{
    private static final int disruptorSize = 4;

    private Disruptor<Event> disruptor;

    private volatile long opId;


    @Before
    public void setUp() throws Exception
    {
        ExecutorService executor = Executors.newCachedThreadPool();

        disruptor = new Disruptor<>(Event::new, disruptorSize, executor, ProducerType.SINGLE, new BusySpinWaitStrategy());
        disruptor.handleEventsWith(this::handler1);
        disruptor.start();
    }

    private void handler1(Event event, long sequence, boolean endOfBatch)
    {
        try
        {
            event.testNPE();
        }
        catch (Exception e)
        {
            System.err.println("id = " + opId + " " + event);
            e.printStackTrace();
            System.exit(-1);
        }
    }


    public void test() throws Exception
    {
        opId = 0L;
        while (true)
        {
            long finalOpId = System.nanoTime();

            Integer obj = new Integer(123);
            String str = "str";

            disruptor.publishEvent(
                    (event, sequence) -> event.publish(finalOpId, obj, str)
            );

            opId++;
        }
    }

    public static void main(String[] args) throws Exception
    {
        final DisruptorTest disruptorTest = new DisruptorTest();
        disruptorTest.setUp();
        disruptorTest.test();
    }

    static class Event
    {
        long id;
        Object obj;
        String str;

        @Override
        public String toString() {
            return "Event{" +
                    "id=" + id +
                    ", obj=" + obj +
                    ", str='" + str + '\'' +
                    '}';
        }

        void publish(long id, Object obj, String str)
        {
//            Objects.requireNonNull(obj);

            this.id = id;
            this.obj = obj;
            this.str = str;
        }

        void testNPE()
        {
            obj.toString();
            obj = null;
        }
    }
}

---------- END SOURCE ----------

CUSTOMER SUBMITTED WORKAROUND :
Inject an Objects.requireNonNull(obj) call as the first line in the publish method.


Comments
I was able to prove that this is a duplicate of JDK-8148752 which was fixed in 8u102 (see JDK-8149473). The problem is that C2 incorrectly inlines method handles with long or double arguments. The test program uses a lambda expression to call Event::public() and passes a long argument 'finalOpId': disruptor.publishEvent((event, sequence) -> event.publish(finalOpId, obj, str)); When compiled, C2 incorrectly treats the second stack slot occupied by the long argument as 'obj', resulting in incorrect code that causes a NullPointerException being thrown at runtime. Closing as duplicate.
11-04-2016

I'm able to reproduce this already with JDK 8 so this seems not to be a regression but an old problem. I'm also able to reproduce the failure with JDK 8u101 and with JDK 9 until b72. Not sure if the problem was fixed or is just hidden by one of the JDK 9 b72 changesets. I think this should be moved from Java Incidents to JDK hotspot/compiler because it clearly is a JIT compiler issue.
11-04-2016

ILW = Incorrect program execution, reproducible, disable compilation or modify code = HHL = P2
11-04-2016

Thanks [~thartmann] for your comments. Moving to jdk based on above comments and removing the regression label.
11-04-2016