JDK-8200282 : Serializing non-zero byte as zero to ByteBuffer
  • Type: Bug
  • Component: hotspot
  • Sub-Component: compiler
  • Affected Version: 9,10,11
  • Priority: P2
  • Status: Closed
  • Resolution: Fixed
  • OS: generic
  • CPU: x86_64
  • Submitted: 2018-03-23
  • Updated: 2019-09-13
  • Resolved: 2018-07-13
The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.
JDK 11 JDK 12
11 b23Fixed 12Fixed
Related Reports
Relates :  
Description
FULL PRODUCT VERSION :
java version "10" 2018-03-20
Java(TM) SE Runtime Environment 18.3 (build 10+46)
Java HotSpot(TM) 64-Bit Server VM 18.3 (build 10+46, mixed mode)


FULL OS VERSION :
Linux XXX 3.10.0-693.17.1.el7.x86_64 #1 SMP Sun Jan 14 10:36:03 EST 2018 x86_64 x86_64 x86_64 GNU/Linux


A DESCRIPTION OF THE PROBLEM :
We have code that takes an IP address and serializes it to a ByteBuffer.  We know that the last octet of the address is non-zero, but in the ByteBuffer, the last octet is written as a zero.  Only the last octet is affected -- the first three bytes of the address are correctly non-zero in the ByteBuffer after serialization.  

The bug only occurs sometimes in production, but we have reproducer code which we'll attach to this bug report where it happens consistently.  It also doesn't happen immediately.  Instead, it occurs only after the code has run a few thousand times, which is evidence that it's a JIT bug.  

Note that the bug is absent from all versions of Java 8 up to and including Update 162.  It appears in Java 9.0.4 and Java 10, however, and it also appears in the early access candidate of Java 11. 

Also note that, though this test uses an InetSocketAddress to generate the 4 bytes to test, the bug has nothing to do with networking code.  I have an alternate test case that randomly generates 4-byte arrays to serialize which also exhibits the problem.  But this other test case exhibits the bug less consistently then the attached test case, so I'm not submitting it here.  Feel free to contact me if you'd like the alternate test case.

THE PROBLEM WAS REPRODUCIBLE WITH -Xint FLAG: No

THE PROBLEM WAS REPRODUCIBLE WITH -server FLAG: Yes

REGRESSION.  Last worked in version 8u162

STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
Compile the source code below with "javac ByteBufferTest.java".
Execute the program with "java ByteBufferTest".



EXPECTED VERSUS ACTUAL BEHAVIOR :
If the code runs to completion without printing anything, there is no bug.  If it prints something, this means it's detected a situation where it has serialized an address but then found a zero in the last octet of the address when it should be non-zero.  
ERROR MESSAGES/STACK TRACES THAT OCCUR :
If the test program prints anything, it means that there's an error.  Sample output will look something like this:

write(): Detected 0 byte in last octet after 24802 th iteration. Original addr bytes written were [-17, -63, -31, 53], read bytes are [-17, -63, -31, 0]
main(): Full Buffer contents when issue is detected: [14, -49, -76, -106, 25, -66, -99, -20, -4, -124, -69, -42, 8, -34, 92, -43, -23, 5, 123, -65, 100, 82, 100, -57, -37, -94, -16, 80, 0, 109, -17, -63, -31, 0, -29, -23, -26, -57, 64, -58, -44, -102]
write(): Detected 0 byte in last octet after 70289 th iteration. Original addr bytes written were [-17, -63, -31, 55], read bytes are [-17, -63, -31, 0]
main(): Full Buffer contents when issue is detected: [13, -61, 94, -31, 122, -44, 81, 122, 91, -10, 70, 117, -78, -33, 66, 12, 104, -60, 65, 116, -109, 35, 113, -12, 97, -39, -1, 74, 0, 109, -17, -63, -31, 0, -93, 14, -111, -102, 64, -73, -99, 69]

Note that the buffer contains more than just the multicast address itself.  It was difficult to get the bug to appear in a test program that serialized only the address.


REPRODUCIBILITY :
This bug can be reproduced always.

---------- BEGIN SOURCE ----------
import java.net.InetSocketAddress;
import java.nio.ByteBuffer;
import java.util.Arrays;
import java.util.Random;

/**
 * Test to illustrate likely JIT bug in which
 * the fourth byte of an IP address, which is known
 * to be non-zero, is serialized as zero.
 */
public class ByteBufferTest
{
    private static String[] addresses = {"239.193.225.53",
                                         "239.193.225.54",
                                         "239.193.225.55",
                                         "239.193.225.56"};

    public static void main(String[] args)
    {
        ByteBuffer buf = ByteBuffer.allocate(256);

        for (int i = 1; i < 100000; i++) {
            buf.position(0);
            buf.limit(42);

            write(buf, i);

            // Read the full buffer content into a byte[] and
            // log if 0 byte was detected at location of last octet.
            buf.position(0);
            byte[] fullBuf = new byte[buf.remaining()];
            buf.get(fullBuf);

            if (fullBuf[33] == 0) {
                System.out.println("main(): Full Buffer contents when issue is detected: "
                                   + Arrays.toString(fullBuf));
            }
            buf.clear();
        }
    }

    private static void write(ByteBuffer buf, int itrCount)
    {
        Random r = new Random();

        // put some ints
        for (int j = 0; j < 5; j++) {
            buf.putInt(r.nextInt());
        }
        buf.putLong(r.nextLong());
        buf.putChar('m');

        // Pick a multicast address from "addresses" at random
	// and serialize it to the buffer.  Note that we know
	// by contruction that the last octet of the address 
	// is non-zero.
        byte[] mcastaddr = new InetSocketAddress(addresses[r.nextInt(4)], 9020)
            .getAddress().getAddress();
        buf.put(mcastaddr);

        // Read back the multicast address bytes that were just written
        // into the buffer.
        buf.position(buf.position() - 4);
        byte[] readAddrBytes = new byte[4];
        buf.get(readAddrBytes);

        // Check if there was a 0 in the last octet of multicast address.
        if (readAddrBytes[3] == 0) {

            // For some reason, this issue does not seem to happen often if
            // we directly log mcastaddr original address bytes.
            // So we copy this into a new temp array for logging.
            byte[] addrBytesCopy = new byte[4];
            System.arraycopy(mcastaddr, 0, addrBytesCopy, 0, 4); 
            
            System.out.println("write(): Detected 0 byte in last octet after "
                               + itrCount + " th iteration. Original addr bytes written were: "
                               + Arrays.toString(addrBytesCopy)
                               + "; bytes read from buffer are: " + Arrays.toString(readAddrBytes)
                               );

        }
        buf.putLong(r.nextLong());
    }

}

---------- END SOURCE ----------

CUSTOMER SUBMITTED WORKAROUND :
Small, seemingly random changes to the code may prevent the bug from occurring.  For example, adding some logging or other tests to the code seem to fix the problem.  But this is unpredictable and unreliable, and not really a workaround.


Comments
Adding Roland to the watchlist.
06-07-2018

I've narrowed this down to the _arraycopy C2 intrinsic. Workarounds are -XX:DisableIntrinsic=_arraycopy or -XX:CompileCommand=exclude,ByteBufferTest::write Since this starts to fail with JDK 9 b66, it's very likely that the bug has been introduced by JDK-8076188 (which already had a rather large bug tail). I've verified this with the following change that basically disables JDK-8076188: diff -r c30c35118303 src/hotspot/share/opto/escape.cpp --- a/src/hotspot/share/opto/escape.cpp Tue Jul 03 02:07:49 2018 +0200 +++ b/src/hotspot/share/opto/escape.cpp Fri Jul 06 11:52:24 2018 +0200 @@ -1051,7 +1051,7 @@ ac->is_arraycopy_validated() || ac->is_copyof_validated() || ac->is_copyofrange_validated()) { - es = PointsToNode::NoEscape; + // es = PointsToNode::NoEscape; } } set_escape_state(arg_ptn, es); Updated-ILW = Incorrect execution of compiled code, easy to reproduce with test but never showed up in other applications, disable arraycopy intrinsic or compilation of affected method = HMM = P2
06-07-2018

Couple of other tests to conclude issue with Tiered compilation 1. interpreter (-Xint) java -Xint ByteBufferTest ==> Pass 2. C1 compiler java -XX:TieredStopAtLevel=1 -XX:+TieredCompilation ByteBufferTest ==> Pass 3. C2 compiler java -XX:TieredStopAtLevel=4 -XX:-TieredCompilation ==> Pass 4. Tiered java -XX:TieredStopAtLevel=4 -XX:+TieredCompilation ==> Fail 5. Tiered Graal java -XX:+UnlockExperimentalVMOptions -XX:+EnableJVMCI -XX:+UseJVMCICompiler -XX:+TieredCompilation ==> Pass
26-03-2018

This issue is reproducible in 9GA, 10GA and 11 ea b05. It is regression introduced in 9 ea b66 onwards 8u172 b05 - Pass 9 ea b65 - Pass 9 ea b66 - Fail // Regression introduced here. 9 GA - Fail 10 GA - Fail 11 ea b05 - Fail
26-03-2018