JDK-8191002 : Arrays.fill() in finalizer mangles newly allocated array in another thread
  • Type: Bug
  • Component: hotspot
  • Sub-Component: gc
  • Affected Version: 8,9,10
  • Priority: P3
  • Status: Closed
  • Resolution: Duplicate
  • OS: generic
  • CPU: x86_64
  • Submitted: 2017-11-06
  • Updated: 2017-11-14
  • Resolved: 2017-11-13
Related Reports
Duplicate :  
Duplicate :  
Relates :  
Description
FULL PRODUCT VERSION :
java version "1.8.0_112"
Java(TM) SE Runtime Environment (build 1.8.0_112-b15)
Java HotSpot(TM) 64-Bit Server VM (build 25.112-b15, mixed mode)

FULL OS VERSION :
OS Name:                   Microsoft Windows 7 Enterprise 
OS Version:                6.1.7601 Service Pack 1 Build 7601


EXTRA RELEVANT SYSTEM CONFIGURATION :
System Manufacturer: VMware, Inc.
System Model: VMware Virtual Platform
System Type: x64-based PC
Processor(s): 2 CPUs x Intel64 Family 6 Model 62 Stepping 4 GenuineIntel ~2693 

A DESCRIPTION OF THE PROBLEM :
In a multi-threaded scenario it is possible that a finalizer performing Arrays.fill() on an instance field may actually mangle the contents of a fresh array allocated by a different thread.
The problem was originally reported on StackOverflow as https://stackoverflow.com/q/46971788/1654233 where user was getting sporadic failures while trying to decrypt some data in multiple threads with PBKDF2WithHmacSHA1 algorithm. Debugging revealed that com.sun.crypto.provider.PBKDF2KeyImpl.getEncoded() method sometimes returns a clone of the field key:byte[] populated with all zeroes which seems to be impossible in the scenario.
After some research it become clear that the problem can be reproduced with the snippet below and the only thing that matters is an array operation performed in finalizer.


THE PROBLEM WAS REPRODUCIBLE WITH -Xint FLAG: No

THE PROBLEM WAS REPRODUCIBLE WITH -server FLAG: Yes

STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
Compile and run the test case with -Xmx10m (can be a larger number but this produces error in a more predictable manner).

REPRODUCIBILITY :
This bug can be reproduced occasionally.

---------- BEGIN SOURCE ----------
package bugs;

import javax.xml.bind.DatatypeConverter;
import java.nio.charset.StandardCharsets;
import java.util.Arrays;
import java.util.Random;
import java.util.stream.IntStream;

public class NewlyAllocatedArrayFilledByOtherInstanceFinalizer
{
    void demonstrate() throws Exception
    {
        final int iterations = 1000000;
        IntStream.range(0, iterations)
                 .parallel()
                 .forEach(i ->
                 {
                     String expectedValue = randomAlphaNumeric(10);
                     byte[] expectedBytes = expectedValue.getBytes(StandardCharsets.UTF_8);
                     ArrayHolder holder = new ArrayHolder(expectedBytes);
                     byte[] actualBytes = holder.getBytes();
                     if (!Arrays.equals(expectedBytes, actualBytes))
                     {
                         String actualValue = new String(actualBytes, StandardCharsets.UTF_8);
                         System.err.printf("Assertion failed: expected='%s' actual='%s' (bytes: %s)%n",
                             expectedValue, actualValue, DatatypeConverter.printHexBinary(actualBytes));
                     }
                 });
    }

    static class ArrayHolder
    {
        private byte[] _bytes;

        ArrayHolder(final byte[] bytes) { _bytes = bytes.clone(); }

        byte[] getBytes() { return _bytes.clone(); }

        @Override
        protected void finalize() throws Throwable
        {
            if (_bytes != null)
            {
                Arrays.fill(_bytes, (byte) 'z');
                _bytes = null;
            }
            super.finalize();
        }
    }

    private static final String ALPHA_NUMERIC_STRING = "ABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789";
    private static final Random RND = new Random();

    static String randomAlphaNumeric(int count)
    {
        final StringBuilder sb = new StringBuilder();
        while (count-- != 0)
        {
            int character = RND.nextInt(ALPHA_NUMERIC_STRING.length());
            sb.append(ALPHA_NUMERIC_STRING.charAt(character));
        }
        return sb.toString();
    }

    public static void main(String[] args)
        throws Exception
    {
        new NewlyAllocatedArrayFilledByOtherInstanceFinalizer().demonstrate();
    }
}

---------- END SOURCE ----------

CUSTOMER SUBMITTED WORKAROUND :
Not identified.


Comments
That the use of Reference.reachabilityFence() fixes the test suggests this could be considered a duplicate of JDK-8055183 or JDK-8177915. One method for achieving the effect of Reference.reachabilityFence pre-jdk9 is "synchronized(x) {}" (as mentioned in JDK-8055183), where in cases like this one "x" would be "this".
13-11-2017

[~kbarrett]: that seems to fix this issue for jdk9+ - thanks! - and makes sense. One problem remaining is that Reference.reachabilityFence() is not available in pre-jdk9 releases.
13-11-2017

The failure might go away if ArrayHolder.getBytes() were changed thusly: byte[] getBytes() { byte[] result = _bytes.clone(); Reference.reachabilityFence(this); return result; } Without that reachability fence, I don't see anything that would ensure the holder doesn't get finalized before the end of the clone call in getBytes.
13-11-2017

This issue is reproducible on latest version of 8 9 and 10. jdk-10-ea+26/bin/java -Xmx10m --add-modules java.xml.bind --add-exports java.xml.bind/javax.xml.bind=ALL-UNNAMED NewlyAllocatedArrayFilledByOtherInstanceFinalizer Assertion failed: expected='EGAKV13GF8' actual='zzzzzzzzzz' (bytes: 7A7A7A7A7A7A7A7A7A7A) Assertion failed: expected='IM5EORW15U' actual='zzzzzzzzzz' (bytes: 7A7A7A7A7A7A7A7A7A7A) Assertion failed: expected='NGBAH1BPQ6' actual='zzzzzzzzzz' (bytes: 7A7A7A7A7A7A7A7A7A7A) Assertion failed: expected='K1S7HRD5XI' actual='zzzzzzzzzz' (bytes: 7A7A7A7A7A7A7A7A7A7A) Assertion failed: expected='YPC3B16H1R' actual='zzzzzzzzzz' (bytes: 7A7A7A7A7A7A7A7A7A7A) Assertion failed: expected='RRTOULQU6O' actual='zzzzzzzzzz' (bytes: 7A7A7A7A7A7A7A7A7A7A) Assertion failed: expected='8STCJR1M00' actual='zzzzzzzzzz' (bytes: 7A7A7A7A7A7A7A7A7A7A) Assertion failed: expected='D11KFCR5PA' actual='zzzzzzzzzz' (bytes: 7A7A7A7A7A7A7A7A7A7A) Assertion failed: expected='4ISTMK06OE' actual='zzzzzzzzzz' (bytes: 7A7A7A7A7A7A7A7A7A7A) Exception in thread "main" java.lang.OutOfMemoryError
09-11-2017