JDK-8136527 : ByteArrayOutputStream.grow uses Arrays.copyOf(buf, Integer.MAX_VALUE)
  • Type: Bug
  • Component: core-libs
  • Sub-Component: java.io
  • Affected Version: 8u25
  • Priority: P4
  • Status: Closed
  • Resolution: Duplicate
  • OS: os_x
  • CPU: x86
  • Submitted: 2015-03-16
  • Updated: 2016-02-02
  • Resolved: 2016-01-11
Related Reports
Duplicate :  
JDK-8055949 - ByteArrayOutputStream capacity should be maximal array size permitted by VM
Description
FULL PRODUCT VERSION :
java version "1.8.0_25"
Java(TM) SE Runtime Environment (build 1.8.0_25-b17)
Java HotSpot(TM) 64-Bit Server VM (build 25.25-b02, mixed mode)

java version "1.7.0_72"
Java(TM) SE Runtime Environment (build 1.7.0_72-b14)
Java HotSpot(TM) 64-Bit Server VM (build 24.72-b04, mixed mode)

ADDITIONAL OS VERSION INFORMATION :
Darwin braeburn 12.6.0 Darwin Kernel Version 12.6.0: Wed Dec 17 19:11:40 PST 2014; root:xnu-2050.48.15~1/RELEASE_X86_64 x86_64


EXTRA RELEVANT SYSTEM CONFIGURATION :
Does not matter.

A DESCRIPTION OF THE PROBLEM :
Application crashes with: java.lang.OutOfMemoryError: Requested array size exceeds VM limit
	at java.util.Arrays.copyOf(Arrays.java:2271)
	at java.io.ByteArrayOutputStream.grow(ByteArrayOutputStream.java:113)
	at java.io.ByteArrayOutputStream.ensureCapacity(ByteArrayOutputStream.java:93)
	at java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:122)

This happens because in ByteArrayOutputStream.grow() Arrays.copyOf is called with Integer.MAX_VALUE 
which does not work. The newCapacity should be smaller e.g. Integer.MAX_VALUE - 2 seems to work. 



STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
Create the following junit test and run it with -Xmx4069m:

  @Test
  public void testByteArrayOutputStream() throws IOException {
    byte[] buf = new byte[0x20000000];
    ByteArrayOutputStream out = new ByteArrayOutputStream(0x3fffffff);
    out.write(buf);
    out.write(buf);
  }


EXPECTED VERSUS ACTUAL BEHAVIOR :
EXPECTED -
No crash.
ACTUAL -
java.lang.OutOfMemoryError: Requested array size exceeds VM limit
	at java.util.Arrays.copyOf(Arrays.java:3236)
	at java.io.ByteArrayOutputStream.grow(ByteArrayOutputStream.java:113)
	at java.io.ByteArrayOutputStream.ensureCapacity(ByteArrayOutputStream.java:93)
	at java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:140)
	at java.io.OutputStream.write(OutputStream.java:75)
	at ArrayTest.testByteArrayOutputStream(ArrayTest.java:21)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:76)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:193)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:52)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:191)
	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:42)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:184)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:236)
	at org.junit.runner.JUnitCore.run(JUnitCore.java:157)
	at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:74)
	at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:211)
	at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:67)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at com.intellij.rt.execution.application.AppMain.main(AppMain.java:134)


ERROR MESSAGES/STACK TRACES THAT OCCUR :
java.lang.OutOfMemoryError: Requested array size exceeds VM limit
	at java.util.Arrays.copyOf(Arrays.java:3236)
	at java.io.ByteArrayOutputStream.grow(ByteArrayOutputStream.java:113)
	at java.io.ByteArrayOutputStream.ensureCapacity(ByteArrayOutputStream.java:93)
	at java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:140)
	at java.io.OutputStream.write(OutputStream.java:75)
	at ArrayTest.testByteArrayOutputStream(ArrayTest.java:21)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:76)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:193)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:52)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:191)
	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:42)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:184)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:236)
	at org.junit.runner.JUnitCore.run(JUnitCore.java:157)
	at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:74)
	at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:211)
	at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:67)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at com.intellij.rt.execution.application.AppMain.main(AppMain.java:134)


REPRODUCIBILITY :
This bug can be reproduced occasionally.

---------- BEGIN SOURCE ----------
import org.junit.Test;

import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.util.Arrays;

public class ArrayTest {

  @Test
  public void testByteArrayOutputStream() throws IOException {
    byte[] buf = new byte[0x20000000];
    ByteArrayOutputStream out = new ByteArrayOutputStream(0x3fffffff);
    out.write(buf);
    out.write(buf);
  }
}
---------- END SOURCE ----------
Comments
This is due to JVM being unable to allocate/commit-reserved memory. Java 8 and 9 have a slightly different requirements for memory, that's why one is throwing OOM, while the other fails the whole jvm. On my local system, with -Xmx3g both 8 and 9 throw OOM. And with -Xmx5g both halt the jvm due to lack of memory.
11-01-2016
This is fixed with JDK-8055949 and is not throwing the "OutOfMemoryError: Requested array size exceeds VM limit" with 8u60/9 versions. However, when trying to run the attached test case had an observation. When running the Test case with argument -Xmx4096m , in the case of JDK 9ea, it ran silently. However, in case of JDK8u60 or 66, it throws the java.lang.OutOfMemoryError: Java heap space. It may be some optimizations in JDK 9, however passing over to dev team to take a look.
15-09-2015
Dup of JDK-8055949
17-03-2015