JDK-6431344 : (fc) FileChannel.transferTo() doesn't work if address space runs out
  • Type: Bug
  • Component: core-libs
  • Sub-Component: java.nio
  • Affected Version: 1.4.2,5.0u16,6u13
  • Priority: P4
  • Status: Closed
  • Resolution: Fixed
  • OS: solaris_10,windows_xp
  • CPU: x86,sparc
  • Submitted: 2006-05-29
  • Updated: 2011-03-07
  • Resolved: 2011-03-07
The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.
JDK 7
7 b109Fixed
Related Reports
Duplicate :  
Relates :  
Relates :  
Relates :  
Description
FULL PRODUCT VERSION :
java version "1.4.2_03"
Java(TM) 2 Runtime Environment, Standard Edition (build 1.4.2_03-b02)
Java HotSpot(TM) Client VM (build 1.4.2_03-b02, mixed mode)


ADDITIONAL OS VERSION INFORMATION :
Microsoft Windows XP [Version 5.1.2600]


A DESCRIPTION OF THE PROBLEM :
If FileChannel.transferTo() or FileChannel.transferFrom() is used to transfer a large block of data, an IOException is unnecessarily thrown in certain cases.

Apparently, because of limitations of the Windows operating system, if the maximum heap size plus the length of the block to be transferred exceeds roughly 1.5 gigabytes, an IOException is thrown. This seems to be happening because the transferred block is always mapped to memory. On Microsoft Windows, it's possible to only allocate about 1.5 GB of address space in the Java VM. If the heap allocation and the transfer block allocation exceed this, an IOException is thrown.

The transferTo() and transferFrom() methods could detect situations where the whole transfer block can't be mapped to memory and make the transfer in smaller chunks, to avoid running out of address space. At least they could just return the number of bytes transferred, which might be less than what was requested to be transferred. This would be OK according to the specification. Throwing an IOException is just an unnecessary limitation specific to the implementation.

Apparently transfer lengths greater than 2 GB are outright rejected, by throwing an IOException.

This bug occurs also with Java 1.5.0 beta 1.


STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
1. Compile the provided source code
2. Make sure there is at least 6 gigabytes free space in the working drive
3. Run java -Xmx1G Test
4. Observe the output


EXPECTED VERSUS ACTUAL BEHAVIOR :
EXPECTED -
Nothing is output, the transfers succeed.

ACTUAL -
None of the transfers succeed, all throw an java.io.IOException.


ERROR MESSAGES/STACK TRACES THAT OCCUR :
java.io.IOException: Not enough storage is available to process this command
        at sun.nio.ch.FileChannelImpl.map0(Native Method)
        at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:705)
        at sun.nio.ch.FileChannelImpl.transferToTrustedChannel(FileChannelImpl.java:418)
        at sun.nio.ch.FileChannelImpl.transferTo(FileChannelImpl.java:491)
        at Test.transfer(Test.java:27)
        at Test.main(Test.java:9)
java.io.IOException: Not enough storage is available to process this command
        at sun.nio.ch.FileChannelImpl.map0(Native Method)
        at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:705)
        at sun.nio.ch.FileChannelImpl.transferToTrustedChannel(FileChannelImpl.java:418)
        at sun.nio.ch.FileChannelImpl.transferTo(FileChannelImpl.java:491)
        at Test.transfer(Test.java:27)
        at Test.main(Test.java:10)
java.io.IOException: The parameter is incorrect
        at sun.nio.ch.FileChannelImpl.map0(Native Method)
        at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:705)
        at sun.nio.ch.FileChannelImpl.transferToTrustedChannel(FileChannelImpl.java:418)
        at sun.nio.ch.FileChannelImpl.transferTo(FileChannelImpl.java:491)
        at Test.transfer(Test.java:27)
        at Test.main(Test.java:11)


REPRODUCIBILITY :
This bug can be reproduced always.

---------- BEGIN SOURCE ----------
import java.io.IOException;
import java.io.RandomAccessFile;
import java.nio.channels.FileChannel;

public class Test
{
    public static void main(String[] args)
    {
        transfer(1000000000L);
        transfer(2000000000L);
        transfer(3000000000L);
    }

    private static void transfer(long length)
    {
        try
        {
            RandomAccessFile fromFile = new RandomAccessFile("from.txt", "rw");
            RandomAccessFile toFile = new RandomAccessFile("to.txt", "rw");

            fromFile.setLength(length);
            toFile.setLength(length);

            FileChannel fromChannel = fromFile.getChannel();
            FileChannel toChannel = toFile.getChannel();

            fromChannel.transferTo(0, length, toChannel);

            fromFile.close();
            toFile.close();
        }
        catch (IOException ioe)
        {
            ioe.printStackTrace();
        }
    }
}

---------- END SOURCE ----------

CUSTOMER SUBMITTED WORKAROUND :
Manually transfer the data in small chunks. This defeats the purpose of the transferTo() and transferFrom() methods, and the performance isn't as good as possible, either.

Comments
EVALUATION For the trusted channel case, we should mmap in chunks.
29-05-2006