United StatesChange Country, Oracle Worldwide Web Sites Communities I am a... I want to...
Bug ID: JDK-6265734 (fc) single FileChannel slower than using multiple FileChannels (win)
JDK-6265734 : (fc) single FileChannel slower than using multiple FileChannels (win)

Details
Type:
Bug
Submit Date:
2005-05-05
Status:
In Progress
Updated Date:
2013-08-13
Project Name:
JDK
Resolved Date:
Component:
core-libs
OS:
windows_xp
Sub-Component:
java.nio
CPU:
x86
Priority:
P3
Resolution:
Unresolved
Affected Versions:
5.0
Targeted Versions:
tbd_major

Related Reports
Duplicate:

Sub Tasks

Description
FULL PRODUCT VERSION :
fails with 1.5_02 and 1.4.2_08

ADDITIONAL OS VERSION INFORMATION :
Windows XP SP 2

A DESCRIPTION OF THE PROBLEM :
The nio package is designed for multithreaded access, but there is a strange performance problem with FileChannel. Using multiple FileChannels with multiple RandomAccessFiles (against the same physical file), is faster than using a single FileChannel against the file. How come?

I originally thought it might be due to internal syncrhonization in the FileChannel, but according to the JavaDoc, the methods are synchronized only if you use the non-positional reads.

Under Linux the times for both are nearly identical - which is what is expected.


STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
run the supplied test program. you can change the usage of multiple FileChannel by changing the MULTICHANNEL variable.

You can also alter the RANDOM variable to control whether or not random vs. sequential reads are performed. In either case, the multichannel version is faster.

EXPECTED VERSUS ACTUAL BEHAVIOR :
EXPECTED -
I would expect the time to be be similar, or that the multiple FileChannel be slower, since the OS may need to do some synchronization due to the multiple file descriptors.
ACTUAL -
The 'multiple' FileChannel version is faster than the single FileChannel version by almost 2 to 1 on my hardware. Having others test it, the difference was not so great, but the multiple channel version always performs much better.

REPRODUCIBILITY :
This bug can be reproduced always.

---------- BEGIN SOURCE ----------
import java.nio.ByteBuffer;
import java.nio.channels.FileChannel;
import java.util.Random;

public class MultiThreadIO {
    static final int NTHREADS = 8;
    static final boolean MULTICHANNEL = true;
    static final boolean RANDOM = true;
    static final int BLOCKSIZE = 1024;
    static final int NBLOCKS = 200000;
    
    static File file = new File("testfile");

    public static void main(String[] args) throws Exception {
        
        RandomAccessFile f = new RandomAccessFile(file,"rw");
        FileChannel ch = f.getChannel();
        
        byte[] buffer = new byte[BLOCKSIZE];
        
        System.err.println("writing...");
        
        for(int i=0;i<NBLOCKS;i++){
            ch.write(ByteBuffer.wrap(buffer),i*(long)BLOCKSIZE);
        }
        
        System.err.println("starting readers");
        
        long stime = System.currentTimeMillis();
        
        Thread[] thread = new Thread[NTHREADS];
        for(int i=0;i<NTHREADS;i++) {
            thread[i] = new Reader(file,ch,i);
        }
        for(int i=0;i<NTHREADS;i++) {
            thread[i].start();
        }
        for(int i=0;i<NTHREADS;i++) {
            thread[i].join();
        }
        
        System.err.println("multichannel="+MULTICHANNEL+", time = "+(System.currentTimeMillis()-stime));
    }
    
    static class Reader extends Thread {
        File f;
        FileChannel ch;
        RandomAccessFile ra;
        int n;
        
        Random r = new Random();
        
        public Reader(File file, FileChannel channel,int n) throws FileNotFoundException {
            f = file;
            ch = channel;
            this.n = n;
            
            if(MULTICHANNEL)
                ch = new RandomAccessFile(file,"rw").getChannel();
        }

        public void run() {
            byte[] buffer = new byte[BLOCKSIZE];
            for(int i=0;i<NBLOCKS;i++){
                try {
                    if(RANDOM)
                        ch.read(ByteBuffer.wrap(buffer),r.nextInt(NBLOCKS)*BLOCKSIZE);
                    else
                        ch.read(ByteBuffer.wrap(buffer),i*BLOCKSIZE);
                } catch (IOException e) {
                    e.printStackTrace();
                }
            }
            System.err.println("reader "+n+" done");
        }
    }
}

---------- END SOURCE ----------

CUSTOMER SUBMITTED WORKAROUND :
write code to manage a pool of FileChannel objects and use a different FileChannel from each thread.

This does not seem to be how nio was designed to be used.

Possible cause... the multiple file descriptiors cause Windows to view them as different files, so each is going to get its own buffer, but this would lead to consistency problems unless the FileChannels are extenally synchronized, and the writer "syncs" its data to disk (before the readers attempt to read the file), but if this were the case there could be buffering on the readers, since each would need to always read from disk. Strange.
###@###.### 2005-05-05 05:42:25 GMT

                                    

Comments
EVALUATION

Most likely the issue here is that Windows doesn't have true pread/pwrite-like support that doesn't impact the global file position. The implication is that the Windows implementation requires additional synchronization that is not required on other platforms.
                                     
2006-06-21
WORK AROUND

For those using jdk7 builds then AsynchronousFileChannel does not suffer from this issue (no global file position).
                                     
2010-08-23
EVALUATION

The issue we have with ReadFile/WriteFile changing the global file position still remains but if we change the positionLock to be a RW lock then we should be able to allow concurrent read/write if we synchronize against operations that depend on the global file position.
                                     
2010-08-23



Hardware and Software, Engineered to Work Together