JDK-6468588 : (fc) Ability to utilise advantages of NCQ in hard drives - java.nio.FileChannel
  • Type: Enhancement
  • Component: core-libs
  • Sub-Component: java.nio
  • Affected Version: 5.0
  • Priority: P5
  • Status: Closed
  • Resolution: Duplicate
  • OS: linux
  • CPU: x86
  • Submitted: 2006-09-08
  • Updated: 2011-02-16
  • Resolved: 2009-03-20
Related Reports
Duplicate :  
Okay, Java doesn't currently support non-blocking I/O for files. I understand that hordes of engineers are rushing to solve this issue. However, what I would like to see is a much more simple (to the application programmer) method of improving file I/O performance in some programs, and that is to allow Java to provide a list of file access requests, which can be passed by the OS directly to the OS. If the hard drive being accessed supports NCQ, then it will re-order the accesses as it sees fit, giving a performance improvement.

The situation is this: I have a large file, and I know that I want to read several different locations, however I don't care the order that this is done. I could just iterate over the list of accesses. This would work perfectly well, but the OS or the hard drive could do it better.

I suggest a new method in the FileChannel interface:

int[] read(ByteBuffer dsts[], long positions[]) throws IOException;

Both input arrays would be expected to have the same length, and the return array would have the same length again.

I have not specified an equivalent write method because writes can already be reordered by the OS and hard drive.

I justify this enhancement on two levels:

1. This would allow the advantages rapidly becoming available to the masses in NCQ-enabled hard drives to be gained by Java programs that need to read from several positions in a file.
2. Implementing this extra method is a completely trivial exercise. Implementing it so that it actually provides a performance improvement is slightly more tricky, but that's beside the point. The method can be added to the API right away without having to do any work. The hard graft of making it actually worthwhile can be done later.

---------- BEGIN SOURCE ----------
Here I provide source code for just the single method. This chunk should be pasted into the java.nio.FileChannel class:

public int[] read(ByteBuffer dsts[], long positions[]) throws IOException {
    int retval[] = new int[dsts.length];
    for (int i = 0; i < dsts.length; i++) {
        retval[i] = read(dsts[i], positions[i]);
    return retval;
---------- END SOURCE ----------

EVALUATION Here is a sample method that is close to what has been requested here. This sample is using the new AsynchronousFileChannel method to initiate many I/O operations in parallel. If NCQ is supported then it will be used transparently by the operating system. int[] readMany(AsynchronousFileChannel ch, ByteBuffer dsts[], long positions[]) throws IOException { int n = dsts.length; Future<?>[] result = new Future<?>[n]; int[] nread = new int[n]; // initiate the I/O operations for (int i=0; i<n; i++) { result[i] = ch.read(dsts[i], positions[i]); } // collect the results int i = 0; while (i < n) { try { nread[i++] = (Integer)result[i].get(); } catch (InterruptedException x) { // ignore } catch (ExecutionException x) { Throwable cause = x.getCause(); if (cause instanceof IOException) throw (IOException)cause; throw new IOException(cause); } } return nread; }

EVALUATION As part of JSR-203 we are defining an API that will allow an application do asynchronous I/O on files and that will likely meet the requirement here.