JDK-4842430 : File should support creation of sparse files
  • Type: Enhancement
  • Component: core-libs
  • Sub-Component: java.io
  • Affected Version: 1.4.1
  • Priority: P4
  • Status: Closed
  • Resolution: Duplicate
  • OS: windows_2000
  • CPU: x86
  • Submitted: 2003-04-03
  • Updated: 2009-02-16
  • Resolved: 2009-02-16
Related Reports
Duplicate :  
Description
Name: rmT116609			Date: 04/03/2003


DESCRIPTION OF THE REQUEST :
Developers on modern operating systems such as Linux have long enjoyed automatic support for "sparse files" where doing a long seek past the end of the file, and then writing a little bit of data will avoid writing a bunch of 0's to disk, thus wasting precious disk space and causing a huge performance penalty for large files.

For instance, an application that skips ahead 1GB and writes a single byte of information can take anywhere from 1 minute to 5 minutes to execute that single write operation due to the fact that 1 GB of 0's must be written to disk.

NTFS5 on Win2k (and maybe XP?) finally includes support for sparse files, but it is not turned on by default, nor is there any way for Java applications to enable this feature.


JUSTIFICATION :
Sparse file support can drastically improve performance for applications that do long seeks beyond the end of the file.  Write operations that used to take over a minute, now take milliseconds.

Since sparse file support is completely transparent to applications that are not aware of it, there is no reason not to use sparse files by default on file systems that support it.

Indeed, sparse files are already used by Java on unix operating systems (because they are an automatic feature of the file system), so there is no reason why sparse files shouldn't be used by default on NTFS5.

EXPECTED VERSUS ACTUAL BEHAVIOR :
Using normal RandomAccessFile methods to seek() past the end of a file and write() a few bytes should only allocate the actual data written, not a bunch of 0's that were seek()'d over.
seek()'ing past the end of a file and write()'ing a few bytes on Windows can take many minutes depending on the size of the seek, when it should be taking on the order of milliseconds.

CUSTOMER SUBMITTED WORKAROUND :
None.  Perhaps some JNI could be written.
(Review ID: 183438) 
======================================================================

Comments
EVALUATION This feature has been addressed by the new file system API defined by JSR-203.
16-02-2009

EVALUATION The bug requests that sparse files be created by default at least by RandomAccessFile. It is probably inappropriate to do this for all files, but adding an API to File (or a new constructor) to explicitly request creation of a sparse file is reasonable. RAF could be modified to always (or optionally) call that API. Random access files are used by applications on many systems in a way where system support for sparse files is of benefit. Existing programs, such as data bases that are implemented in C often take advantage of this. It should be possible to do so in Java as well on OSs that support this. As noted in the description, we already support this for unix. To support this on NTFS, will need to add a call to the win32 API DeviceIoControl with the FSCTL_SET_SPARSE flag after the file has been created. We might also need to add File APIs to get the compressed verses total size of the sparse file.
16-11-2005