JDK-6631352 : File{OutputStream,Writer} should implement atomic append mode using FILE_APPEND_DATA (win)
  • Type: Bug
  • Component: core-libs
  • Sub-Component: java.io
  • Affected Version: 7
  • Priority: P3
  • Status: Resolved
  • Resolution: Fixed
  • OS: windows_2000
  • CPU: generic
  • Submitted: 2007-11-17
  • Updated: 2010-04-02
  • Resolved: 2008-03-22
The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.
JDK 7
7 b25Fixed
Related Reports
Relates :  
JDK-6709457 - (fc) lock/tryLock() throws IOException "Access is denied" when file opened for append [win]
Relates :  
JDK-4960438 - (process) Need IO redirection API for subprocesses
Relates :  
JDK-6633375 - FileOutputStream_md.c should be merged into FileOutputStream.c
Description
It is possible to open a FileOutputStream (and FileWriter) in append mode.
This is implemented on Windows by opening the underlying file in the ordinary way,
and seeking to the end of the file before writing to the file.
Presumably this was done because it was not obvious how to open a
Windows file HANDLE in append mode.

This technique has two deficits: 

It is not atomic.  If multiple threads append to a file,
one of the appends may get lost.

If the underlying FileDescriptor or HANDLE is extracted from the FileOutputStream,
it is open in ordinary write mode, so does not inherit "append mode" semantics.
A "natively" append-mode HANDLE appears to be necessary to implement 
append mode redirection, as needed by
4960438: (process) Need IO redirection API for subprocesses
Comments
EVALUATION It is counterintuitive that the use of security permission flags like FILE_APPEND_DATA should affect the runtime behavior of a file handle, but the win32 API is not the best example of elegant software design.
17-11-2007
EVALUATION Here's a test case: import java.io.File; import java.io.FileOutputStream; import java.util.concurrent.Executors; import java.util.concurrent.ExecutorService; import java.util.concurrent.TimeUnit; public class AtomicAppend { // Before the fix for // 6631352: Implement atomic append mode using FILE_APPEND_DATA (win) // this would fail intermittently on windows void test(String[] args) throws Throwable { final int nThreads = 10; final int writes = 1000; final File file = new File("foo"); file.delete(); try { final ExecutorService es = Executors.newFixedThreadPool(nThreads); for (int i = 0; i < nThreads; i++) es.execute(new Runnable() { public void run() { try { FileOutputStream s = new FileOutputStream(file, true); for (int j = 0; j < 1000; j++) { s.write((int) 'x'); s.flush(); } s.close(); } catch (Throwable t) { unexpected(t); }}}); es.shutdown(); es.awaitTermination(10L, TimeUnit.MINUTES); equal(file.length(), (long) (nThreads * writes)); } finally { file.delete(); } } //--------------------- Infrastructure --------------------------- volatile int passed = 0, failed = 0; void pass() {passed++;} void fail() {failed++; Thread.dumpStack();} void fail(String msg) {System.err.println(msg); fail();} void unexpected(Throwable t) {failed++; t.printStackTrace();} void check(boolean cond) {if (cond) pass(); else fail();} void equal(Object x, Object y) { if (x == null ? y == null : x.equals(y)) pass(); else fail(x + " not equal to " + y);} public static void main(String[] args) throws Throwable { new AtomicAppend().instanceMain(args);} void instanceMain(String[] args) throws Throwable { try {test(args);} catch (Throwable t) {unexpected(t);} System.out.printf("%nPassed = %d, failed = %d%n%n", passed, failed); if (failed > 0) throw new AssertionError("Some tests failed");} }
17-11-2007
EVALUATION Indeed. If we look at the CreateFile API documentation http://msdn2.microsoft.com/en-us/library/aa363858.aspx ------------ You can get atomic append by opening a file with FILE_APPEND_DATA access and _without_ FILE_WRITE_DATA access. If you do this then all writes will ignore the current file pointer and be done at the end-of file. The append behavior is properly synchronized between multiple writes (with or without multiple handles), where the typical way I've seen this implemented (by seeking to EOF and then writing) has a race condition if multiple threads / processes are appending to the same file. ------------ @@ -190,9 +190,16 @@ jlong winFileHandleOpen(JNIEnv *env, jstring path, int flags) { + /* To implement O_APPEND, we use the strategy from + http://msdn2.microsoft.com/en-us/library/aa363858.aspx + "You can get atomic append by opening a file with + FILE_APPEND_DATA access and _without_ FILE_WRITE_DATA access. + If you do this then all writes will ignore the current file + pointer and be done at the end-of file." */ const DWORD access = - (flags & O_RDWR) ? (GENERIC_WRITE | GENERIC_READ) : + (flags & O_APPEND) ? FILE_APPEND_DATA : (flags & O_WRONLY) ? GENERIC_WRITE : + (flags & O_RDWR) ? (GENERIC_READ | GENERIC_WRITE) : GENERIC_READ; const DWORD sharing = FILE_SHARE_READ | FILE_SHARE_WRITE;
17-11-2007