JDK-6371642 : (fs) Concurrent read or write and file length change not properly synchronized
  • Type: Bug
  • Component: core-libs
  • Sub-Component: java.nio
  • Affected Version: 5.0,7
  • Priority: P4
  • Status: Closed
  • Resolution: Future Project
  • OS: windows_xp,windows_vista
  • CPU: x86
  • Submitted: 2006-01-12
  • Updated: 2012-01-07
  • Resolved: 2010-08-31
The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.
Other
tbd_majorResolved
Related Reports
Relates :  
Description
FULL PRODUCT VERSION :
java version "1.5.0_06"
Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_06-b05)
Java HotSpot(TM) Client VM (build 1.5.0_06-b05, mixed mode, sharing)

ADDITIONAL OS VERSION INFORMATION :
Microsoft Windows XP [Version 5.1.2600]

EXTRA RELEVANT SYSTEM CONFIGURATION :
Intel 3.06GHz P4 --- hyperthreaded

A DESCRIPTION OF THE PROBLEM :
If a FileChannel.write(ByteBuffer b, long position) or FileChannel.read(ByteBuffer b, long position) takes place at the same time as a RandomAccessFile.setLength call on another thread where the channel is the one associated with the RandomAccessFile, then the read or write may take place at an unexpected position or the setLength may result in an unexpected file length.
These read and write methods are supposed to be independent of methods which affect the file position (and setLength is not declared as such anyway although that is probably how it is implemented).

STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
Just run the attached code. As with most multithreading problems the result is somewhat variable but on my machine it does hit one of the failure points every time (which one varies).

Changing the code to use a common lock for read/write and setLength operations prevents the problem occuring.

EXPECTED VERSUS ACTUAL BEHAVIOR :
EXPECTED -
A successful run should just print "Test completed"
ACTUAL -
Invalid data in file: 0,0 != 3,15
Test completed



REPRODUCIBILITY :
This bug can be reproduced often.

---------- BEGIN SOURCE ----------
import java.io.RandomAccessFile;
import java.io.File;
import java.io.IOException;
import java.nio.channels.FileChannel;
import java.nio.ByteBuffer;
import java.nio.ByteOrder;
import java.util.Random;

/** Test concurent read, write and length changes.
 * It appears that RandomAccessFile.setLength interacts with FileChannel.write
 * @author mThornton
 * @since 22-Dec-2005 11:23:52
 */
public class TestConcurrency implements Runnable
{
	private static final int PAGE_SIZE = 1024;

	private int group=8;
	private File file;
	private RandomAccessFile raf;
	private FileChannel channel;
	private int size;
	private int maximumSize;
	private volatile boolean complete;
	private Object lock = new Object();
	
	private Object extendLock = new Object();
	// private Object extendLock = lock; //new Object();

	public static void main(String[] args)
	{
		TestConcurrency test = new TestConcurrency();
		try
		{
			test.init(10, 1000);
			new Thread(test).start();
			test.modifyFileContent();
		}
		catch (IOException e)
		{
			e.printStackTrace();
		}
		test.cleanup();
		System.out.println("Test completed");
	}

	void init(int initialSize, int maxSize) throws IOException
	{
		file = File.createTempFile("testConcurrency", "tmp");
		raf = new RandomAccessFile(file, "rw");
		channel = raf.getChannel();
		size = initialSize;
		raf.setLength(size*(long)PAGE_SIZE);
		maximumSize = maxSize;
	}

	void cleanup()
	{
		if (raf != null)
		{
			try
			{
				raf.close();
			}
			catch (IOException e)
			{
			}
		}
		if (channel != null)
		{
			try
			{
				channel.close();
			}
			catch (IOException e)
			{
			}
		}
		if (file != null)
		{
			if (!file.delete())
				System.err.println("Unable to delete temporary file: "+file);
		}
	}

	private synchronized int size()
	{
		return size;
	}

	private void extendFile()
	{
		try
		{
			while (size < maximumSize)
			{
				synchronized (this)
				{
					wait();
				}
				if (complete)
					break;
				int n = size + 20;
				synchronized (extendLock)
				{
					raf.setLength(n*(long)PAGE_SIZE);
				}
				synchronized (this)
				{
					size = n;
				}
			}
		}
		catch (IOException e)
		{
			e.printStackTrace();
		}
		catch (InterruptedException e)
		{
		}
		complete = true;
	}

	private synchronized void trigger()
	{
		notify();
	}

	private void modifyFileContent()
	{
		ByteBuffer buffer = ByteBuffer.allocateDirect(PAGE_SIZE);
		buffer.order(ByteOrder.nativeOrder());
		Random rand = new Random();
		int sequence = 0;
		try
		{
			test:
			while (!complete)
			{
				sequence++;
				int limit = size();
				int index = rand.nextInt(limit-group);
				// first write data
				long position = index*(long)PAGE_SIZE;
				trigger();
				for (int i=0; i<group; i++, position += PAGE_SIZE)
				{
					buffer.clear();
					buffer.putInt(sequence);
					buffer.putInt(index+i);
					buffer.clear();
					int n;
					synchronized (lock)
					{
						n = channel.write(buffer, position);
					}
					if (n != buffer.capacity())
					{
						System.err.println("Incomplete write: "+n);
						break test;
					}
				}
				// now verify data
				position = index*(long)PAGE_SIZE;
				for (int i=0; i<group; i++, position += PAGE_SIZE)
				{
					buffer.clear();
					int n;
					synchronized (lock)
					{
						n = channel.read(buffer, position);
					}
					if (n != buffer.capacity())
					{
						System.err.println("Incomplete read: "+n);
						break test;
					}
					buffer.flip();
					int s = buffer.getInt();
					int x = buffer.getInt();
					if (s != sequence || x != index+i)
					{
						System.err.println("Invalid data in file: "+s+","+x+" != "+sequence+","+(index+i));
						break test;
					}
				}
			}
		}
		catch (IOException e)
		{
			e.printStackTrace();
		}
		complete = true;
		trigger();
	}

	public void run()
	{
		extendFile();
	}
}

---------- END SOURCE ----------

CUSTOMER SUBMITTED WORKAROUND :
Synchronize around read/write and setLength on a common object.
J2SE Version (please include all output from java -version flag):
jre-7-ea-bin-b142-windows-i586-12_may_2011.exe

Does this problem occur on J2SE 1.5.x or 6ux?  Yes / No (pick one)
Yes, it occurs for java 5, and 6 too

Operating System Configuration Information (be specific):
Microsoft Windows Operating System Vista

Bug Description:
Find a very bad thread bug in RandomAccessFile. We share the RandomAccessFile
instance 
over multiple threads and have synchronized the IO operations. 
But it worked like 
without synchronized.

The only workaround seems that after every seek(long), have to verify it 
and set repeat the seek(long).

Steps to reproduce:

It can be reproduced with the attached test file TestThreadFileIO.java. 
If all is ok then there is no output on the console.
You can see it if you use only one thread. 
There is also a test result that we receive.
CAP members change the sample code to use a FileChannel. 
but it still sahow th esame thread problem. 
Attached the modified test case TestThreadFileIOnew.java

For the previous test case, they did not mixed RandomAccessFile and FileChannel,
only use RandomAccessFile. The new sample is more a mix because need the
 RandomAccessFile to create the FileChannel.

Not see what is wrong in both samples, not access the file at the sam time. 
The access is synchronized. It is a sequencial call of more as one thread 
to the same file.
The updated test case uses raf.length() which isn't thread safe. If the test is changed to use channel.size() instead of raf.length() then it work as expected. For a future release we will nede to re-examine RandomAccessFile's implementation. Another thing to point out is that FileChannel defines read/write methods that take a position parameter. These methods do not change the global position and are designed for cases where multiple threads need to operation on the same file concurrently. There is also AsynchronousFileChannel in jdk7.

Comments
SUGGESTED FIX It turns out that passing the offset in the OVERLAPPED structure does allow multiple concurrent threads to do positional operations. However a side effect is that file position (as observed by RandomAccessFile's getFilePointer or the channel's position method) is also changed. So it appears that complete pread/pwrite semantics are not possible. Even worse is that asynchronous IO operations are also adjusting the file position - that is sure to cause trouble in the future.
15-01-2006

SUGGESTED FIX The FileDispatcher pread and pwrite implementation should be changed to pass an OVERLAPPED structure to ReadFile and WriteFile. Here's the relevant section of the spec for ReadFile: "If hFile is not opened with FILE_FLAG_OVERLAPPED and lpOverlapped is not NULL, the read operation starts at the offset specified in the OVERLAPPED structure. ReadFile does not return until the read operation has been completed. " The description is similar for WriteFile. If this fix can be verified then we should be able to eliminate the locking in the FileDispatcher implementation and improve concurrency.
12-01-2006

EVALUATION The issue is specific to Windows where the positional read/write implementation is temporarily adjusting the file pointer. This is synchronized in the FileChannel implementation and doesn't impact multiple threads doing I/O on the file channel. It isn't however synchronized with RandomAccessFile's seek, setLength, or getFilePointer implementations.
12-01-2006