JDK-4938372 : (fs) Overlapping mapped files cannot be unmapped (windows)
  • Type: Bug
  • Status: Closed
  • Resolution: Fixed
  • Component: core-libs
  • Sub-Component: java.nio
  • Priority: P3
  • Affected Version: 1.4.2,6
  • OS: windows_xp
  • CPU: x86
  • Submit Date: 2003-10-15
  • Updated Date: 2007-03-10
  • Resolved Date: 2011-05-18
The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availabitlity Release.

To download the current JDK release, click here.
JDK 7
7 b10Fixed
Related Reports
Duplicate :  
Relates :  
Description

Name: rmT116609			Date: 10/15/2003


FULL PRODUCT VERSION :
java version "1.4.2_01"
Java(TM) 2 Runtime Environment, Standard Edition (build 1.4.2_01-b06)
Java HotSpot(TM) Client VM (build 1.4.2_01-b06, mixed mode)

FULL OS VERSION :
Microsoft Windows XP [Version 5.1.2600]

EXTRA RELEVANT SYSTEM CONFIGURATION :
Pentium 4/3.06GHz HYPERTHREADED

A DESCRIPTION OF THE PROBLEM :
java.lang.Error: Cleaner terminated abnormally
is reported during the execution of the attached class. This error is from the Sun runtime supporting memory mapping of files. Once the cleaner has terminated, memory mapping objects accumulate and the process can eventually exhaust available address space.

STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
Run the attached class. Note that because the cleaner process is running on a separate thread it is likely to be susceptible to timing differences.

EXPECTED VERSUS ACTUAL BEHAVIOR :
EXPECTED -
The code should run without reporting any errors.

Test file: C:\DOCUME~1\MTHORN~1\LOCALS~1\Temp\exp44667tmp
2 buffers, mapping 23592960 bytes
2 buffers, mapping 26542080 bytes
...
15 buffers, mapping 248789966 bytes
17 buffers, mapping 279888711 bytes
19 buffers, mapping 314874799 bytes
test completed

ACTUAL -
As for a good run up to this point

15 buffers, mapping 248789966 bytes
java.lang.Error: Cleaner terminated abnormally
	at sun.misc.Cleaner.clean(Cleaner.java:125)
	at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:123)
Caused by: java.io.IOException: The process cannot access the file because another process has locked a portion of the file
	at sun.nio.ch.FileChannelImpl.unmap0(Native Method)
	at sun.nio.ch.FileChannelImpl.access$000(FileChannelImpl.java:26)
	at sun.nio.ch.FileChannelImpl$Unmapper.run(FileChannelImpl.java:644)
	at sun.misc.Cleaner.clean(Cleaner.java:123)
	... 1 more
17 buffers, mapping 279888711 bytes
19 buffers, mapping 314874799 bytes
test completed


ERROR MESSAGES/STACK TRACES THAT OCCUR :
java.lang.Error: Cleaner terminated abnormally
	at sun.misc.Cleaner.clean(Cleaner.java:125)
	at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:123)
Caused by: java.io.IOException: The process cannot access the file because another process has locked a portion of the file
	at sun.nio.ch.FileChannelImpl.unmap0(Native Method)
	at sun.nio.ch.FileChannelImpl.access$000(FileChannelImpl.java:26)
	at sun.nio.ch.FileChannelImpl$Unmapper.run(FileChannelImpl.java:644)
	at sun.misc.Cleaner.clean(Cleaner.java:123)
	... 1 more


REPRODUCIBILITY :
This bug can be reproduced often.

---------- BEGIN SOURCE ----------
import java.io.File;
import java.io.RandomAccessFile;
import java.nio.ByteBuffer;
import java.nio.channels.FileChannel;
import java.nio.channels.FileLock;
import java.util.ArrayList;

public class ExpandingFileMap
{
	FileChannel channel;
	ByteBuffer[] buffers;
	int initialSize = 20480*1024;
	int maximumMapSize = 16*1024*1024;
	int maximumFileSize = 300000000;
	
	public static void main(String[] args)
	{
		new ExpandingFileMap().test();
	}
	
	public void test()
	{
		try
		{
			File file = File.createTempFile("exp", "tmp");
			System.out.println("Test file: "+file);
			file.deleteOnExit();
			RandomAccessFile f = new RandomAccessFile(file, "rw");
			f.setLength(initialSize);
			channel = f.getChannel();
			// FileLock lock = channel.lock(0, Long.MAX_VALUE, false);
			buffers = new ByteBuffer[128];
			buffers[0] = channel.map(FileChannel.MapMode.READ_WRITE, 0, initialSize);
			int currentBuffer = 0;
			int currentSize = initialSize;
			int currentPosition = 0;
			
			ArrayList junk = new ArrayList();
			
			while (currentPosition+currentSize < maximumFileSize)
			{
				int inc = Math.max(1000*1024, (currentPosition+currentSize)/8);
				int size = currentPosition+currentSize+inc;
				f.setLength(size);
				while (currentSize+inc > maximumMapSize)
				{
					if (currentSize < maximumMapSize)
					{
						buffers[currentBuffer] = channel.map(FileChannel.MapMode.READ_WRITE, currentPosition, maximumMapSize);
						fillBuffer(buffers[currentBuffer], currentSize);
					}
					currentPosition += maximumMapSize;
					inc = currentSize+inc-maximumMapSize;
					currentSize = 0;
					currentBuffer++;
					if (currentBuffer == buffers.length)
					{
						ByteBuffer[] old = buffers;
						buffers = new ByteBuffer[currentBuffer+currentBuffer/2];
						System.arraycopy(old, 0, buffers, 0, currentBuffer);
					}
				}
				currentSize += inc;
				if (currentSize > 0)
				{
					buffers[currentBuffer] = channel.map(FileChannel.MapMode.READ_WRITE, currentPosition, currentSize);
					fillBuffer(buffers[currentBuffer], currentSize-inc);
				}
				long t = System.currentTimeMillis();
				while (System.currentTimeMillis() < t+500)
				{
					// waste time
					junk.add(String.valueOf(t));
					if (junk.size() > 100000)
						junk.clear();
				}

				System.out.println((currentBuffer+1)+" buffers, mapping "+size+" bytes");
			}
		}
		catch (Exception ex)
		{
			ex.printStackTrace();
		}
		for (int i=0; i<20; i++)
		{
			System.gc();
			try
			{
				Thread.sleep(20);
			}
			catch (InterruptedException ex)
			{
			}
		}
		System.out.println("test completed");
	}
	
	private void fillBuffer(ByteBuffer buf, int from)
	{
		int limit = buf.limit();
		for (int i=from; i<limit; i++)
		{
			buf.put(i, (byte)i);
		}
	}
}

---------- END SOURCE ----------
(Incident Review ID: 214797) 
======================================================================

Comments
EVALUATION It turns out the issue is that the unmap attempts to flush dirty pages prior to unmapping the region. If the memory system is already in the process of flushing then the FlushViewOfFile call fails with a locking violation. For the same reason, invoking the buffer's force method may throw an IOException for the same reason. We have a preliminary fix for the issue and we will try to fix early in jdk7.
2007-02-06

EVALUATION The are two problems here. One is that the failure of a cleaner should cause the VM to exit; I've submitted 4954921 to cover that. The other is that Windows apparently has difficulties dealing with overlapping mapped memory regions. This requires further investigation, though a workaround does exist. -- ###@###.### 2003/11/15
181-11-11 0

WORK AROUND Instead of creating many file mappings, create just one mapping and use buffer slices to access the various regions of interest. -- ###@###.### 2003/11/15
181-11-11 0