United StatesChange Country, Oracle Worldwide Web Sites Communities I am a... I want to...
JDK-4938372 : (fs) Overlapping mapped files cannot be unmapped (windows)

Details
Type:
Bug
Submit Date:
2003-10-15
Status:
Closed
Updated Date:
2007-03-10
Project Name:
JDK
Resolved Date:
2011-05-18
Component:
core-libs
OS:
windows_xp
Sub-Component:
java.nio
CPU:
x86
Priority:
P3
Resolution:
Fixed
Affected Versions:
1.4.2,6
Fixed Versions:

Related Reports
Duplicate:
Relates:

Sub Tasks

Description

Name: rmT116609			Date: 10/15/2003


FULL PRODUCT VERSION :
java version "1.4.2_01"
Java(TM) 2 Runtime Environment, Standard Edition (build 1.4.2_01-b06)
Java HotSpot(TM) Client VM (build 1.4.2_01-b06, mixed mode)

FULL OS VERSION :
Microsoft Windows XP [Version 5.1.2600]

EXTRA RELEVANT SYSTEM CONFIGURATION :
Pentium 4/3.06GHz HYPERTHREADED

A DESCRIPTION OF THE PROBLEM :
java.lang.Error: Cleaner terminated abnormally
is reported during the execution of the attached class. This error is from the Sun runtime supporting memory mapping of files. Once the cleaner has terminated, memory mapping objects accumulate and the process can eventually exhaust available address space.

STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
Run the attached class. Note that because the cleaner process is running on a separate thread it is likely to be susceptible to timing differences.

EXPECTED VERSUS ACTUAL BEHAVIOR :
EXPECTED -
The code should run without reporting any errors.

Test file: C:\DOCUME~1\MTHORN~1\LOCALS~1\Temp\exp44667tmp
2 buffers, mapping 23592960 bytes
2 buffers, mapping 26542080 bytes
...
15 buffers, mapping 248789966 bytes
17 buffers, mapping 279888711 bytes
19 buffers, mapping 314874799 bytes
test completed

ACTUAL -
As for a good run up to this point

15 buffers, mapping 248789966 bytes
java.lang.Error: Cleaner terminated abnormally
	at sun.misc.Cleaner.clean(Cleaner.java:125)
	at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:123)
Caused by: java.io.IOException: The process cannot access the file because another process has locked a portion of the file
	at sun.nio.ch.FileChannelImpl.unmap0(Native Method)
	at sun.nio.ch.FileChannelImpl.access$000(FileChannelImpl.java:26)
	at sun.nio.ch.FileChannelImpl$Unmapper.run(FileChannelImpl.java:644)
	at sun.misc.Cleaner.clean(Cleaner.java:123)
	... 1 more
17 buffers, mapping 279888711 bytes
19 buffers, mapping 314874799 bytes
test completed


ERROR MESSAGES/STACK TRACES THAT OCCUR :
java.lang.Error: Cleaner terminated abnormally
	at sun.misc.Cleaner.clean(Cleaner.java:125)
	at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:123)
Caused by: java.io.IOException: The process cannot access the file because another process has locked a portion of the file
	at sun.nio.ch.FileChannelImpl.unmap0(Native Method)
	at sun.nio.ch.FileChannelImpl.access$000(FileChannelImpl.java:26)
	at sun.nio.ch.FileChannelImpl$Unmapper.run(FileChannelImpl.java:644)
	at sun.misc.Cleaner.clean(Cleaner.java:123)
	... 1 more


REPRODUCIBILITY :
This bug can be reproduced often.

---------- BEGIN SOURCE ----------
import java.io.File;
import java.io.RandomAccessFile;
import java.nio.ByteBuffer;
import java.nio.channels.FileChannel;
import java.nio.channels.FileLock;
import java.util.ArrayList;

public class ExpandingFileMap
{
	FileChannel channel;
	ByteBuffer[] buffers;
	int initialSize = 20480*1024;
	int maximumMapSize = 16*1024*1024;
	int maximumFileSize = 300000000;
	
	public static void main(String[] args)
	{
		new ExpandingFileMap().test();
	}
	
	public void test()
	{
		try
		{
			File file = File.createTempFile("exp", "tmp");
			System.out.println("Test file: "+file);
			file.deleteOnExit();
			RandomAccessFile f = new RandomAccessFile(file, "rw");
			f.setLength(initialSize);
			channel = f.getChannel();
			// FileLock lock = channel.lock(0, Long.MAX_VALUE, false);
			buffers = new ByteBuffer[128];
			buffers[0] = channel.map(FileChannel.MapMode.READ_WRITE, 0, initialSize);
			int currentBuffer = 0;
			int currentSize = initialSize;
			int currentPosition = 0;
			
			ArrayList junk = new ArrayList();
			
			while (currentPosition+currentSize < maximumFileSize)
			{
				int inc = Math.max(1000*1024, (currentPosition+currentSize)/8);
				int size = currentPosition+currentSize+inc;
				f.setLength(size);
				while (currentSize+inc > maximumMapSize)
				{
					if (currentSize < maximumMapSize)
					{
						buffers[currentBuffer] = channel.map(FileChannel.MapMode.READ_WRITE, currentPosition, maximumMapSize);
						fillBuffer(buffers[currentBuffer], currentSize);
					}
					currentPosition += maximumMapSize;
					inc = currentSize+inc-maximumMapSize;
					currentSize = 0;
					currentBuffer++;
					if (currentBuffer == buffers.length)
					{
						ByteBuffer[] old = buffers;
						buffers = new ByteBuffer[currentBuffer+currentBuffer/2];
						System.arraycopy(old, 0, buffers, 0, currentBuffer);
					}
				}
				currentSize += inc;
				if (currentSize > 0)
				{
					buffers[currentBuffer] = channel.map(FileChannel.MapMode.READ_WRITE, currentPosition, currentSize);
					fillBuffer(buffers[currentBuffer], currentSize-inc);
				}
				long t = System.currentTimeMillis();
				while (System.currentTimeMillis() < t+500)
				{
					// waste time
					junk.add(String.valueOf(t));
					if (junk.size() > 100000)
						junk.clear();
				}

				System.out.println((currentBuffer+1)+" buffers, mapping "+size+" bytes");
			}
		}
		catch (Exception ex)
		{
			ex.printStackTrace();
		}
		for (int i=0; i<20; i++)
		{
			System.gc();
			try
			{
				Thread.sleep(20);
			}
			catch (InterruptedException ex)
			{
			}
		}
		System.out.println("test completed");
	}
	
	private void fillBuffer(ByteBuffer buf, int from)
	{
		int limit = buf.limit();
		for (int i=from; i<limit; i++)
		{
			buf.put(i, (byte)i);
		}
	}
}

---------- END SOURCE ----------
(Incident Review ID: 214797) 
======================================================================

                                    

Comments
EVALUATION

It turns out the issue is that the unmap attempts to flush dirty pages prior to unmapping the region. If the memory system is already in the process of flushing then the FlushViewOfFile call fails with a locking violation. For the same reason, invoking the buffer's force method may throw an IOException for the same reason. We have a preliminary fix for the issue and we will try to fix early in jdk7.
                                     
2007-02-06
WORK AROUND

Instead of creating many file mappings, create just one mapping and use buffer
slices to access the various regions of interest.

-- ###@###.### 2003/11/15
                                     
181-11-11 0
EVALUATION

The are two problems here.  One is that the failure of a cleaner should cause
the VM to exit; I've submitted 4954921 to cover that.  The other is that
Windows apparently has difficulties dealing with overlapping mapped memory
regions.  This requires further investigation, though a workaround does exist.

-- ###@###.### 2003/11/15
                                     
181-11-11 0



Hardware and Software, Engineered to Work Together