JDK-4189011 : java.io: Cannot open more than 2035 files (win32)
  • Type: Bug
  • Component: core-libs
  • Sub-Component: java.io
  • Affected Version:
    1.1.5,1.3.1_02,1.4.0,1.4.2_09,1.4.2_18,5.0 1.1.5,1.3.1_02,1.4.0,1.4.2_09,1.4.2_18,5.0
  • Priority: P4
  • Status: Resolved
  • Resolution: Fixed
  • OS:
    generic,solaris_8,windows_nt,windows_2000,windows_2003 generic,solaris_8,windows_nt,windows_2000,windows_2003
  • CPU: generic,x86,sparc
  • Submitted: 1998-11-11
  • Updated: 2011-02-11
  • Resolved: 2003-08-15
The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.
Other Other
1.4.2_31-revFixed 1.4.2_33Fixed
Related Reports
Duplicate :  
Duplicate :  
Relates :  
Relates :  
Relates :  
Description

Name: diC59631			Date: 11/11/98


1. Attached is the source code of a class test.java
2. I have tried with JDK 1.1.5 and 1.2 beta 2

To test ->
  1. Create dir say 'maxFiles'
  2. Copy test.class in this dir
  3. create 'test' directory in 'maxFiles'
  4. run java test 3000 1 10 testWith the above command line the 'test' will create 10 thread and each thread will attempt to open 3000 files
for single thread use:
java test 3000 1 1 test
 I have even tried with ( -ms33554432 -mx67108867 ) flags for runtime, it does not make any difference.

 In short, it does not matter how many thread are opened or what is Java heap size, I am not able 
to open in total more than 2035  files.

// -------------------------------- Sorce Code of test.java -------------
import java.io.*;
import java.util.*;



/**
Class to test max. number of open files.

Author: Bharat Gogia

E-mail: ###@###.###

Dt:23 Feb. 1998

**/



class test extends Thread
{

   static Vector m_BrnIdxArray = null;
	
	int startVal;
	int maxVal;
	String path;
	int thID;
	
	
	static void zleep(long ms)
		{
		try
			{
			Thread.currentThread().yield();
			Thread.currentThread().sleep(ms);
			} 
		catch (InterruptedException ie)
			{
			}
		}
	
	public test(int stVal, int mxVal, String sPath, int tID)
		{
		maxVal = mxVal;
		startVal = stVal;
		path = sPath;
		}

	static public void main(String [] args)
		{
		int maxVal = Integer.parseInt(args[0]);
		int startVal = Integer.parseInt(args[1]);
		int totThread = Integer.parseInt(args[2]);
		String path = new String (args[3]);
		
		for (int i =0; i< totThread ; i++ )
			{
			test t1 = new test( (startVal + (i*maxVal)), maxVal, path, i);
			t1.start();
			zleep(10);
			}
		
		}

	public void run()
		{
		byte buf[] = new byte[512];
		m_BrnIdxArray = new Vector();
  		RandomAccessFile m_brnDatFile = null;
	 	System.out.println("Starting " + System.currentTimeMillis() + "\r\n" );
		for (int i=startVal; i < (startVal+maxVal) ; i++ )
			{
			String fname ; 
			fname = (path + FormatIntToString(i, "0000000000",'d') + ".tst");
		    try
		      	{
		      	m_brnDatFile = new RandomAccessFile(fname, "rw");
			    m_brnDatFile.write(buf);
				m_BrnIdxArray.addElement(m_brnDatFile);
				zleep(20);
		      	}
		    catch (Exception eio)
		      	{
		      	System.out.println("\r\n" + eio.getMessage() + " " + eio);
	 			System.out.println("Aborting " + System.currentTimeMillis() + "\r\n" );
				m_brnDatFile  = null;
				m_BrnIdxArray.removeAllElements();
				System.exit(0);
		      	}
			System.out.print("Opened " + fname + " Thread: " + thID + "\r" );
			fname = null;
			}
	 	System.out.println("Closing " + System.currentTimeMillis() + "\r\n" );
		}


    public static String FormatIntToString(int i, String frmt, char type)
      	{
        String target;
        StringBuffer strbuf = new StringBuffer(frmt);
		 String strInt;

		 
        try
          	{	
           if(type == 'x' || type == 'X')
				strInt = Integer.toHexString(i);
			else 
			   	strInt = Integer.toString(i);
				
			if (strInt.length() > frmt.length())
			   	{
			   	strInt = strInt.substring(0, frmt.length());
				}
			   
	        strbuf.insert( (frmt.length() - strInt.length()), strInt);
			   
           strbuf.setLength(frmt.length());
           target =  new String(strbuf);  
			 
          	} 
        catch (StringIndexOutOfBoundsException sie)
            {
			 System.out.println("Exception in FormatIntToString " + sie.getMessage());
            target = null;
	         }
            
      	return(target);
      	}



}
//----------------------------------- End of Soruce Code ----------

 To clarify the issue, on win32 system there are three 
ways to open a file:
1: Using Win32 API
2: Using MFC class framework lib.
3: using C-Library API (open() and fopen())

Other than the third option, i.e. option 1 and 2 have practically no 
limitation in opening number of files. The third method is restricted 
(for the reason not known to me) to open only approx. 2035 files. That 
is why MS JVM is able to open unlimited (practically) files, but SUN JVM 
fails after 2035 files (my guess is it is using 3rd method to open 
file).

 The case is true only only on Win32 OS. The reason this is serious 
problem ans we have to revert to MS JVM is because we werr writing high 
scalable server in Java. I am sure you understand that Java as an API 
must compatible on all platform, some of which are Solaris, NT(Win32), 
Novell etc...

Hope this helps you to update your JVM.
(Review ID: 25523)
======================================================================

Name: nt126004			Date: 06/05/2002


FULL PRODUCT VERSION :
C:\>java -version
java version "1.4.0"
Java(TM) 2 Runtime Environment, Standard Edition (build 1.4.0-b92)
Java HotSpot(TM) Client VM (build 1.4.0-b92, mixed mode)


FULL OPERATING SYSTEM VERSION :
Windows 2000 SERVER

ADDITIONAL OPERATING SYSTEMS :

Windows 2000 Professional


A DESCRIPTION OF THE PROBLEM :
File creation is seriously broken in the JDK 1.4 for
Windows.

Unlike most unix systems, Windows 2000 has *NO*
file descriptor or open file limitations. A simple
5 line C program can create as many new files
as desired (I created about 50,000 as a test).

I repeat this: there are *NO* operating system
limitations to creating files [on Windows 2000].

However, with JDK 1.4, there is a "too many open files"
exception after creating about 2033 files. This is
true, regardless of whether I am running on Windows
2000 Server or Professional. (By the way, I am doing
everything as "Administrator").

This is clearly a serious JVM bug that needs to be
corrected ASAP. Serious server side applications cannot
run if artificially constrained by this JVM limitation.


STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
1. see attached program.
2. java maxfiles 10000
3. see attached stack trace

EXPECTED VERSUS ACTUAL BEHAVIOR :
Expected: Successful completion

Actual: "too many open files" exception.

ERROR MESSAGES/STACK TRACES THAT OCCUR :
C:\javatesting\javatesting\misc\filetest>java maxfiles 10000
Exception in thread "main" java.io.FileNotFoundException: file2033 (Too many
open files)
        at java.io.FileOutputStream.open(Native Method)
        at java.io.FileOutputStream.<init>(FileOutputStream.java:176)
        at java.io.FileOutputStream.<init>(FileOutputStream.java:131)
        at maxfiles.main(maxfiles.java:14)

This bug can be reproduced always.

---------- BEGIN SOURCE ----------
import java.io.*;
import java.util.*;

/**
Is there a JVM limitation to create files on Windows 2000 Server ?

Usage:
java maxfiles <number-of-files>

Note:
Creates files in the same directory where "java" was invoked.
Create a test directory for this purpose !
**/
public class maxfiles
{
static int count = 0;
static List files = new ArrayList();

public static void main(String args[]) throws Exception
	{
    for (int n = 0; n < Integer.parseInt(args[0]); n++) {
		File f = new File("file" + count++);
		//save ref, so not gc'ed
		files.add(new PrintStream(new FileOutputStream(f)));
		}
    Iterator it = files.iterator();
    while (it.hasNext()) {
		PrintStream out = ( PrintStream) it.next();
		out.println("foo");
		out.flush();
		}
	System.out.println("current files open: " + files.size());
	} //~main

} //~class maxfiles
---------- END SOURCE ----------
(Review ID: 148202)
======================================================================

Comments
CONVERTED DATA BugTraq+ Release Management Values COMMIT TO FIX: tiger FIXED IN: tiger INTEGRATED IN: tiger tiger-b16
14-06-2004

WORK AROUND Name: diC59631 Date: 11/11/98 NO WORKAROUND! ======================================================================
11-06-2004

EVALUATION Yes, this could be a real problem for big server apps. We've already migrated some of the win32 I/O code to use the raw win32 API rather than the MS C runtime library; we should finish the task. -- mr@eng 1998/11/11 Maybe in Mantis. ###@###.### 2002-10-18 Unfortunately even though windows can deal with more win32 handles like you get using CreateFile, we need the C library handles for FileDescriptor and those are limited to 2048. ###@###.### 2002-10-25 java.io has been converted to use handles on windows. ###@###.### 2003-08-09
25-10-2002