JDK-4925707 : cannot open more than 2035 files on Win98
  • Type: Bug
  • Component: core-libs
  • Sub-Component: java.io
  • Affected Version: 5.0
  • Priority: P2
  • Status: Closed
  • Resolution: Fixed
  • OS: solaris_8,windows_98
  • CPU: generic,x86
  • Submitted: 2003-09-20
  • Updated: 2004-03-27
  • Resolved: 2004-02-06
The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.
Other
5.0 b38Fixed
Related Reports
Relates :  
Relates :  
Relates :  
Description

Name: ngR10089			Date: 09/19/2003


 
The JCK tests run fails on Windows-98 when compiler is started in the
sameJVM mode by Javatest Agent. The failure is reproduced under jdk 1.5.0
b19. jdk 1.5.0-b15 executes the tests successully.

To reproduce failure:
(1) put the attached rw-jck.sh and jck.jti in a directory on win98 machine,
    (log file, jtb file and 'wd' directory will be created here during the
    test execution);
(2) fix JH and JCKDIR references in rw-jck.sh according to your network
    drive mapping. 
(3) run rw-jck.sh in MKS KornShell window
     ./rw-jck.sh > log 2>&1
After successfully execution more than 600 tests the compiler reports
the errors:

error: cannot read: 
tests\lang\CONV\conv053\conv05303m100111\conv05303m100111.java
1 error
error: cannot read: 
tests\lang\CONV\conv053\conv05303m100122\conv05303m100122.java
1 error

The compiler messages may be found in log file. The results of test 
executions
may be found in 'wd' directory.

======================================================================

###@###.### 2003-11-21

Here is a variant of Pavel's program that reproduces the failure without JCK tests at all.
Put both files in the same directory, change the path to JDK in the script and run it under MKS ksh in win98. The script creates 2000 copies of the java file
changing its name and then compiles all of them in the same JVM.
Compilations begin to fail after 1000 or so steps (close to 1024 btw ;)

--------- r --------------
JH=r:/jdk/1.5/promoted/all/b28/binaries/windows-i586

k=2000
n=$k
while [ $n -gt 0 ]
do
   echo $n
   sed -e "s/public class CompileSameVM/public class CompileSameVM$n/"       CompileSameVM.java  > CompileSameVM${n}.java
   n=$(($n-1))
done

${JH}/bin/javac -classpath ${JH}/lib/tools.jar -d . CompileSameVM.java
${JH}/bin/java -version
${JH}/bin/java -cp "${JH}/lib/tools.jar;." CompileSameVM $k 2>&1|tee log
---------end of r ----------------
------------------CompileSameVM.java-----------------

public class CompileSameVM {

    public static void main(String args[]) {
        int N = 0;
        System.out.println("args#: " + args.length);
        if ( args.length > 0 ) {
          try {
            N = Integer.valueOf(args[0]).intValue();
          } catch (NumberFormatException e) {
            System.out.println("invalid argument: " + args[1]);
          }
        }
        System.out.println("# of compilations: " + N);
        for (int i = 1; i <= N ; i++) {
            System.out.println(i + " : " );
            try {
                String[] files = new String[1];
                files[0] = "CompileSameVM" + i + ".java";
                String[] pre_args = {"-d", "."};
                String[] cargs = new String[pre_args.length + files.length];
                System.arraycopy(pre_args, 0, cargs, 0, pre_args.length);
                System.arraycopy(files, 0, cargs, pre_args.length, files.length);
                
                int status = com.sun.tools.javac.Main.compile(cargs);

            } catch (Exception e) {
                System.err.println(e.getMessage());
	        e.printStackTrace();
                return;
            }
        }
    }
}
--------------------------------------------------------

---------------output------------------------
args#: 1
# of compilations: 2000
1 : 
2 : 
....
1011 : 
1012 : 
1013 : 
CompileSameVM1013.java:2: error while writing CompileSameVM1013: .\CompileSameVM1013.class (Bad file descriptor)
(source unavailable)
1 error
1014 : 
CompileSameVM1014.java:25: package com.sun.tools.javac does not exist
(source unavailable)
1 error
1015 : 
CompileSameVM1015.java:25: package com.sun.tools.javac does not exist
(source unavailable)
1 error
1016 : 
CompileSameVM1016.java:25: package com.sun.tools.javac does not exist
(source unavailable)
1 error
1017 : 
CompileSameVM1017.java:25: package com.sun.tools.javac does not exist
(source unavailable)
1 error
1018 : 
CompileSameVM1018.java:25: package com.sun.tools.javac does not exist
(source unavailable)
1 error
1019 : 
CompileSameVM1019.java:25: package com.sun.tools.javac does not exist
(source unavailable)
1 error
1020 : 
CompileSameVM1020.java:25: package com.sun.tools.javac does not exist
(source unavailable)
1 error
1021 : 
Fatal Error: Unable to locate package java.lang in classpath or bootclasspath
1022 : 
error: cannot read: CompileSameVM1022.java
1 error
1023 : 
error: cannot read: CompileSameVM1023.java
1 error
1024 : 
...
---------------------------------------------

=====================================================================

Comments
CONVERTED DATA BugTraq+ Release Management Values COMMIT TO FIX: tiger-beta2 FIXED IN: tiger-beta2 INTEGRATED IN: tiger-b38 tiger-beta2 VERIFIED IN: tiger-beta2
14-06-2004

SUGGESTED FIX --- /u/martin/ws/win98/webrev/src/windows/native/java/io/io_util_md.c- 2004-01-27 18:36:59.777307000 -0800 +++ io_util_md.c 2004-01-27 18:36:49.117548000 -0800 @@ -93,22 +93,16 @@ } void fileOpen(JNIEnv *env, jobject this, jstring path, jfieldID fid, int flags) { - if (onNT) { DWORD access = 0; DWORD sharing = FILE_SHARE_READ | FILE_SHARE_WRITE; DWORD disposition = OPEN_EXISTING; DWORD flagsAndAttributes = FILE_ATTRIBUTE_NORMAL; HANDLE h = NULL; int pathlen = 0; - WCHAR *pathbuf = pathToNTPath(env, path, JNI_TRUE); - if (pathbuf == NULL) { - /* Exception already pending */ - return; - } /* Note: O_TRUNC overrides O_CREAT */ if (flags & O_TRUNC) { disposition = CREATE_ALWAYS; } else if (flags & O_CREAT) { @@ -131,20 +125,33 @@ } if (flags == 0) { access = GENERIC_READ; } + if (onNT) { + WCHAR *pathbuf = pathToNTPath(env, path, JNI_TRUE); + if (pathbuf == NULL) { + /* Exception already pending */ + return; + } + h = CreateFileW( pathbuf, /* Wide char path name */ access, /* Combine read and/or write permission */ sharing, /* File sharing flags */ NULL, /* Security attributes */ disposition, /* creation disposition */ flagsAndAttributes, /* flags and attributes */ NULL); free(pathbuf); + } else { + WITH_PLATFORM_STRING(env, path, _ps) { + h = CreateFile(_ps, access, sharing, NULL, disposition, + flagsAndAttributes, NULL); + } END_PLATFORM_STRING(env, _ps); + } if (h == INVALID_HANDLE_VALUE) { int error = GetLastError(); if (error == ERROR_TOO_MANY_OPEN_FILES) { JNU_ThrowByName(env, JNU_JAVAIOPKG "IOException", @@ -153,23 +160,10 @@ } throwFileNotFoundException(env, path); return; } SET_FD(this, (jlong)h, fid); - } else { - jlong fd = -1; - WITH_PLATFORM_STRING(env, path, _ps) { - fd = JVM_Open(_ps, flags, 0666); - } END_PLATFORM_STRING(env, _ps); - fd = _get_osfhandle((jint)fd); - if (fd >= 0) { - SET_FD(this, fd, fid); - } - if (fd < 0) { - throwFileNotFoundException(env, path); - } - } } /* These are functions that use a handle fd instead of the old C style int fd as is used in HPI layer */
11-06-2004

EVALUATION This is not a supported mode for running the compiler. javac is a command-line tool. Almost nothing has changed in the compiler between these two builds. You are most likely seeing a VM or nio change. Is sameJVM mode a supported operational mode for our customers using the language tests? ###@###.### 2003-09-29 This is more likely a VM or NIO problem. The same problem can be shown using version 1.4 of JCK_Compiler, which hasn't changed in a long time. The problem can also be reproduced on solaris. The compiler is failing to read the file because the VM process has run out of file handles. ###@###.### 2003-10-21 ugg!! ###@###.### 2003-10-21 Here is a much simpler test case that isolates the problem. ----------------------------------------------------------- import java.io.*; public class HandleLeak { interface Thunk { void doit() throws Exception; } public static void doN(int iterations, Thunk thunk) { int i = 0; try { while (++i < iterations) thunk.doit(); } catch (Exception e) { System.out.println("iterations=" + i); e.printStackTrace(); } } public static void main(String[] args) throws Exception { final File f = new File("HandleLeak.tmp"); if (args.length > 0) doN(10000, new Thunk() { public void doit() throws Exception { new FileOutputStream(f).close();}}); else doN(10000, new Thunk() { public void doit() throws Exception { new FileInputStream(f).close();}}); } } ----------------------------------------------------------- Now java HandleLeak out; java HandleLeak generates this output: iterations=2035 java.io.FileNotFoundException: HandleLeak.tmp (Bad file descriptor) at java.io.FileOutputStream.open(Native Method) at java.io.FileOutputStream.<init>(FileOutputStream.java:179) at java.io.FileOutputStream.<init>(FileOutputStream.java:131) at HandleLeak$1.doit(HandleLeak.java:20) at HandleLeak.doN(HandleLeak.java:9) at HandleLeak.main(HandleLeak.java:19) iterations=2035 java.io.FileNotFoundException: HandleLeak.tmp (Bad file descriptor) at java.io.FileInputStream.open(Native Method) at java.io.FileInputStream.<init>(FileInputStream.java:106) at HandleLeak$2.doit(HandleLeak.java:23) at HandleLeak.doN(HandleLeak.java:9) at HandleLeak.main(HandleLeak.java:22) On Windows98, only 2035 files streams can be opened, for input or output. This seems very related to the bug 4189011 java.io: Cannot open more than 2035 files It appears likely that the bug is in src/windows/native/java/io/io_util_md.c There needs to be a pairing of open and closes that is not being obeyed in the Win98 world - JVM_Open is being called, but there is no corresponding JVM_Close. It looks like calling CreateFile instead of JVM_Open on Win98 will work correctly. The Microsoft Knowledge Base Article 99173 http://support.microsoft.com/default.aspx?scid=http://support.microsoft.com:80/support/kb/articles/Q99/1/73.ASP&NoWebContent=1 says... There are multiple types of file handles that can be opened using the Win32 API and the C Run-time: Returned Type File Creation API API Set --------------------------------------------- HANDLE CreateFile() Win32 HFILE OpenFile()/_lcreat() Win32 int _creat()/_open() C Run-time FILE * fopen() C Run-time In general, these file I/O "families" are incompatible with each other. On some implementations of the Win32 application programming interfaces (APIs), the OpenFile()/_lcreat() family of file I/O APIs are implemented as "wrappers" around the CreateFile() family of file I/O APIs, meaning that OpenFile(), _lcreat(), and _lopen() end up calling CreateFile(), returning the handle returned by CreateFile(), and do not maintain any state information about the file themselves. However, this is an implementation detail only and is NOT a design feature. NOTE: You cannot count on this being true on other implementations of the Win32 APIs. Win32 file I/O APIs may be written using different methods on other platforms, so reliance on this implementation detail may cause your application to fail. The rule to follow is to use one family of file I/O APIs and stick with them--do not open a file with _lopen() and read from it with ReadFile(), for example. This kind of incorrect use of the file I/O APIs can easily be caught by the compiler, because the file types (HFILE and HANDLE respectively) are incompatible with each other and the compiler will warn you (at warning level /w3 or higher) when you have incorrectly passed one type of file handle to a file I/O API that is expecting another, such as passing an HFILE type to ReadFile(HANDLE, ...) in the above example. ###@###.### 2003-12-12 The above approach was implemented and seems to work well. Six previously failing tests have started passing on Windows 98: java/io/FileOutputStream/ManyFiles.jtr java/io/pathNames/GeneralWin32.jtr java/nio/channels/FileChannel/MapTest.jtr java/nio/channels/FileChannel/Transfers.jtr java/nio/channels/Selector/Connect.jtr sun/nio/ch/TempBuffer.jtr ###@###.### 2004-01-29
29-01-2004