JDK-6339493 : (process) Runtime.exec does not close all file descriptors on Solaris 9
  • Type: Bug
  • Component: core-libs
  • Sub-Component: java.lang
  • Affected Version: 1.4.2
  • Priority: P3
  • Status: Resolved
  • Resolution: Fixed
  • OS: solaris_9
  • CPU: sparc
  • Submitted: 2005-10-20
  • Updated: 2016-04-15
  • Resolved: 2006-04-01
The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.
Other JDK 6
1.4.2_13Fixed 6 b79Fixed
Related Reports
Relates :  
JDK-4725923 - (process) UNIXProcess_forkAndExec hangs (1.4.0_02)
Relates :  
JDK-8068370 - (process) Executing recently modified file results in ETXTBSY
Description
FULL PRODUCT VERSION :
java version "1.5.0_05"
Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_05-b05)
Java HotSpot(TM) Server VM (build 1.5.0_05-b05, mixed mode)


java version "1.4.2_09"
Java(TM) 2 Runtime Environment, Standard Edition (build 1.4.2_09-b05)
Java HotSpot(TM) Client VM (build 1.4.2_09-b05, mixed mode)


ADDITIONAL OS VERSION INFORMATION :
SunOS mymachine 5.9 Generic_112233-12 sun4u sparc SUNW,Sun-Fire-V240

A DESCRIPTION OF THE PROBLEM :
When running Runtime.exec it seems that the JVM only closes the first 1024 inherited filedescriptors. If the JVM has more than 1024 fildescriptors open then the child process will inherit all filedescriptors > 1024.

Tip, have a look at the C functions:

#include <stdlib.h>

 void closefrom(int  lowfd);

 int fdwalk(int (*func)(void *, int), void *cd);

They can be used to close all filedescriptors without prior knowledge to which are opened.

I've tested on several 1.4 releases, including the latest one 1.4.2_09 as well as the latest 1.5 relase. Problem occurs on all of them.



STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
Compile and run the attached testprogram.

First run with 10 open files.

java -cp . FileDescriptorTest 10

The program will open 10 files before doing Runtime.exec on /bin/bash.

Now do,

pfiles <pid of child (bash)>

In my system I get:
>ptree 22749

        22749 java -cp . FileDescriptorTest 10
          22761 /bin/bash


>pfiles 22761
22761:  /bin/bash
  Current rlimit: 8192 file descriptors
   0: S_IFIFO mode:0000 dev:316,0 ino:12545723 uid:57219 gid:205 size:0
      O_RDWR
   1: S_IFIFO mode:0000 dev:316,0 ino:12545724 uid:57219 gid:205 size:0
      O_RDWR
   2: S_IFIFO mode:0000 dev:316,0 ino:12545725 uid:57219 gid:205 size:0
      O_RDWR

This is as expected, only STDIN, STDOUT and STDERR is open.

Now rerun the program with more than 1024 files open in the parent:

java -cp . FileDescriptorTest 1030

The child now has more filedescriptors open:

pfiles 15686:
15686:  /bin/bash
  Current rlimit: 8192 file descriptors
   0: S_IFIFO mode:0000 dev:316,0 ino:12539305 uid:57219 gid:205 size:0
      O_RDWR
   1: S_IFIFO mode:0000 dev:316,0 ino:12539306 uid:57219 gid:205 size:0
      O_RDWR
   2: S_IFIFO mode:0000 dev:316,0 ino:12539307 uid:57219 gid:205 size:0
      O_RDWR
1024: S_IFREG mode:0644 dev:271,0 ino:93892 uid:57219 gid:205 size:0
      O_RDONLY|O_LARGEFILE
1025: S_IFREG mode:0644 dev:271,0 ino:93893 uid:57219 gid:205 size:0
      O_RDONLY|O_LARGEFILE
1026: S_IFREG mode:0644 dev:271,0 ino:93894 uid:57219 gid:205 size:0
      O_RDONLY|O_LARGEFILE
1027: S_IFREG mode:0644 dev:271,0 ino:93895 uid:57219 gid:205 size:0
      O_RDONLY|O_LARGEFILE
1028: S_IFREG mode:0644 dev:271,0 ino:93896 uid:57219 gid:205 size:0
      O_RDONLY|O_LARGEFILE
1029: S_IFREG mode:0644 dev:271,0 ino:93897 uid:57219 gid:205 size:0
      O_RDONLY|O_LARGEFILE
1030: S_IFREG mode:0644 dev:271,0 ino:93898 uid:57219 gid:205 size:0
      O_RDONLY|O_LARGEFILE
1031: S_IFREG mode:0644 dev:271,0 ino:93899 uid:57219 gid:205 size:0
      O_RDONLY|O_LARGEFILE
1032: S_IFREG mode:0644 dev:271,0 ino:93900 uid:57219 gid:205 size:0
      O_RDONLY|O_LARGEFILE
1033: S_IFREG mode:0644 dev:271,0 ino:93901 uid:57219 gid:205 size:0
      O_RDONLY|O_LARGEFILE
1034: S_IFREG mode:0644 dev:271,0 ino:93902 uid:57219 gid:205 size:0
      O_RDONLY|O_LARGEFILE



EXPECTED VERSUS ACTUAL BEHAVIOR :
EXPECTED -
>pfiles 22761
22761:  /bin/bash
  Current rlimit: 8192 file descriptors
   0: S_IFIFO mode:0000 dev:316,0 ino:12545723 uid:57219 gid:205 size:0
      O_RDWR
   1: S_IFIFO mode:0000 dev:316,0 ino:12545724 uid:57219 gid:205 size:0
      O_RDWR
   2: S_IFIFO mode:0000 dev:316,0 ino:12545725 uid:57219 gid:205 size:0
      O_RDWR
ACTUAL -
pfiles 15686:
15686:  /bin/bash
  Current rlimit: 8192 file descriptors
   0: S_IFIFO mode:0000 dev:316,0 ino:12539305 uid:57219 gid:205 size:0
      O_RDWR
   1: S_IFIFO mode:0000 dev:316,0 ino:12539306 uid:57219 gid:205 size:0
      O_RDWR
   2: S_IFIFO mode:0000 dev:316,0 ino:12539307 uid:57219 gid:205 size:0
      O_RDWR
1024: S_IFREG mode:0644 dev:271,0 ino:93892 uid:57219 gid:205 size:0
      O_RDONLY|O_LARGEFILE
1025: S_IFREG mode:0644 dev:271,0 ino:93893 uid:57219 gid:205 size:0
      O_RDONLY|O_LARGEFILE
1026: S_IFREG mode:0644 dev:271,0 ino:93894 uid:57219 gid:205 size:0
      O_RDONLY|O_LARGEFILE
1027: S_IFREG mode:0644 dev:271,0 ino:93895 uid:57219 gid:205 size:0
      O_RDONLY|O_LARGEFILE
1028: S_IFREG mode:0644 dev:271,0 ino:93896 uid:57219 gid:205 size:0
      O_RDONLY|O_LARGEFILE
1029: S_IFREG mode:0644 dev:271,0 ino:93897 uid:57219 gid:205 size:0
      O_RDONLY|O_LARGEFILE
1030: S_IFREG mode:0644 dev:271,0 ino:93898 uid:57219 gid:205 size:0
      O_RDONLY|O_LARGEFILE
1031: S_IFREG mode:0644 dev:271,0 ino:93899 uid:57219 gid:205 size:0
      O_RDONLY|O_LARGEFILE
1032: S_IFREG mode:0644 dev:271,0 ino:93900 uid:57219 gid:205 size:0
      O_RDONLY|O_LARGEFILE
1033: S_IFREG mode:0644 dev:271,0 ino:93901 uid:57219 gid:205 size:0
      O_RDONLY|O_LARGEFILE
1034: S_IFREG mode:0644 dev:271,0 ino:93902 uid:57219 gid:205 size:0
      O_RDONLY|O_LARGEFILE



REPRODUCIBILITY :
This bug can be reproduced always.

---------- BEGIN SOURCE ----------
import java.io.*;

public class FileDescriptorTest
{
  public static void main(String[] args)
    throws Exception
  {
    if (args.length != 1)
    {
      System.err.println("Usage: FileDescriptorTest <numfiles>");
      System.exit(1);
    }
    
    File[] files = new File[Integer.parseInt(args[0])];
    InputStream[] fileStreams = new FileInputStream[files.length];
    for (int i=0; i<files.length; i++)
    {
      files[i] = File.createTempFile("fdtest",null);
      files[i].deleteOnExit();
      fileStreams[i] = new FileInputStream(files[i]);
    }

    System.out.println(files.length + " files now open!");
    
    Runtime.getRuntime().exec("/bin/bash");
    
    System.out.println("Started child process");
    
    System.out.println("Sleeping 60 seconds");
    Thread.currentThread().sleep(60 * 1000);
    
  }
}
---------- END SOURCE ----------

CUSTOMER SUBMITTED WORKAROUND :
None that I'm aware of.
Comments
EVALUATION It appears that we can workaround 6395699 /proc/self/fd fails to report file descriptors >= 1024 on Solaris 9 by using readdir64 instead of readdir
23-03-2006
SUGGESTED FIX --- /tmp/geta22124 2006-03-23 11:53:50.901373200 -0800 +++ UNIXProcess_md.c 2006-03-23 11:43:09.021127000 -0800 @@ -269,15 +269,15 @@ } } static int closeDescriptors(void) { DIR *dp; - struct dirent *dirp; + struct dirent64 *dirp; int from_fd = 3; /* We're trying to close all file descriptors, but opendir() might * itself be implemented using a file descriptor, and we certainly * don't want to close that while it's in use. We assume that if * opendir() is implemented using a file descriptor, then it uses * the lowest numbered file descriptor, just like open(). So we @@ -285,15 +285,15 @@ close(from_fd); /* for possible use by opendir() */ close(from_fd + 1); /* another one for good luck */ if ((dp = opendir("/proc/self/fd")) == NULL) return 0; - while ((dirp = readdir(dp)) != NULL) { + while ((dirp = readdir64(dp)) != NULL) { int fd; if (isdigit(dirp->d_name[0]) && (fd = strtol(dirp->d_name, NULL, 10)) >= from_fd + 2) close(fd); } closedir(dp);
23-03-2006
SUGGESTED FIX At the very least, we should mitigate the problem by setting the CLOEXEC flag on all file descriptors created by the Java core libraries, where the underlying fd would in any case not be accessible to subprocesses. The underlying bug is in Solaris 9.
23-10-2005
EVALUATION I can reproduce this on Solaris 9, but not Solaris 10. Seems like there is a bug in Solaris that fails to return more than 1024 fds when iterating via readdir() over /proc/self/fd. We cannot easily use the submitter's suggestion of using closefrom() or fdwalk() since those are not available on Solaris 8 or Linux. Unfortunately, "closing all file descriptors" is surprisingly difficult. For another non-obvious trap to fall into, see 4725923: (process) UNIXProcess_forkAndExec hangs (1.4.0_02)
23-10-2005