JDK-8035897 : Better memory allocation for file descriptors greater than 1024 on macosx
  • Type: Bug
  • Status: Closed
  • Resolution: Fixed
  • Component: core-libs
  • Sub-Component: java.net
  • Priority: P1
  • Affected Version: 7u25,8,9
  • OS: os_x
  • CPU: generic
  • Submit Date: 2014-02-26
  • Updated Date: 2014-10-15
  • Resolved Date: 2014-03-02
The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availabitlity Release.

To download the current JDK release, click here.
JDK 7 JDK 8 JDK 9
7u60Fixed 8u11Fixed 9 b04Fixed
Related Reports
Duplicate :  
Duplicate :  
Relates :  
Relates :  
Description
JDK-8021820 adds -D_DARWIN_UNLIMITED_SELECT to the build, but the fd_set struct is still limited to 1024.

Snippet from man select(2):

   "Although the provision of getdtablesize(2) was intended to allow user programs to be written
     independent of the kernel limit on the number of open files, the dimension of a sufficiently
     large bit field for select remains a problem.  The default size FD_SETSIZE (currently 1024) is
     somewhat smaller than the current kernel limit to the number of open files.  However, in order
     to accommodate programs which might potentially use a larger number of open files with select,
     it is possible to increase this size within a program by providing a larger definition of
     FD_SETSIZE before the inclusion of <sys/types.h>.

Either:
 1) FD_SETSIZE needs to be set to a larger value, but what value, the kernel limit, or other?
     This is wasteful for most typical apps that don't use large numbers of file descriptors. Or,
 2) If fd is greater than 1024, then an appropriate amount of memory could be allocated and cast
     to an fd_set. The FD_SET macro will write past FD_SETSIZE.

Option 2 above seems most reasonable. Will proceed with making the appropriate changes.
Comments
Related Regression test: test/java/net/ServerSocket/AnotherSelectFdsLimit.java Platform for Verification: sc22bk02.us.oracle.com MacOSX 10.8.4 Testing Results: Run with Unfixed version of java (jdk9/b03) 22bk02:JDK-8035897 aurora$ /net/sqenfs-1/export1/jdk/re/9/promoted/all/b03/binaries/macosx-amd64/bin/java AnotherSelectFdsLimit 1025 Using 1025 fds. Exception in thread "Thread-1023" Exception in thread "Thread-1024" java.io.UncheckedIOException: java.net.SocketException: Socket closed at AnotherSelectFdsLimit$WorkerThread.run(AnotherSelectFdsLimit.java:74) Caused by: java.net.SocketException: Socket closed at java.net.PlainSocketImpl.socketAccept(Native Method) at java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:404) at java.net.ServerSocket.implAccept(ServerSocket.java:545) at java.net.ServerSocket.accept(ServerSocket.java:513) at AnotherSelectFdsLimit$WorkerThread.run(AnotherSelectFdsLimit.java:70) java.io.UncheckedIOException: java.net.SocketException: Socket closed at AnotherSelectFdsLimit$WorkerThread.run(AnotherSelectFdsLimit.java:74) Caused by: java.net.SocketException: Socket closed at java.net.PlainSocketImpl.socketAccept(Native Method) at java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:404) at java.net.ServerSocket.implAccept(ServerSocket.java:545) at java.net.ServerSocket.accept(ServerSocket.java:513) at AnotherSelectFdsLimit$WorkerThread.run(AnotherSelectFdsLimit.java:70) sc22bk02:JDK-8035897 aurora$ /net/sqenfs-1/export1/jdk/re/9/promoted/all/b03/binaries/macosx-amd64/bin/java AnotherSelectFdsLimit Using 1600 fds. Exception in thread "Thread-1271" Exception in thread "Thread-1279" Exception in thread "Thread-1283" Exception in thread "Thread-1285" java.io.UncheckedIOException: java.net.SocketException: Invalid argument at AnotherSelectFdsLimit$WorkerThread.run(AnotherSelectFdsLimit.java:74) Caused by: java.net.SocketException: Invalid argument at java.net.PlainSocketImpl.socketAccept(Native Method) at java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:404) at java.net.ServerSocket.implAccept(ServerSocket.java:545) at java.net.ServerSocket.accept(ServerSocket.java:513) at AnotherSelectFdsLimit$WorkerThread.run(AnotherSelectFdsLimit.java:70) Exception in thread "Thread-1294" Exception in thread "Thread-1291" java.io.UncheckedIOException: java.net.SocketException: Invalid argument at AnotherSelectFdsLimit$WorkerThread.run(AnotherSelectFdsLimit.java:74) Caused by: java.net.SocketException: Invalid argument ......... _______________________________________________________ Run with fixed version of Java (JDK9/b04): 22bk02:JDK-8035897 aurora$ /net/sqenfs-1/export1/jdk/re/9/promoted/all/b04/binaries/macosx-amd64/bin/java AnotherSelectFdsLimit 1025 Using 1025 fds. sc22bk02:JDK-8035897 aurora$ /net/sqenfs-1/export1/jdk/re/9/promoted/all/b04/binaries/macosx-amd64/bin/java AnotherSelectFdsLimit 1600
2014-03-28

SQE OK to take the fix into 7u60
2014-03-14

Removing CPU14_02-critical-watch. Management have indicated that a patch will be provided from the sustaining group, if needed. This change will make its way into the next available regular update in 7uXX and 8uXX.
2014-03-13

Adjusting Priority, to P1, as per ILW ( Impact:H, Likelihood:M, Workaround:H )
2014-03-11

bsd_close.c has the same problem. It uses getrlimit() to figure out what the actual kernel limit is before allocating its fd array.
2014-02-27