Bug ID: JDK-8291060 OPEN_MAX is no longer the max limit on macOS >= 10.6 for RLIMIT

Type: Enhancement
Component: hotspot
Sub-Component: runtime

Priority: P4
Status: Closed
Resolution: Fixed
OS: os_x

Submitted: 2022-07-26
Updated: 2024-01-24
Resolved: 2022-08-01

JDK 20
20 b09Fixed

We are currently using setrlimit(RLIMIT_NOFILE, OPEN_MAX) for XX:+MaxFDLimit, but we can go higher than that.

My own testing on macOS 12.3.1 reveals that we can ask for setrlimit(RLIMIT_NOFILE, RLIM_INFINITY), though in reality we will get back 32765, which is still more than OPEN_MAX (10240)

Change was backed out - hence Verify -> Fix failed
14-12-2023
The PR for the backout is out https://github.com/openjdk/jdk20/pull/102
12-01-2023
I tested on the default macOS terminal i.e. "zsh" and it did not even occur to me that another terminal might be treating the RLIMIT_NOFILE value in a way that could trigger a crashing bug. I thought the os itself would be responsible for handling it without direct involvement of any shells.
12-01-2023
> in reality we will get back 32765 This seems not to be the case as per JDK-8299258.
23-12-2022
Changeset: 1df77ec1 Author: Gerard Ziemski <gziemski@openjdk.org> Date: 2022-08-01 16:07:11 +0000 URL: https://git.openjdk.org/jdk/commit/1df77ec1375fd3260c683ac77e5f17676fbff944
01-08-2022
A pull request was submitted for review. URL: https://git.openjdk.org/jdk/pull/9650 Date: 2022-07-26 21:54:11 +0000
26-07-2022
https://github.com/openjdk/jdk/pull/9650
26-07-2022
Here is a suggestion on how we could fix it: diff --git a/src/hotspot/os/bsd/os_bsd.cpp b/src/hotspot/os/bsd/os_bsd.cpp index 49b5ce1cb50..f97e3508bcb 100644 --- a/src/hotspot/os/bsd/os_bsd.cpp +++ b/src/hotspot/os/bsd/os_bsd.cpp @@ -1996,16 +1996,19 @@ jint os::init_2(void) { if (status != 0) { log_info(os)("os::init_2 getrlimit failed: %s", os::strerror(errno)); } else { + // On macOS >= 10.6 if we define either _DARWIN_UNLIMITED_STREAMS or _DARWIN_C_SOURCE + // (we define _DARWIN_C_SOURCE) we can ask for RLIM_INFINITY nbr_files.rlim_cur = nbr_files.rlim_max; - -#ifdef __APPLE__ - // Darwin returns RLIM_INFINITY for rlim_max, but fails with EINVAL if - // you attempt to use RLIM_INFINITY. As per setrlimit(2), OPEN_MAX must - // be used instead - nbr_files.rlim_cur = MIN(OPEN_MAX, nbr_files.rlim_cur); -#endif - status = setrlimit(RLIMIT_NOFILE, &nbr_files); +#ifdef __APPLE__ + if (status != 0) { + // On macOS < 10.6 Darwin returns RLIM_INFINITY for rlim_max, + // but fails with EINVAL if you attempt to use RLIM_INFINITY. + // As per setrlimit(2), OPEN_MAX must be used instead. + nbr_files.rlim_cur = MIN(OPEN_MAX, nbr_files.rlim_cur); + status = setrlimit(RLIMIT_NOFILE, &nbr_files); + } +#endif // #ifdef __APPLE__ if (status != 0) { log_info(os)("os::init_2 setrlimit failed: %s", os::strerror(errno)); }
26-07-2022

Relates :	JDK-8299258 - Launching ksh (shell) from within Java through ProcessBuilder causes ksh process to crash with EXC_BAD_ACCESS (SIGSEGV)
Relates :	JDK-8324577 - [REDO] - [IMPROVE] OPEN_MAX is no longer the max limit on macOS >= 10.6 for RLIMIT_NOFILE
Relates :	JDK-8286345 - runtime/NMT/ThreadedMallocTestType.java failed with java.lang.RuntimeException, we also see "exitValue = 134" and time out
Relates :	JDK-8300088 - [IMPROVE] OPEN_MAX is no longer the max limit on macOS >= 10.6 for RLIMIT_NOFILE
Relates :	JDK-8300055 - [BACKOUT] OPEN_MAX is no longer the max limit on macOS >= 10.6 for RLIMIT_NOFILE
Relates :	JDK-8291067 - macOS should use O_CLOEXEC instead of FD_CLOEXEC