Bug ID: JDK-8360401 [AIX] java/lang/ProcessBuilder/FDLeakTest/FDLeakTest.java fails since JDK-8210549

Type: Bug
Component: core-libs
Sub-Component: java.lang
Affected Version: 26

Priority: P4
Status: New
Resolution: Unresolved
OS: aix
CPU: ppc

Submitted: 2025-06-24
Updated: 2025-08-20

The new tests 
java/lang/ProcessBuilder/FDLeakTest/FDLeakTest.java#fork
java/lang/ProcessBuilder/FDLeakTest/FDLeakTest.java#posix_spawn
fail on AIX.  
The tests were added with  JDK-8210549 .

Error output is 
java/lang/ProcessBuilder/FDLeakTest/FDLeakTest.java#fork

Opened and leaked ./testfile_FDLeaker.txt (4)
----------System.err:(13/647)----------
*** Parent leaked file descriptor 9 ***
*** Parent leaked file descriptor 10 ***
java.lang.RuntimeException: Failed
	at FDLeakTest.main(FDLeakTest.java:69)
	at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:104)
	at java.base/java.lang.reflect.Method.invoke(Method.java:565)
	at com.sun.javatest.regtest.agent.MainWrapper$MainTask.run(MainWrapper.java:138)
	at java.base/java.lang.Thread.run(Thread.java:1474)
 
 
java/lang/ProcessBuilder/FDLeakTest/FDLeakTest.java#posix_spawn


----------System.out:(1/46)----------
Opened and leaked ./testfile_FDLeaker.txt (4)
----------System.err:(13/647)----------
*** Parent leaked file descriptor 9 ***
*** Parent leaked file descriptor 10 ***
java.lang.RuntimeException: Failed
	at FDLeakTest.main(FDLeakTest.java:69)
	at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:104)
	at java.base/java.lang.reflect.Method.invoke(Method.java:565)
	at com.sun.javatest.regtest.agent.MainWrapper$MainTask.run(MainWrapper.java:138)
	at java.base/java.lang.Thread.run(Thread.java:1474)

Seems markDescriptorsCloseOnExec https://github.com/openjdk/jdk/blob/4ffd2a8aa45fa78c2546e84dc908263e7f342484/src/java.base/unix/native/libjava/childproc.c#L86 needs modification on AIX because using the /proc filesystem is not reliable . Btw. there seems to be similar code here : (markDescriptorsCloseOnExec in libjdwp) https://github.com/openjdk/jdk/blob/40bc083267c0bac64b1092c63325b8d245937b0e/src/jdk.jdwp.agent/unix/native/libjdwp/exec_md.c#L81 maybe we have to change this too?
20-08-2025
HI Thomas, regarding your helpful notes: When calling the leak tester exe directly from the command line I get also a leaked fd on AIX. $ ./jtreg/test/jdk/jtreg/native/FDLeakTester * Parent leaked file descriptor 4 *
02-07-2025
[~azeller] Well, then you have a lead to follow. Find out who opens that file descriptor on AIX in this simple program. Note that it could be multiple FDs: the test fails with two open fds, your command line invocation with one open fd. To my understanding, and the test relies on that, a unix program started should have exactly three open fds: 0, 1, 2.
02-07-2025
Hi Thomas [~stuefe] , > No idea. I would look at /proc/pid/fd to find out information about open FDs This seems not to work , even for one of the processes of my user I get on AIX more /proc/35652088/fd/* /proc/35652088/fd/15: Permission denied /proc/35652088/fd/16: Permission denied /proc/35652088/fd/17: Permission denied /proc/35652088/fd/18: Permission denied /proc/35652088/fd/19: Permission denied I am not aware of instrumentation libraries used in this test, but will check with my colleagues.
01-07-2025
[~mbaesken] No idea. I would look at /proc/pid/fd to find out information about open FDs (would have to be added to the exeFDLeaker.c test program) See also https://www.ibm.com/docs/en/aix/7.2.0?topic=files-proc-file possibly helpful notes: - mark close at exec generally seems to work I think - if fcntl in childproc.c had returned an error, we would get a different test error. - The failing FD is not the test FD we deliberately open without cloexec in the test agentlib. That one was number 4. The failing FDs are 9 and 10. I would try to find out if these FDs were even open in the parent process (the JVM), so if the mark-close-on-exec logic did not work. I wonder whether they were instead opened anew at the start of the child program (somewhere before main) possibly some interposition library or similar. A very simple way to check this is to call the exeFDLeaker just on the command line. Do they also report open FDs? Possibly some instrumentation library interposing every process start? Again, would be very helpful to know what files/sockets these FDs point to.
01-07-2025
The test could be enhanced with extra diagnostics by lookups of /proc/self/fd for each of the failing fds.
25-06-2025
Hi [~stuefe] , any ideas what might be different on AIX ? We do not see failures on other platforms. and from what I read, the fcnl should work on AIX too.
25-06-2025