JDK-8068370 : (process) Executing recently modified file results in ETXTBSY
  • Type: Bug
  • Component: core-libs
  • Sub-Component: java.lang
  • Affected Version: 7u71,8,9
  • Priority: P4
  • Status: Open
  • Resolution: Unresolved
  • OS: linux
  • CPU: x86,x86_64
  • Submitted: 2014-12-03
  • Updated: 2020-04-10
The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.
Other
tbd_minorUnresolved
Related Reports
Relates :  
JDK-6339493 - (process) Runtime.exec does not close all file descriptors on Solaris 9
Relates :  
JDK-8044841 - java/lang/Runtime/exec/SleepyCat.java Exception: Read from closed pipe hangs
Relates :  
JDK-4843136 - (process) pipe file descriptor from Runtime.exec not being closed
Relates :  
JDK-8210549 - Runtime.exec: in closeDescriptors(), use FD_CLOEXEC instead of close()
Relates :  
JDK-8041711 - (process) Process stream lost with bash background scripts (lnx)
Relates :  
JDK-8001334 - Remove use of JVM_* functions from java.io code
Description
FULL PRODUCT VERSION :
java version "1.7.0_71"
Java(TM) SE Runtime Environment (build 1.7.0_71-b14)
Java HotSpot(TM) 64-Bit Server VM (build 24.71-b01, mixed mode)


ADDITIONAL OS VERSION INFORMATION :
Linux tuohi 3.16.5-gentoo #1 SMP Thu Oct 30 08:55:22 EET 2014 x86_64 Intel(R) Core(TM) i5-2400 CPU @ 3.10GHz GenuineIntel GNU/Linux

EXTRA RELEVANT SYSTEM CONFIGURATION :
Filesystem is ext3

A DESCRIPTION OF THE PROBLEM :
See the attachment. There is a shell script that compiles and runs a java program that reproduces the problem.

In a nutshell: many threads run a loop that deletes and recreates a thread-specific temporary directory, writes a shell script under the directory and tries to run it. Sometimes the script run fails and throws java.io.IOException: error=26, Text file busy.

STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
See attachment

EXPECTED VERSUS ACTUAL BEHAVIOR :
EXPECTED -
No exception thrown, all script runs work.
ACTUAL -
A lot of stack traces, then

done, errors: 35
failures: 35
successes: 4965
failure ratio 0.7%
missing out files: 35


ERROR MESSAGES/STACK TRACES THAT OCCUR :
java.io.IOException: error=26, Text file busy.

REPRODUCIBILITY :
This bug can be reproduced occasionally.

---------- BEGIN SOURCE ----------
Ah, you don't support sending attachments? So, here's a base64 encoded tar.gz that contains mentioned script and java file:

H4sIAL8Gf1QAA+1ZbW/bNhDOV+lXsFqLyEssv8SJgaQJlnTJFqCtizpDP3T9IEu0zVqmPJKKk63+
7zu+SbJj56UN0hUzgUDi8e54vHvujlYYnrA0ziJcFfhK9EmCexm/rm085qjDaO/uymejvVsvP+3Y
aDRbO+29+k5zp7VRb7R2dhobaPdRrVgxMi5ChtDGlEyy2/juWv9BB1sef86ix8PAw+O/22jW1/F/
inFL/FM2eBwMfEX82/X2Ov5PMe6If0pGISMU029BwsPj397b21vH/ynGveN/SZIEfx0KHhz/ZgM4
1vF/ivHA+JeZetk9+8PD499qNtf5/yTjG+N/AfMzmJ/A/CQbBJ/Dy/DGHjLAe63Wqvi3m+29+fjv
1Bs7zQ1UfwoH/M/jPwmjUTjACIId5MEOVLCDhWAfuC4ZT1ImkIxyQNJARv5gGfEDIwKzG0vnndOr
CE8ESen8WiZIEhwzFl6/JlysWuNLFlbwRymNMsYwFUEo0jGJgmP1OKcCD6RlVkSeOwQnDDHIjMcp
5fYMf4Ae2NGdZL2ERChKQs7RAt7RP67rTBi5DAVGACQBjHIZifHkV8LQIaJ4qii+VwPaYv54lYMb
8n1CwwQRKtA0ZSPMXqUZvB+i3fptvCLkI8V5cv1BiYEEdNFVInPeQP2QJBnD3Ng7t+ivNHFeB8+i
CHO+Wglo0Z40Si5TEqNxSKjfFYC6wcdPKGQDXkFiyNIpRzlUwMmOQ/rIl8tBgulADNERqlfUgnPT
1ZLvY/2TNNyZwV+ME19zKdJ4FBPGyxQJopcXQ4bD+Mh4PT+GBeXLI+UJp58y5EuPE+CoH8DjZTlQ
QNjaMpZpjfI88qH1aZovX3WcfFLZRt606qEtRNQOjrEgCOPY18L5UdTuVi/at8ZaVwSymAn/3uyf
U3B/zt295gKPA8xYAAGnIqG+F6cUbyMgpYzvI2mkBYsSWyZiGe7JngNH8+fTpQL9XD1igKIUvQga
/Rcv/qTetjyR4wPmz9DP+a7BAIM7UK1QqykVBUinVkOQ1VjWgUxMMsFR7xrxiJGJ4PsSdRDnMeEc
4NmRqzLiZQxMNQamixiY5hjIWYVmFcB6I1eBnAs4OhsU09twjEFsCm7xFECEwoejSoy2GJbhRRI6
/a4y3LeSGkwqc55p5gBfAZI5HN/s5ZQOt7Wl+R2onYLQDOvprGyUXMKqGuUVMpDgkrOLVDP5eq/y
7kYswH9lYcJzA+WxIHKFNSrzVZqcQ/8ZhEkXSgXOC4HvXYCkQolVoeUcBzRZh0xDrljMppXSMWa3
AN14QmpBskIbOJYcJDXNbtZ7Xcd0TVGBgbdKXrOewSwwq/agdx5T8ysQ41jbEZtSNVttgyxzygBp
fskCOV2MPMMiY9QmvmRTXIRDScSRSNl1zqoQrPRGQ5LEUEUUawL6JLWEJmmA4tE+n5W0ayOAAQt8
fz9o/jk/qLOtdoTu0qYDQotP8BgwwNH7jNKwB2eQDTsXKtonBSypimCLsqHllhIeUJ2NmtVRuzu/
dC4BRSSWQDTtTcWCZVBXFypAqWEsqQBF23AEu7YZofyuS8tCk9PtC1pHTS9L59Ai61UwckFLNEAs
00v78GF5g5wJ9tAl0bNqpvJ6p5j4cNvUS5+ajDadzPKq6PNhwLE4vcJRJmQYfMEyXOT93Ug4UxBA
IgUQjtM4t2Rm7FcdWtefIxSN49dwjYXD6GtjEHK5DsbK4n/c42mSCfwuFEM/N/MdS2V/QBPz1I4w
1JMMQA2oMIorQZxniS1AhUuLHmwcAKGHTpQlsmwa9cE0JOIs1Zcr6yXDVNxs7uEYu73j6eKPIM8F
OGpK4I5EU1r9G7NUJUemq6LepHCfehTNkdCIqZQ5pvFvWFj7ZigKRTREfnEpw7mRea9dIexgXWXB
9Gh0wcII+zeL8uqyVoAtL27baL4f5XfG0g8MZR30d5oKlHHNrO73VZJuSyCxTDdRkMUI7uKoh1Xf
HWIms1kmoV/8lgEzSsmhaapmWhRPGfyOmGAa++XGs7xK2f4917bNkYqqo2v0koyX1aMK4tVyzi/Z
xmjkK/RrgsF2kfxzZB0pSBxzNu+nZ7UeobVeyIeydS+sQpKjKs6WrOBomCJrrywTgPNNOV90g7YP
GDatFuMHUCbsFUMfeOZ+71/PP/5Y8f0H+lfAh4+0h/rIs/r7D6w1F///02rvrr//PMUo5bPrmvR1
3d9P358ePvehzal09Z7XvYolyvt+QugIVfuwIImw5r56fdztnnYPFaGmrmGYu+q+gaoTYDQMnute
vHln2KCiuWyMqkxqArJXFlBz91XnzZvO2+55x2rOi3jwOWTuR7BXKrdcHvqEvnyB8obQdCBP05lf
3RwKMdmv1QD2aSMYh5eYBvI7p3prlrSXX5tBqzStwlTuvQn1x5VfnCJUjSbz21Tj0onRc2gUNDbO
kp9WgUH5dfNn9cF0s6IVWT1abr+s8B6f54LFb1PGhd8bYeuxHuuxHuvxXxz/Ahs5mbgAKAAA

---------- END SOURCE ----------

CUSTOMER SUBMITTED WORKAROUND :
I haven't tried, but maybe retry can help. But that would require parsing the exception message, and there is a risk that we suppress other problems by accident.
Comments
Specifying O_CLOEXEC doesn't completely resolve the problem, but makes it happen less frequent.
10-06-2016
My suspicion is that the reason is due to the following race: When one thread starts a child process, it forks the parent process and copies the table of opened file descriptors. If another thread at that moment finishes writing, closes file and tries to execute it, it would fail because the first forked process still has the file opened. Update: the same behavior observed even with -Djdk.lang.Process.launchMechanism=VFORK When vfork() is used, the parent should get suspended until the child calls exec_(). exec() functions inherit all the opened files unless they have FD_CLOEXEC flag set on the descriptor. Update2: a comment in ProcessImpl_md.c says: "Files are created FD_CLOEXEC in Java.", but in fact they are not. Adding O_CLOEXEC to the open flags resolves the issue. Now, need to understand what would be a drawback of adding this flag to every open() call.
14-04-2016
Managed to reproduce it with nio -------------------------- import java.nio.*; import java.util.*; import java.nio.file.*; import java.nio.file.attribute.*; public class U { public static void main(String[] args) throws Throwable { for (int j = 0; j != 3; ++j) { Path subdir = Files.createDirectory(Paths.get("subdir" + j)); new Thread(() -> { for (int i = 0; i != 1000000; ++i) { try { Path sh = subdir.resolve(Paths.get("script" + i)); Files.write(sh, "#!/bin/bash\n".getBytes()); Files.setPosixFilePermissions(sh, Collections.singleton(PosixFilePermission.OWNER_EXECUTE)); new ProcessBuilder(sh.toString()).start().waitFor(); } catch (Exception e) { throw new RuntimeException(e); } }} ).start(); } } }
13-04-2016
I confirm the failure is observed with 7u, 8u and 9. The testcase can be reduced to the following short block being run concurrently in many threads: File sh = new File(workerDir, "script"); FileOutputStream wr = new FileOutputStream(sh); wr.write("#!/bin/bash\n".getBytes()); wr.close(); sh.setExecutable(true); new ProcessBuilder(sh.getAbsolutePath()).start().waitFor(); Here workerDir is unique for each run, so no threads should write to the same file. On my machine it results in one 'Text file busy' failures on every ~250 cycles. I suspect it may have something to do with the descriptor recycling. If fd.sync() is added right before close, the number of failures is greatly increased -- up to almost 1 failure on every 2 cycles.
19-01-2015
Comment from the original submitter: "Please run the program, even if you doubt it. No external process is locking any files, it's just the reproducer program itself. It's not my place to speculate, but if I have to, I think it's possible that a call to File.delete or File.setExecutable or Process.waitFor is not synchronous i.e. it keeps the file locked after returning. Or then it is something else, but definitely the bug is in the standard library, if not in the JVM."
19-01-2015
I have not looked at the reproducer but I doubt this is a JDK bug. The error "Text file busy" just means that someone has the script/program open for writing.
30-12-2014
Attached the test case file send by the submitter. Couldn't reproduce the issue as mentioned. However, moving thi sup for further review.
30-12-2014
Requested the submitter to send the script example as an attachment or any other useful information to be able to reproduce this issue.
04-12-2014