FULL PRODUCT VERSION :
java version "1.8.0_40"
Java(TM) SE Runtime Environment (build 1.8.0_40-b26)
Java HotSpot(TM) 64-Bit Server VM (build 25.40-b25, mixed mode)
ADDITIONAL OS VERSION INFORMATION :
Linux smidge 3.13.0-63-generic #103-Ubuntu SMP Fri Aug 14 21:42:59 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux
A DESCRIPTION OF THE PROBLEM :
If a service class is loaded from jar file via the java.util.ServiceLoader, if that jar file is subsequently deleted, a native file handle is leaked. This occurs when the jar file is NOT on the system classpath, but on a child URLClassLoader.
The most easily seen use case is the redeployment of a webapp context: the webapp context is deployed as a war file, unpacked by the web server to a temporary location, and a URLClassLoader created for the jars in the unpacked WEB-INF/lib location. If one of these jars contains a javax.servlet.ServletContainerInitializer that is loaded via the java.util.ServiceLoader, a subsequent redeploy of the webapp (involving the deletion of the previous temporary unpack location) will leave a file handle open, marked as deleted like so:
java 18249 janb DEL REG 8,5 17705538 /tmp/foo-1443160317112/sci-1.0-SNAPSHOT.jar
java 18249 janb 72r REG 8,5 2142 17705538 /tmp/foo-1443160317112/sci-1.0-SNAPSHOT.jar (deleted)
Note that the URLClassLoader was properly disposed of, with a call to close() first, and that the ServiceLoader's reload() method was also called to flush it's internal cache.
This is NOT specific to the ServletContainerInitializer, it happens with any class that is loaded via the ServiceLoader.
STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
1. Create a simple interface eg MyInterface
2. Create a simple implementation of that interface eg MyImplementation and put it in a jar eg impl.jar
3. Create an application that only has the interface on the runtime classpath, and that copies the impl.jar to a tmp location and then creates a URLClassloader with this location. The app should then use the ServiceLoader to load MyImplementation using MyInterface.class. Then, the app should clear out the ServiceLoader's cache, close the URLClassLoader and delete the jar file in the temporary location.
4. Using lsof -p <pid> you will see the file handles remaining to the impl.jar.
I have a small example maven project that I could attach to this issue if necessary.
EXPECTED VERSUS ACTUAL BEHAVIOR :
EXPECTED -
Expected result is that the impl.jar should disappear from the list of open files.
ACTUAL -
File handles remain to the deleted impl.jar until the app's process is stopped.
REPRODUCIBILITY :
This bug can be reproduced always.
---------- BEGIN SOURCE ----------
MyInterface.java
-----------------------
package leaker;
public interface MyInterface
{
public long getLong();
}
MyImpl.java
-----------------
package leaker;
public class MyImpl implements MyInterface
{
public long getLong()
{
return 1L;
}
}
META-INF/services/leaker.MyInterface
----------------------------------------------------
leaker.MyImpl
App.java
-------------
package leaker;
import java.io.File;
import java.io.IOException;
import java.net.URL;
import java.net.URLClassLoader;
import java.nio.file.*;
import java.nio.file.attribute.*;
import java.util.ServiceLoader;
public class App
{
ClassLoader old;
ServiceLoader<MyInterface> services;
URLClassLoader ucl;
File tmpdir;
public App()
{
}
public void copy (File from, File to)
throws IOException
{
Files.copy(from.toPath(), to.toPath());
}
public void setupLoader ()
throws Exception
{
File jars = new File("../sci/target");
File f = new File(jars, "sci-1.0-SNAPSHOT.jar");
tmpdir = new File("/tmp/foo-"+System.currentTimeMillis());
tmpdir.mkdirs();
File copy = new File (tmpdir, "sci-1.0-SNAPSHOT.jar");
copy(f, copy);
System.err.println("Copied jar containing service to: "+copy.getAbsolutePath());
URL u = copy.toURL();
old = Thread.currentThread().getContextClassLoader();
ucl = new URLClassLoader(new URL[]{u}, old);
Thread.currentThread().setContextClassLoader(ucl);
services = ServiceLoader.load(MyInterface.class);
for (MyInterface m:services)
{
System.err.println("Loaded service: "+m.getClass().getName());
}
}
public void teardownLoader()
throws Exception
{
//replace the classloader
Thread.currentThread().setContextClassLoader(old);
//clear services cache
services.reload();
//close all urls in classloader
ucl.close();
ucl = null;
services = null;
System.gc();
//delete the tmp directory
Files.walkFileTree(tmpdir.toPath(), new SimpleFileVisitor<Path>()
{
@Override
public FileVisitResult visitFile(Path file, BasicFileAttributes attrs)
throws IOException
{
Files.delete(file);
return FileVisitResult.CONTINUE;
}
@Override
public FileVisitResult postVisitDirectory(Path dir, IOException e)
throws IOException
{
if (e == null)
{
Files.delete(dir);
return FileVisitResult.CONTINUE;
}
else
{
// directory iteration failed
throw e;
}
}
});
}
public static final void main (String[] args)
{
App app = new App();
try
{
System.err.println();
System.err.println();
System.err.println("Do \"ps -ef | grep maven\" to find the pid for this process using another window.");
System.err.println();
System.err.println("Setting up service loader ...");
app.setupLoader();
System.err.println("Services loaded. Do \"lsof -p <this pid> | grep sci\" to see the file handle to the jar containing the service class.");
System.err.println("Sleeping 30sec ...");
Thread.currentThread().sleep(30000);
System.err.println("Clearing loader ...");
app.teardownLoader();
System.err.println("Cleared loader. Do \"lsof -p <this pid> | grep sci\" to see the leaked deleted file handle to the jar containing the service class.");
System.err.println("Sleeping 20sec ...");
Thread.currentThread().sleep(20000);
System.err.println("Done");
}
catch (Exception e)
{
e.printStackTrace();
}
}
}
---------- END SOURCE ----------