JDK-6399443 : ThreadPoolExecutor leak
  • Type: Bug
  • Component: core-libs
  • Sub-Component: java.util.concurrent
  • Affected Version: 6
  • Priority: P3
  • Status: Closed
  • Resolution: Fixed
  • OS: linux_redhat_9.0
  • CPU: x86
  • Submitted: 2006-03-16
  • Updated: 2015-12-16
  • Resolved: 2006-04-14
The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.
JDK 6
6 b81Fixed
Related Reports
Relates :  
JDK-8145304 - Executors.newSingleThreadExecutor().submit(runnable) throws RejectedExecutionException
Description
If you create a singlethreaded executor, submit a single task into it and forget its reference, both the ThreadPoolExecutor and its associated Thread will stay alive forever although there is an obvious attempt at shutting down the thread in such case (ThreadPoolExecutor's finalizer)

While ThreadPoolExecutor.finalize() calls the shutdown() method which should tear down the waiting threads, this never happens, as the threads keep reference to the TPE instance on the stack ("this"). This causes a leak.

While we have found this problem through a bug on our side (which we already fixed), I still believe this is actual bug and the scenario with creating executors and forgetting about them should be supported.
Comments
A ThreadPoolExecutor can never be reclaimed by the GC as long as it has any live core pool threads. This means that either the pool has to be configured to have core threads that timeout when idle (a new capability added in mustang - see TPE.allowCoreThreadTimeout) or the pool has to be shutdown explicitly using the shutdown() or shutdownNow() methods. Arguably additional overloads of the Executors factory methods could be added that allow a core timeout to be set - but the point of the factory methods was to provide a simple set of common cases, not to try and cover all desirable configurations. Creating your own SingleThreadExecutor using ThreadPoolExecutor directly is not hard - the only complication arises if you want to guarantee that noone can change the number of threads used, which is what the package-private DelegatedExecutorService does. Correct use of any ExecutorService requires that it be shutdown when you are finished with it. An application that loses all references without shutting down is just poorly written. I'm wondering whether weak references could be used to deal with this. My initial thought was no because the pool must ensure all submitted tasks are executed even if the pool is no longer reachable by the main application code. But it may be possible.
16-12-2015
EVALUATION It's tricky to fix this, but there are some improvements that can be made. Doug Lea is working on a fix. Doug writes: "We (the ex-JSR166 folks) spent a while discussing options for this. We sympathize that some people will expect some thread pools to be reclaimable when no longer referenced, even when not shut down. However, in most cases, this cannot be done without introducing subtle incompatibilities with existing usages. So, we settled on 1. Change Executors.newSingleThreadedExecutor to be automatically reclaimable. (It turns out that this case CAN be handled compatibly and in a very very simple way.) 2. Add javadoc wording explaining the need to shut down ExecutorServices and in particular, those created using Executors.newFixedThreadPool. 3. Add javadoc instructions in ThreadPoolExecutor about how to configure finalizable (vs non-finalizable) pools."
23-03-2006
WORK AROUND One could create a static ExecutorService selfShutDowningExecutorService(ExecutorService es) (with a better name of course) wrapper that delegated everything to its wrapped ExecutorService, but also has a finalize() method that simply shutdown()s its wrapped ExecutorService, by analogy with Collections.checkedList Of course, this has the downside that users would have to explicitly use it, and it would further confuse users of an already confusing API.
17-03-2006
SUGGESTED FIX Weak references could be used to solve this issue, but it won't be as easy as it seems. The code would have to be refactored, the queue of the tasks to process (i.e. the object on which the Worker calls blocking getTask()) would have to be a separate class and the executor would be only a facade to the queue. Executor would reference the queue (and maybe threads), but neither the queue nor threads's stacks would strong reference the executor (maybe not even weakly). The queue would be strictly internal object, not accessible from the user code.
17-03-2006
EVALUATION Doug Lea writes: -------------- The behavior is intentional, but the documentation should be clearer. The Executors.singleThreadExecutor factory method creates a pool that guarantees that there will always be exactly one thread available, until you explicitly shut it down. This is different than a pool that creates at most one thread at a time, and if idle for long enough dies, and then, if unreferenced, eventually finalizes itself. To get this effect, you would do something like: new ThreadPoolExecutor(0, 1, 60, TimeUnit.SECONDS, new LinkedBlockingQueue<Runnable>()); You might argue that the Executors factory method should have used this configuration, but I don't think we can change this now. --------------
16-03-2006
EVALUATION Here's an actual test code for reproducing this: (run via: jver 6 jr -Xmx8m Bug) ---------------------------------------- /* * @test %I% %E% * @bug * @summary * @author Martin Buchholz */ import java.io.*; import java.util.*; import java.util.concurrent.*; import java.util.concurrent.atomic.*; public class Bug { private static void realMain(String[] args) throws Throwable { for (int i = 1; i < 100000; i++) { if (i % 1000 == 0) { System.gc(); System.runFinalization(); Thread.sleep(10); System.gc(); System.runFinalization(); Thread.sleep(10); System.out.println(i); } ExecutorService es = Executors.newFixedThreadPool(1); es.submit(new Runnable(){public void run(){}}); } } //--------------------- Infrastructure --------------------------- static volatile int passed = 0, failed = 0; static void pass() {passed++;} static void fail() {failed++; Thread.dumpStack();} static void fail(String msg) {System.out.println(msg); fail();} static void unexpected(Throwable t) {failed++; t.printStackTrace();} static void check(boolean cond) {if (cond) pass(); else fail();} public static void main(String[] args) throws Throwable { try {realMain(args);} catch (Throwable t) {unexpected(t);} System.out.printf("%nPassed = %d, failed = %d%n%n", passed, failed); if (failed > 0) throw new AssertionError("Some tests failed");} } ---------------------------------------- which will eventually die with various symptoms related to lack of memory. A good stress test for our out-of-memory handling. Here are some results, which may indeed be bugs in our handling of constrained resources: 1000 2000 3000 java.lang.OutOfMemoryError: unable to create new native thread at java.lang.Thread.start0(Native Method) at java.lang.Thread.start(Thread.java:602) at java.util.concurrent.ThreadPoolExecutor.addIfUnderCorePoolSize(ThreadPoolExecutor.java:437) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:875) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:78) at Bug.realMain(Bug.java:22) at Bug.main(Bug.java:34) # # An unexpected error has been detected by Java Runtime Environment: # # java.lang.OutOfMemoryError: requested 6000 bytes for char in /BUILD_AREA/jdk6.0/hotspot/src/share/vm/utilities/hashtable.cpp. Out of swap space? # # Internal Error (414C4C4F434154494F4E0E494E4C494E450E4850500017 01), pid=21376, tid=2 # # Java VM: Java HotSpot(TM) Server VM (1.6.0-beta2-b75 mixed mode) # An error report file with more information is saved as hs_err_pid21376.log # # If you would like to submit a bug report, please visit: # http://java.sun.com/webapps/bugreport/crash.jsp # 1000 2000 3000 java.lang.OutOfMemoryError: unable to create new native thread at java.lang.Thread.start0(Native Method) at java.lang.Thread.start(Thread.java:602) at java.util.concurrent.ThreadPoolExecutor.addIfUnderCorePoolSize(ThreadPoolExecutor.java:437) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:875) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:78) at Bug.realMain(Bug.java:22) at Bug.main(Bug.java:37) Passed = 0, failed = 1 Exception in thread "main" java.lang.AssertionError: Some tests failed at Bug.main(Bug.java:39) (mb29450@suttles) ~/src/toy/6399443 $ Java HotSpot(TM) Server VM warning: Exception java.lang.OutOfMemoryError occurred dispatching signal SIGINT to handler- the VM may need to be forcibly terminated
16-03-2006