Bug ID: JDK-8051843 Public API alternative for sun.misc.Cleaner

The topic of this issue is covered by the java.lang.ref.Cleaner and can be closed as a duplicate of 8138696.
04-12-2015
Summary of another week of discussions related to the latest patch. First of all, Andrew pointed out that GC & weak reference tracking can be source of very surprising behavior - I see that as important, but independent issue and I decided to track it separately as JDK-8055183. Roger pointed out (in http://mail.openjdk.java.net/pipermail/core-libs-dev/2014-August/028092.html) that the proposed solution is very close to already existing sun.misc.Cleaner. I believe this is a positive sign: - NetBeans needs activeReferenceQueue(), JDK needs Cleaner - there must be a need for this kind of lightweight finalization, then! - JDK9 promises to be more secure by disallowing access to sun.* packages. Together with such promise the team announced that some useful utilities will be turned into official API (sun.misc.Unsafe particularly). Looks like Cleaner is another useful tool as well - so let's treat my quest for lightweight finalization as an attempt to come up with an official API to replace Cleaner. Technically both activeReferenceQueue and Cleaner seem similar, except following differences: - Cleaner's run() method is called on internal JDK thread. If the client cleanup code misbehaves, no References will be queued into their ReferenceQueues. That is probably too fragile for public API. My patch which introduces Reference.activeQueue() is doing client code callbacks on the finalizer thread which already has all the sufferings with client code - e.g. no increased security/stability risks. In the process of seeking official API for lightweight finalization I suggest to use the finalizer thread for callbacks. - Cleaner creates own instance of PhantomReference while activeQueue can be used with any type of Reference (just choose your own and make it implement Runnable). For many usecases the activeQueue approach is more efficient. Often one needs WeakReference or SoftReference to the object and also the cleanup code - with Cleaner current design this requires two references - one client one and one internal PhantomReference in the Cleaner. activeQueue needs just the client one. Even if one does not need own reference, the activeQueue approach needs one object less - as the PhantomReference and Runnable are merged into one. - Cleaner keeps internal pointer to the created PhantomReference instance ensuring the cleanup code will always be executed. This is not the case with current implementation of activeQueue, client code is supposed to hold pointer to the Reference by itself. If the Reference is GCed sooner than its referent, no cleanup is performed. This may or may not be desirable. Possibly having an API alternative for both behaviors would be beneficial.
15-08-2014
Introducing new ReferenceQueue.removeWhileExists static method: https://bugs.openjdk.java.net/secure/attachment/21429/X.diff Moving internal data from ReferenceQueue into Lock object (so now the Lock class acts like a monitor). This allows us to keep the current logic for remove/poll/enqueue, without holding an internal reference to ReferenceQueue - as such it can be GCed, as the provided CanGCWhenRemoving test demonstrates.
06-08-2014
Alternative proposal which brings the functionality of "lightweight finalizer" into JDK itself: https://bugs.openjdk.java.net/secure/attachment/21640/ActiveQueueFinalizer.diff
06-08-2014
#4 is a somewhat overly concise statement so here is what I wrote in email: "Any time you choose to introduce a new thread to perform a task you have to be concerned about the lifetime of the task and thus the thread. Telling a thread when it should stop a repetitive task is one of the key design points of multi-threaded solutions. Automatic threading solutions work fine for one-shot tasks, from a lifecycle management perspective, but otherwise you either have to expose management methods, or else assume an environment when the thread's lifecycle is that of the "application". And as per the case at hand the latter is a problem when we have these environments that almost, but not quite, provide independent execution contexts." and "Equating lifecycle with "reachability" is convenient but it is inexact. Though you could argue that the problem here is an extension of the normal "cyclical garbage" problem - the GC can't know that the processing thread in this case is itself "garbage" because nothing can submit any more work for it to do. Maybe that is a solvable problem, but to me, here and now, responsibility for shutting down the thread lies with the subsystem that created it. " In short I don't accept "the need and right of library writers to write activeReferenceQueue-like API, without repeated polling and with proper cleanup code based on GC based-lifecycle management". GC is not the right solution for all resource management problems. In this case the issue is the thread lifecycle management which has been internalized and basically not managed - there is no way to shut it down. While I accept it may be too late to rewrite activeReferenceQueue in a way that makes the thread management explicit, that does not mean that a fix to this problem belongs as a part of the API of a general purpose facility. To again quote from email (which is in itself paraphrasing what many JDK library developers have said over the years): "To modify the existing library requires establishing that the new functionality is sufficiently general and "carries its weight" - otherwise everyone has an improvement that helps their particular use-case. "
01-08-2014
Summary of the 1st week of discussions on the mailing list where Brian Goetz, David Holmes, Peter Levart, and Florian Weimer participated in the discussion. In discussion of the need for activeReferenceQueue-like concept a historical NetBeans state was described: Many modules, each using its own ReferenceQueue with a dedicated thread waiting for it to clean its queued references up. Each thread occupying valuable system resources. The solution was to come up with centralized ReferenceQueue and a single thread to process the queued references and properly dispatch to each's reference cleanup code. Nobody questioned logic of this step. Next part of the discussion focused on explaining what happens when activeReferenceQueue starts to be used in a web application. Once the dedicated cleanup thread starts, it faces a tough question: when to stop? WAR development is based on frequent re-loading of whole WAR (basically a collection of libraries, including activeReferenceQueue API) and each reload loads new version of classes with new classloader. The old classes stop being referenced by the container and are eligible for garbage collection. However they don't get garbage collected: there is the cleanup thread blocked in ReferenceQueue.remove() holding reference to ReferenceQueue and never being woke up again keeping the whole content of the WAR file classes in memory. That is description of the current problem. Here are the discussed solution options: #1 - use reflection to get on hold of ReferenceQueue internals. This is our current state, but it got broken in JDK9 due to fixes which subtly changed the internal semantics. The reflection can be improved, but the whole purpose of this discussion is to avoid reflection in the future. #2 - use repeated polling. E.g. call remove(15 * 1000). NetBeans performance team hates threads that repeatedly wake up and wants us to avoid them. Plus, the problem is that the pointer to ReferenceQueue is still kept for 15s, and only for shorter amount of time it may not be: how to ask the VM for GC cycle? System.gc every 15s is not going to be effective. #3 - Peter Levart suggested to add a static method into ReferenceQueue http://mail.openjdk.java.net/pipermail/core-libs-dev/2014-July/028020.html which would allow to remove an object from a queue without holding strong reference to the queue. Peter's solution is similar to the initial one proposed here, yet he seems to create it independently. Hopefully a sign of something... #4 - David Holmes (and sort of Florian Weimer) would like to use automatic lifecycle management to manage the state of the thread. This is sort of hard to do in general purpose library: while there are ways to do so in NetBeans Platform, OSGi, web applications, each of them is different. The only lifecycle management available in pure Java is garbage collection: sort of inaccurate (clean up happens later, when GC kicks in), but better late then never. If we accept the need and right of library writers to write activeReferenceQueue-like API, without repeated polling and with proper cleanup code based on GC based-lifecycle management, then Peter's and mine suggestion to add single static method removing from a Reference<? extends ReferenceQueue> seems like gentle, safe and sufficient extension to existing APIs provided by JDK. PS: I am trying to avoid walking the #4 path: I am afraid starting a discussion about lifecycle management in Java SE would turn into massive task with enormous amount of contributions and I'd rather keep the change low and focused.
01-08-2014
A thread has been started on this topic here: http://mail.openjdk.java.net/pipermail/core-libs-dev/2014-July/027991.html
28-07-2014

Relates :	JDK-6853696 - (ref) ReferenceQueue.remove(timeout) may return null even if timeout has not expired
Relates :	JDK-8138696 - java.lang.ref.Cleaner - an alternative to finalization