JDK-8093972 : OutOfMemoryError after pressing Ctrl+Alt+Del or screen lock while animating a canvas
  • Type: Bug
  • Component: javafx
  • Sub-Component: graphics
  • Affected Version: 7u6
  • Priority: P3
  • Status: Resolved
  • Resolution: Fixed
  • Submitted: 2012-07-10
  • Updated: 2015-06-12
  • Resolved: 2014-05-09
The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.
JDK 8
8u20Fixed
Related Reports
Relates :  
JDK-8093950 - [D3D] Re-enable DirectX 9Ex to avoid losing textures when surface lost
Relates :  
JDK-8102206 - JavaDocs for new Canvas node
Relates :  
JDK-8103438 - Canvas should clear its buffered commands on certain clearRect, fillRect calls
Relates :  
JDK-8092801 - [Canvas] Canvas buffers commands indefinitely when it isn't visible leading to Out of Memory Exception
Description
To reproduce this issue:
 - use the same sample test app as RT-23311 
 - run the app with java -client switch rather than the java -server switch
 - while the app is running press Ctrl+Alt+Del (which on win7 brings up a screen with a bunch of options).
 - press cancel at the win7 options screen to return to the windowing system.
 - app will have died with an OutOfMemoryException

The workaround documented in the sample test app (writing data to a writable image then putting the image to the canvas rather than directly setting the pixels in the canvas via a pixelwriter) for RT-23311 also solves this issue as well - though again I have no idea why.
Comments
Dup of 2 bugs seems best. In general, "Fixed" implied pushing a changeset that fixes the bug...
09-05-2014
Actually, no. I think if you remove the fix for RT-24903, then this case still wouldn't be fixed even if we use D3D9ex because part of the reason for the OOM is not being able to get the buffer to a rendering pass and part of it is not being able to do anything with the buffer once it gets there. So fixing this bug required both efforts... Dup of 2 bugs? Or fixed as the result of 2 efforts?
09-05-2014
Hmmm... Originally I thought that this was covering both of those bugs, but now I'm not sure. I marked it as resolved because all of our "Canvas not draining" issues were solved by 2 separate efforts, but this particular bug appears to be targeted directly at the screen-lock issues which are a D3D9ex issue, so perhaps its should be marked as a dup...
09-05-2014
So this should be resolved as duplicate, right?
09-05-2014
With the fixes for RT-36571 and RT-24903, this should no longer be an issue for any Canvas that is attached to a window that has been shown, whether it is iconified, part of a non-visible sub-tree, or even screen-locked out...
09-05-2014
"the application should not be doing any UI or rendering work as is standard for iconified or background applications." This is not always possible or practical - I'm painting to the Canvas in response to data collected in real time. I have no need to collect the data in any other way, and I can't buffer it indefinitely. I can't stop just because the app is iconified. When the UI is restore the Canvas must look as if I haven't skipped any data. If such an optimization is implemented within JavaFX, I would want a means to opt-out. I realize there may be other ways to implement such an app (e.g. I may be able to render to a WritableImage), but I don't think it is reasonable for Canvas to leak in this case and place the burden of withholding commands in this case on the application. The leak prevention should happen behind the scenes in the Canvas implementation. Also don't forget to test it with JFXPanel. I was hoping to ditch the Swing side of my app soon, but I still need it to get native window handles via JAWT for rendering my real-time data.. Canvas was my hope to save me from that, but it leaks and it is significantly slower. (I have a Native ByteBuffer of raw image data that I need to get to the screen, and grabbing a window handle with JAWT and blitting it via a JNI call is many times faster than writing that same ByteBuffer to a Canvas.)
12-09-2012
I agree with Jim that these are separate (but related) cases. Btw, a separate bug is already filed for the case of a non-visible node: RT-24903 (I just linked that issue as "related to" this issue).
12-09-2012
If the Canvas node is in an active scene, but just not currently visible, then we should still be flushing the render buffer on every frame, but it would seem that visibility optimizations avoid visiting the Canvas during the synchronization step to trigger its flush. We may need a special flag to request that the synchronization always deep dives to a Canvas node, or trigger the flush off of a global mechanism that doesn't depend on tree culling. This is not the same condition as the original bug report since in the original bug report we aren't processing any frames at all and the application should not be doing any UI or rendering work as is standard for iconified or background applications. In the case of a non-visible Canvas in an active scene, on the other hand, it is appropriate for the application to update its nodes. A separate bug should be filed for that case.
12-09-2012
This is a significant bug. Simply having a Canvas in the non-visible Tab of a TabPane or scrolled out of view in a ScrollPane will cause the out-of-memory exception. The idea of fixing this in the "docs" is ridiculous. The implementation is severely flawed and must be fixed.
11-09-2012
This is an inherent limitation of Canvas, which uses a buffer of graphics commands which are processed when the Canvas is rendered each frame. If the buffer is not drained, it will eventually cause an OOM. The javadocs will be updated to indicate the conditions under which the buffer of graphics commands will be drained, which is basically: 1) The Canvas is attached to a Scene 2) The Scene is attached to a Window that is showing 3) The Window is not iconified 4) The System is not "screen locked" (e.g., Windows-L or CTRL-ALT-DEL) The first three conditions can easily be detected by the application. The latter cannot, so we will need to add an event to allow the application to know when the window is inactive. As a possibly workaround for the lack of a proper event for #4, you may be able to use a lost focus on the Window to indicate that you shouldn't keep drawing.
10-07-2012