JDK-8090159 : [Canvas] Canvas objects need to maintain a persistent pixel backing store
  • Type: Bug
  • Component: javafx
  • Sub-Component: graphics
  • Affected Version: 7u6
  • Priority: P4
  • Status: Open
  • Resolution: Unresolved
  • Submitted: 2014-04-01
  • Updated: 2018-09-05
The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.
Other
tbdUnresolved
Related Reports
Relates :  
JDK-8092128 - [Canvas] Implement paint/damage drawing model for FX Canvas
Relates :  
JDK-8095137 - [Canvas] Maximize Windows with Canvas as Drawing surface produce Runtime Errors
Relates :  
JDK-8088286 - Managed Resources should have a "save yourself" callback as an alternative to marking textures Permanent
Description
Because Canvas objects guarantee that rendering commands issued to them will be represented until the Canvas is dropped on the floor and garbage collected, they need to maintain some sort of permanent history of the results of that rendering.  Currently that is a pixel store.

In order to be able to manage those resources more effectively, we need to have a mechanism that is cooperative with the application so that they can restore the contents of a Canvas if we find need to release its pixel store.

This mechanism could be any of:

- An observable property that indicates when the pixels were destroyed.

- An event that can be handled and the painting reissued.

- A paint/update mechanism similar to Swing/AWT, but more limited in scope (it would just deal with repaints).
Comments
Some discussion occurred with respect to how to notify an application about loss of backing store for a non-persistent instance. In particular, whether or not to use an event for repainting (as opposed to subclassing and overriding a paint() method or using a painter attribute on Canvas). The following issues need to be considered for a solution that uses an event listener/handler as the painter: - Originally we thought of making this a one-shot "canvas becomes non-persistent and stays that way" in the constructor because there is no way for a node to detect the comings and goings of event handlers. While it may be possible to add such a mechanism, events tend to be "in case anyone is interested" and we actually care if someone is interested in just this one case (because if they aren't, then we have a problem and we need to worry about pixel persistence). - If we want to consider "persistent vs. non-persistent" on the fly, then we can't really use an event handler - we need something more positive and trackable than that (due to the issue I just described in the last bullet point). - Event listeners are not unique so with events we can end up with multiple painters on a canvas. This could even be unintentional if a paranoid programmer adds their painter on every frame. With a property or setter or constructor argument, we can ensure a single unique painter (and track it). - Even with the attempted "supply your event handler in the constructor" solution, a developer could come along and manually "removeHandler" the painter and then we would have a non-persistent canvas with nobody to paint it. All in all, I think it makes more sense to use a painter attribute (required to be non-null for non-persistent canvases) or even a subclass rather than a repaint event. I prefer an attribute to a subclass.
02-04-2014
A note about a mechanism to move the pixels out of vram doesn't solve the problem, it just moves one resource problem into another resource pool. The pixels must still be tracked. That might help partially since systems tend to be more limited in vram than they are in heap memory, but many applications can already redraw the entire canvas as needed, they just need a way to tell us that they can, and a simple way to discover when they need to - and then we don't need to save the pixels anywhere...
02-04-2014
Another thing to consider is the initialization of the GraphicsContext. Currently the GCtx starts out initialized to default values when the Canvas is created and maintains all changes even across calls to cv.getGCtx(). This matches the persistent state of the GCtx used in HTML5. Many damage repair models construct a brand new GCtx for every paint call (this is somewhat controllable in Windows where you can set attributes to control whether HDC's keep state between uses). The GCtx model we adopted from HTML5 also has save/restore that can be used to isolate state changes for some drawing operations, but it does not save/restore all state (notably the path is left intact across a save/restore boundary). Also, from the NeWS days of using the PostScript model for window repair, save/restore mismatches ended up causing problems. I seem to recall that one of the few items of input that James Gosling gave us when we created Java2D was not to use save/restore. ;) Given that we already have a persistent GCtx on a persistent Canvas model with the "evil" save/restore, when we introduce a non-persistent Canvas model we may want to revisit our GCtx persistence as well. My proposal would be that cv.getGCtx still returns a GCtx in the "last used" state, but that we add a "reset()" method to GCtx that resets it to all default values, erases the save/restore stack and clears the path and that can be used to start paint requests on an "even playing field".
02-04-2014
If we want to have persistence be dynamic, then we really must have a property or setter and we should probably use the following semantics: - if you add a painter to a canvas that had none, then at some point in the near future persistence will no longer be guaranteed and the lock needed on the resources to guarantee persistence will be relaxed. - if you remove the painter from a canvas (set it to null), then it would be forcibly cleared to transparent immediately and pixels rendered after that operation are guaranteed (this is to prevent a case where the old data might or might not be around by the time you render over it, we'll just guarantee that it will always be forced to transparent in that case)...
02-04-2014
After examining the problems that developers are currently experiencing with resource usage on Canvas, it was apparent that this particular issue is not rearing its head as much as the issue of not always flushing the command buffer of graphics commands as per RT-24903. It would be nice to have a mechanism by which we could relax the persistence of Canvas, and we do have an internal test app that can run into this problem, but in practice developers seem to be only running into our fragile buffer flushing limitations so far.
02-04-2014
So the idea behind this work is to keep the existing "draw anytime" model and provide and indication that the entire contents needs to be drawn? I think we need a concrete example that shows the problem that we are trying to solve. Let's attach one to this JIRA. All of the solutions discussed so far require the application to be recoded. It would be much nicer if we could have some other solution, perhaps copying the pixels out of VRAM (slow, bad) or something else.
02-04-2014
The old usage won't be deprecated in any formal sense, but adding a notification/callback to replace the pixels will open up new possibilities for programmers. For cases where they can replace the pixels on demand they can then use more Canvases more often. Until then, they have to worry about using too many Canvas objects just like they sort of have to worry about using too many Image objects right now (though Images can be managed a little better by us on their behalf because they are reconstructable). I don't have a handle on the extent of HTML5 Canvas drawing in HTML5 to compare the two. One thing that HTML5 has that we don't is an ability to "reload" the page to cause the HTML5 canvas commands to run again. They wouldn't want to do that for an active page, but they could do it for pages in the history. We don't really have a way to tell the user "uh oh, your canvas went out of scope for a while so you have to start over now" that a browser has by reloading the page. In the case of FX application life cycles, "draw once and we'll maintain it" is a permanent sentence. We have run into resource usage problems internally in the (unpublished) CanvasTest toys. The problems only arose when we started running them on retina displays which use 4x the memory of regular displays and it was quite a surprise that the use of Canvas in one of the toys was enough to deplete our VRAM.
01-04-2014
[deleted a comment meant for another bug]
01-04-2014
1) My understanding is that the previous Canvas drawing model where you draw any time you want will be discouraged/deprecated? If not, is the idea that in addition to application code that can draw anywhere and anytime, application programmers should also be able to draw the complete contents of their canvas on demand when asked by the system? 2) Can you provide some JIRA that will be fixed by this callback? FX Canvas is based on HTML5 canvas and HTML5 canvas does not seem to have a problem drawing. I realize that FX Canvas implementation and the browser implementations of HTML5 canvas are radically different.
01-04-2014
There is an existing Jira issue suggesting that we add the 3rd of the mechanisms described in the Description - RT-36394.
01-04-2014