JDK-8091155 : View.uploadPixels could take a source rectangle to upload only a portion of the pixels
  • Type: Enhancement
  • Component: javafx
  • Sub-Component: window-toolkit
  • Affected Version: 8
  • Priority: P3
  • Status: Open
  • Resolution: Unresolved
  • Submitted: 2012-08-10
  • Updated: 2018-09-05
The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.
Other
tbdUnresolved
Related Reports
Blocks :  
JDK-8091385 - PresentingPainter and UploadingPainter disregarding dirty clip rect
Description
There are two use cases for having a variant of View.uploadPixels that copies only a portion of the pixels in a Pixels buffer: The first one is necessary. The second would be a nice bonus.

1) In the case of dirtyopts we know that we only need to update the union of all of the dirty regions. Currently we are ignoring this clipping information, but there is a JIRA filed to optimize this case, RT-20356. Implementing that JIRA for the software pipeline or transparent windows case depends on this issue.

2) For the J2D and SW pipelines, we allocate a buffer of pixels to upload to HW whenever the window changes size. As a potential optimization we could avoid this allocation if the window was resized smaller, and only allocate a new buffer when it was resized larger.
Comments
Not part of the FX 8 PRD, consider for future release
23-04-2013
@David: yes, this might work. However, this would require the calling code to always provide the full texture for every tiny update. As Jim points out, this solution may not be as effective as we would want. Also, we won't be able to process arbitrary combinations of x, yDst/Src by means of setting a clip to the destination surface only (e.g. x,yDst == 0 && x,ySrc >0). So it looks like we all agree on the isPartialUploadSupported() + 2 * uploadPixels() API as described in my comment from Aug, 31 2012 01:54 PM. Should we possibly extend the isPartialUploadSupported() to: boolean isPartialUploadSupported(boolean translatedSrc); ? So that the calling code could determine whether x,ySrc may be >0 ? In this case Glass could dynamically detect the presence of cairo 1.10 and use a more efficient implementation (provided higher level code is able to employ this API).
14-09-2012
If the sub-region call simply copies the entire buffer then you have a problem with how to implement multi-rectangle dirty regions. If the sub-region copying is truly supported, then the fastest way to update the screen is (probably, modulo overhead) to request an upload for each of the multiple dirty areas. But, if sub-region copying simply forwards to full-region copying then you've just multiplied the work it was doing by the number of rectangles that it tried to individually upload. So, while it might simplify the upload API to "just do the entire rectangle anyway", the caller really does need to know if the sub-region was being respected otherwise the caller would use a different tactic.
31-08-2012
cant you use the full image with: cairo_rectangle(cr, 0, 0, w, h); // set a sub region in the image cairo_clip(cr); before ? cairo_set_source_surface(context, cairo_surface, 0, 0); cairo_set_operator (context, CAIRO_OPERATOR_SOURCE); cairo_paint(context);
31-08-2012
This depends on whether Graphics are OK with this contract. If they want to create a new, small buffer based on the dirty area size and repaint just it, then I guess this won't work. Note that in case of GTK with cairo 1.9-, we still can do a partial update. It's just that we expect the pixels to contain the data at (0, 0) origin. With your proposal applied we'll force full updates on old GTK even though we could do partial updates.
31-08-2012
A thought: The biggest reason to support a partial upload is to reduce the copy time/bandwidth of the dirty area. While this may be less of an issue on desktop, I certainly have seen it affect performance in past projects on embedded. My understanding is that for all of our current use cases, we will have the full "image" already in pixels. So how about setting the "contract" that: public void View.uploadPixels(int xDst, yDst, w, h, xSrc, ySrc, Pixels pixels); will copy the sub region *If it can*, else it will copy the full pixels provided. This will also allow for a short cut on some platforms (or just as we are implementing all 6 glass impls) to just forward this call to public void View.uploadPixels(Pixels); This way, there is no need to throw an exception, and no need to tinker with the behavior in quantum to conditionalize the behavior.
31-08-2012
Regarding the potential benefits of this improvement: uploadPixels() is used for software Prism pipeline (either j2d or sw), and also for transparent windows on MS Windows and Linux regardless of the rendering pipeline used. Note that MS Windows only supports full window updates for transparent windows. I.e. the isPartialUploadSupported() will always return false for views in transparent windows on MS Windows. While improving performance for software rendering path is important and certainly worth doing for Lombard at some point, I don't consider this issue urgent right now because it won't improve performance where it's most required (non-opaque windows on Windows). The rest of uses cases either aren't critical (when using software rendering), or won't be affected at all by this fix (e.g. when using an accelerated Prism implementation).
31-08-2012
The proposed Glass API is as follows: ***************************************************************************************** // Returns true if this View supports partial pixel uploads. // The result may change during view lifecycle public boolean View.isPartialUploadSupported(); // Performs partial pixels upload (if supported) // from pixels[xSrc, ySrc, w, h] to view[xDst, yDst, w, h]. // If xSrc, ySrc != 0, throws an exception (at least on GTK, see below) // If the operation is not supported for this view, throws an exception. public void View.uploadPixels(int xDst, yDst, w, h, xSrc, ySrc, Pixels pixels); // Performs full view update - this is the current uploadPixels() method public void View.uploadPixels(Pixels); ***************************************************************************************** Regarding xSrc, ySrc != 0: we could support it on Linux with cairo 1.10+ since it requires cairo_surface_create_for_rectangle(). Note that cairo 1.10 was released in Sep 2010. Ubuntu 10.04 which we claim to support has been released in Apr 2010 and contains older version of cairo. When/if we support Ubuntu 11.04+ (not sure actually, the exact version number needs checking), then we can support xSrc, ySrc != 0 on Gtk.
31-08-2012
With this fix we must also document that a Pixels object can be reused by means of updating the content of the buffer the Pixels object has been created initially with.
14-08-2012
That's right. Note that on MS Windows we use the ::UpdateLayeredWindow() for transparent windows only, and this function only allows one to update the entire window. However, for opaque windows (i.e. when j2d/sw pipelines are used) we use the ::SetDIBitsToDevice() function. And it looks like it is possible to perform a partial update in this case. So perhaps the isPartialUploadSupported() must be a function on Window class (or rather the View, since uploadPixels() itself belongs to views).
14-08-2012
Note that View.uploadPixels is used on all platforms when running the software pipelines (either prism-j2d or prism-sw).
13-08-2012
We'll still have to add an isPartialUploadSupported() method so that Quantum could decide whether it makes sense to request partial updates on a given platform.
13-08-2012
What about the following idea: 1. Introduce new View.uploadPixels(Pixels, Rectangle) method 2. Forward View.uploadPixels(Pixels) to View.uploadPixels(Pixels, whole View area) 3. On platforms that support partial uploading (DFB) implement the former method and take the given rectangle into consideration 4. On other platforms (Windows), implement the former method by skipping the given rectangle and uploading the whole Pixels array Of course, it will add one more method to Glass API, but I don't have any better ideas. No, that's not true: another idea is to have uploadPixels(Pixels, Rectangle) only and always pass the right rectangle from Quantum, even if it's equal to the whole View area
13-08-2012
View.uploadPixels() is also used in the embedded DirectFB port, which uses software rendering. Memory bandwidth is a real issue in ARM-based embedded devices, so this issue will be relevant there.
13-08-2012
+1. MS Windows API ::UpdateLayeredWindow() does not support partial pixel uploads. Only the whole window can be repainted at once.
13-08-2012
As a summary: this bug should be closed as not a defect (impossible to implement on Windows, makes no sense on Linux/GTK) + another bug should be filed to change GtkApplication.shouldUpdateWindow() to return false + yet another bug should be filed against Prism to make es2 work with PresentingPainter for translucent windows. Kevin, Chien, Morris, your comments?
13-08-2012
Right now View.uploadPixels() is only used on Windows and Linux/GTK. On Windows, uploadPixels() can only accept the full window image, so clips should be handled internally in Quantum, nothing can be done in Glass. GTK is different... I suspect that GtkApplication.shouldUpdateWindow() returns true only to workaround some bug in es2. Indeed, if shouldUpdateWindow() is changed to return false (which causes View.uploadPixels() to be skipped), translucent windows are drawn correctly with j2d, but not with es2.
13-08-2012