Bug ID: JDK-8162783 [Windows] HiDPI screenshot artifacts using glass Robot

Type: Bug
Component: javafx
Sub-Component: graphics
Affected Version: 9,10

Priority: P2
Status: Resolved
Resolution: Fixed

Submitted: 2016-07-29
Updated: 2018-03-01
Resolved: 2016-08-10

JDK 9
9Fixed

JDK9 b128 + Windows 10 + HiDPI screen (scale > 100% was used).


Please compile/run the attached test code (TestSnapshot.java + BMPOutputStream.java) using
  -XaddExports:javafx.graphics/com.sun.glass.ui=ALL-UNNAMED
and compare the following images (all of them are attached):

- utils.png (screenshot taken via SwingFXUtils) - looks reasonable
- glassRobot.bmp (screenshot taken by means of glass robot) - NOK (see the artifacts); please note also that HiDPI flag here set to true
- AWTRobot.bmp (screenshot taken by means of AWT robot) - NOK (the same)

the image on the screen looks normally.

When running the same code for scale = 100%, no issues occur (at least the screenshots look reasonable)

Please file a new bug for Mac. This bug was Windows specific (although the fix also refactored the Linux code, there should be no changes in the functionality for LInux). No Mac code was touched for this fix.
07-09-2016
Same things happen in 8 on MacOS (8u102, 8u112), screenshots attached
07-09-2016
it should not be forgotten from my side to check if the issue isn't reproducible on Linux as well. For now it is totally shadowed by GDK3 issues (JDK-8159892), at least for supported Ubuntu 16 Linux
16-08-2016
The target build for this fix is jdk-9+132
10-08-2016
Changeset: 89a5de54b7dc Author: kcr Date: 2016-08-10 09:44 -0700 URL: http://hg.openjdk.java.net/openjfx/9-dev/rt/rev/89a5de54b7dc 8162783: [Windows] HiDPI screenshot artifacts using glass Robot Reviewed-by: flar
10-08-2016
I filed JDK-8163795 to track the removal of the now-dead code.
10-08-2016
I think I've verified that it can't happen. Using y as an example... pminy = floor(y * uiScale); rely = (y + iy + 0.5) * uiScale - (pminy + 0.5); We know that (iy >= 0). The first part of that must therefore be larger than pminy, so the smallest that rely can be would be: rely = (number larger than pminy) - (pminy + 0.5) >= -0.5 irely = floor(rely); So, irely >= -1 If irely is -1, then the interp function works fine. A shorter proof is that all samples must come from within the buffer so the worst case is you are blending the left column (or top column) with the 0 sampled from just to the left of (or above) it. In that case, perhaps the first sample taken is out of bounds, but the next sample has to be within bounds. I'm happy with the fix, even if abused with anomalous LoDPI scale values...
09-08-2016
I tried various values with -Dglass.win.uiScale and did not see a problem, but this was just an empirical test. The interp method up-scaled or down-scaled the image correctly (as near as I could tell visually), and without an AIOOBE. I specifically tested the following: 1.01, 0.99, 0.51, 0.50, 0.49, 0.3. This is by no means exhaustive, nor did I analyze the code.
09-08-2016
I don't think any system would give us a value less than 1.0, but it could be manually overridden. Do we expect the robot to behave well (i.e. not throw exceptions) if they do so and then use it to grab a screen capture? The worst outcome would be an array index OOB on a Java array, but they are doing something pretty odd for a function primarily used for display testing.
09-08-2016
I'll file a bug to remove the dead code in the Windows native implementation. I didn't look closely at the implementation of the interp method, since it was just promoted from the subclass to the base class without modification. Do you think we should file a new bug for this, too? If it really can only happen when screen scale < 1.0 I I don't see it as an issue, since that should never happen (right?)
08-08-2016
Should there be a bug to remove the dead code in the Windows implementation? Also, I'm not sure if the relative coordinate in the interp method could ever be < -1, if it was then we don't protect against trying to load the pixel data for that coordinate +1 in the code that does bounds checking. I think this can only happen if the screen scale was <1.0, though, so do we care?
08-08-2016
Webrev: http://cr.openjdk.java.net/~kcr/8162783/webrev.00/ This fix changes the Windows implementation of Robot.getScreenCapture (in WinRobot) to match the Linux implementation (in GtkRobot). In order to not duplicate code, I refactored it, moving the implementation of _getScreenCapture(int, int, int, int, boolean) from the GtkRobot subclass to the Robot base class (it was abstract in the base class), along with the two "interp" utility classes. As part of this, I had to make _getScreenCapture(int, int, int, int, int[]) protected rather than private, along with adding a method in the base class. Finally, I changed the implementation of _getScreenCapture(int, int, int, int, int[]) in the native Windows code to not do scaling. A couple things to note: * In order to make the change less intrusive, I did not remove the code path in the native GetScreenCapture() method that calls StretchBlt to scale on copy if copying to a destination of a different size than the source, but I note that that path is no longer used. * I made the implementation of _getScreenCapture(int, int, int, int, int[]) in the base class a concrete implementation, that throws UnsupportedOperation, so as not to have to modify the Glass platforms that will never use it (e.g., MacRobot, IosRobot, MonocleRobot). I could change this if desired.
05-08-2016
There are two bugs in the Robot.getScreenCapture implementation on Windows: 1. The "isHiDPI" flag is ignored (treated as if it were false) 2. The Windows GDI StretchBlt method does a horrible job of down-scaling a region on the screen in the case where isHiDPI==false The Linux implementation in GtkRobot addresses both of these issues for Linux Hi-DPI; using that code for the Windows implementation will fix the problem. I have tested a prototype of this, and it fixes both modes of getScreenCapture.
05-08-2016
> a bad assumption in the test program reuploaded...
03-08-2016
Given that both AWT and Glass Robot have the same issue, this could point to a problem in the underlying Windows GDI StretchBlt method being used for this case (or in some state setting). I did a quick hack to prototype the isHiDPI mode of screenCapture in glass Robot, and the image looks good. Note that I had to fix a bad assumption in the test program, since it assumes that the width and height of the returned Pixels object is the same as is passed into the capture method, which is not the case when HiDPI == true (you need to use Pixels.getWidth() and getHeight() instead). We still will likely need to fix the non-HiDPI mode (or provide a workaround).
02-08-2016
One problem I see is that the isHiDPI flag in the screenCapture method is ignored on Windows.
02-08-2016
> what was the Windows scale factor 125%, 150%, 175% (the attached screenshots - 175%)
02-08-2016
@Alexander: what was the Windows scale factor when you captured the attached screen shots? 125%? 150%?
02-08-2016
AWT Robot and JavaFX Glass Robot are two different things, so a new client-libs bug should be filed for AWT Robot. We can leave this one for Glass Robot.
02-08-2016
probably this may be considered as a client-libs bug, as the AWT Robot seems to be affected as well
02-08-2016
Jemmy is taking the screenshot by the means of other tools, such as java.awt.Robot or the FxRobot. Which one is used in this case?
29-07-2016

Blocks :	JDK-8163795 - [Windows] Remove call to StretchBlt in native GetScreenCapture method
Relates :	JDK-8162959 - [HiDPI] screenshot artifacts using AWT Robot