JDK-8351907 : [XWayland] [OL10] Robot.mousePress() is delivered to wrong place
  • Type: Bug
  • Component: client-libs
  • Sub-Component: java.awt
  • Affected Version: 8,11,17,21,23,24,25
  • Priority: P2
  • Status: Resolved
  • Resolution: Fixed
  • OS: linux_oracle
  • Submitted: 2025-03-13
  • Updated: 2025-06-09
  • Resolved: 2025-05-21
The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.
JDK 11 JDK 17 JDK 21 JDK 25 JDK 8
11.0.29-oracleFixed 17.0.17-oracleFixed 21.0.9-oracleFixed 25 b24Fixed 8u471Fixed
Related Reports
Relates :  
JDK-8357584 - [XWayland] [OL10] Robot.mousePress() is delivered to wrong place
Relates :  
JDK-8358452 - JNI exception pending in Java_sun_awt_screencast_ScreencastHelper_remoteDesktopKeyImpl of screencast_pipewire.c:1214 (ID: 51119)
Relates :  
JDK-8357142 - [XWayland] add extra mouse buttons support for remote desktop portal
Relates :  
JDK-8347154 - [Wayland] Robot.mouseMove does not visually move mouse cursor
Relates :  
JDK-8357144 - [XWayland] improve token management
Description
Testsuite name: Regression
Product(s) tested: 24.0.1 b9
OS/architecture: Oracle Linux  (tested on oraclelinux-release-10.0-1.0.7.el10.x86_64)

On OEL 10 
a) Robot.mouseMove() event is occuring 
b)Visually, the mouse cursor move is not happening and 
c)Robot.mousePress() is also delivered to the wrong place (It occurs at the point at which the mouse is originally located before the robot attempts to move it).

The Mouse cursor visibility issue is seen on OEL 9.5 as well and there is already an open bug for that -JDK-8347154. But on OEL 9.5 the mousePress event occurs properly at correct place.


Reproducible: Always
Is it a Regression: No. Failure seen while OS baselining .
Is it reproducible on other trains: Tried it on JDK 21u, 17u, 11u , 8u and its reproducible.
Is it a platform specific issue: Yes, issue is seen only with OEL 10 platforms

Is it reproducible on JDK25: Yes tested on jdk-25-ea+9 its reproducible
Is it reproducible on OEL 9.5 : No
Comments
[jdk24u-fix-request] Approval Request from Aleksandr Zvegintsev This resolves the conformance issue, wherein java.awt.Robot did not function properly in the only available Wayland session on OL10.
21-05-2025
A pull request was submitted for review. Branch: master URL: https://git.openjdk.org/jdk24u/pull/223 Date: 2025-05-21 19:36:31 +0000
21-05-2025
Changeset: 2dfbf41d Branch: master Author: Alexander Zvegintsev <azvegint@openjdk.org> Date: 2025-05-21 17:21:05 +0000 URL: https://git.openjdk.org/jdk/commit/2dfbf41d2a3dbcd44f9ed9a58a1b0932d7536977
21-05-2025
A pull request was submitted for review. Branch: master URL: https://git.openjdk.org/jdk/pull/25265 Date: 2025-05-16 10:40:22 +0000
16-05-2025
Based on Oracle Linux PM, they expect to GA in June 2025. Even if this is extended to July, this would still be a GAed and supported platform for JDK 25 and if this issue is open, it will be a TCK-RED. If possible, I'd like to see this addressed earlier than to wait for the OS to go GA
08-04-2025
It looks like the Remote Desktop XDG portal is a possible solution for us. https://flatpak.github.io/xdg-desktop-portal/docs/doc-org.freedesktop.portal.RemoteDesktop.html It has sessions and restore_token like in a ScreenCast we are using for taking screenshots. At first glance, it has everything we need: NotifyPointerMotionAbsolute = mouseMove but it is absolute to a screen device(stream) / NotifyPointerButton = mouserPress/mouseRelease NotifyPointerAxisDiscrete = mouseWheel NotifyKeyboardKeycode = keyPress/keyRelease I did a quick test with a Python script (as it was the quickest way to test the API), it successfully moves the mouse, performs mouse clicks, and presses keyboard keys. So now I am working on the OpenJDK prototype. BTW At we might be able to drop ScreenCast in favor of Remote Desktop, since it also provides screen access via pipewire(need to double-check). ------- Another option is to file an external bug to add an option to disable the XTEST - EI connection on the system side, this should allow us to bring back the old behavior for machines where it is needed. However, it will be a long time before this issue is fixed and resolved in OEL10. https://mail.openjdk.org/pipermail/wakefield-dev/2025-March/000197.html > Olivier Fourdan > > > Or is there a way to disable this connection between XTEST and libEI? (as > > a temporary solution, so that the XTestFake* calls only affect the Xwayland > > without user prompt) > > > > Xwayland itself does not enable it by default and has a command line option > to explicitly enable the EI portal support (-enable-ei-portal) but with > Xwayland rootless, Xwayland is started by the Wayland compositor > automatically and GNOME Shell / mutter does not offer a way to customize > the command line so there is no way to disable it in the default Xwayland > rootless case. > > GNOME Shell mutter has options to disable selected X11 extensions so it is > possible to disable XTEST support entirely, but that would break your tests > completely. > > Easiest would be to add a way in mutter to optionally disable > „-enable-ei-portal“ when it starts Xwayland, but that's a bit of a hack, > not sure it would be acceptable for mutter, but we can try. https://mail.openjdk.org/pipermail/wakefield-dev/2025-March/000199.html > Olivier Fourdan: > > Anyway, if you want to have an option in mutter to not Xwayland with EI > support (the temporary workaround you mentioned), best would be to file an > issue against mutter upstream [1] as usual and we can work from there. > [1] https://gitlab.gnome.org/GNOME/mutter/-/issues/
18-03-2025
In our robot implementation, we emulate mouse/keyboard actions using the XTEST extension protocol. This issue is related to the libEI support is now wired to the XTEST, and we have two parts of the issue: 1. XTEST is now wired to libEI, it now asks for a user confirmation to allow interaction(see the attached confirmation.png), so we should now be able to click on native applications, window titles, etc. However, our robot implementation is mostly uses XTEST, but there is an exception: for mouse move we use XWarpPointer, which is not part of the XTEST, and it is not propagated outside of Xwayland server. So we have a situation where mouse is moving only inside Xwayland server, and we see the mouse enter/exit events, on mouse press we call XTestFakeButtonEvent and it is handled by libei, so the click happens where is the actual mouse cursor is in the system. That is why the test fails. To avoid this situation we should use XTestFakeMotionEvent for mouseMove in Xwayland case. I did a quick test with the XTestFakeMotionEvent, and it works with the provided test, and it allows to click on the window title as well as expected. I also attached a native reproducer to play with XWarpPointer vs XTestFakeMotionEvent behavior. 2. The second part of this issue is the user confirmation itself: It doesn't persist across reboots, it may timeout after a period of inactivity. It still seems to conform the spec, but we will not be able to run automated tests in this case. I am still researching what we can do about this, at the moment I do not have a solution for this part.
14-03-2025