JDK-8217964 : [macos] Press&Hold macOS feature doesn't work when hold down an accent key using Robot
  • Type: Bug
  • Component: client-libs
  • Sub-Component: java.awt
  • Affected Version: 9,11,12,13,17,18
  • Priority: P3
  • Status: Open
  • Resolution: Unresolved
  • OS: os_x
  • CPU: x86
  • Submitted: 2019-01-18
  • Updated: 2021-09-02
The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.
Other
tbdUnresolved
Related Reports
Relates :  
Relates :  
Description
ADDITIONAL SYSTEM INFORMATION :
macOS >= 10.7
Oracle JDK build 11.0.2+7-LTS / HotSpot build 11.0.2+7-LTS

A DESCRIPTION OF THE PROBLEM :
Accents popup menu should appear instead of the key repeat when holding down an accent key with Java Robot if ApplePressAndHoldEnabled=1 (default value for macOS >= 10.7).

REGRESSION : Last worked in version 8u202

STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
1. Check that ApplePressAndHoldEnabled=1 using 'defaults read -g ApplePressAndHoldEnabled'
If ApplePressAndHoldEnabled=0 then set it to 1 using 'defaults write -g ApplePressAndHoldEnabled -boolean true’

2. Using several Robot keyPresses hold down a key that can be accented, for example, A for English keyboard layout:

for (int i = 0; i < 10; i++) {
    robot.keyPress(KeyEvent.VK_A);
}
robot.keyRelease(KeyEvent.VK_A);

Note: MacOS accessibility permission should be granted for the application launching the java test (IDE, Terminal), so Java Robot may access the keyboard.

Or just run the HoldDownAccentKey test (using Java or JTReg):

<JAVA_HOME>/bin/javac HoldDownAccentKey.java
<JAVA_HOME>/bin/java HoldDownAccentKey

or 

<JTREG_HOME>/bin/jtreg -testjdk:<JAVA_HOME> HoldDownAccentKey.java

EXPECTED VERSUS ACTUAL BEHAVIOR :
EXPECTED -
Accent menu popup appears, so user may select an accent

<jdk1.8.0_202.jdk>/Contents/Home/bin/java -showversion HoldDownAccentKey
java version "1.8.0_202"
Java(TM) SE Runtime Environment (build 1.8.0_202-b08)
Java HotSpot(TM) 64-Bit Server VM (build 25.202-b08, mixed mode)

ApplePressAndHoldEnabled = 1
TEST PASSED
ACTUAL -
A key is pressed several times

<jdk-11.0.2.jdk>/Contents/Home/bin/java -showversion HoldDownAccentKey

java version "11.0.2" 2018-10-16 LTS
Java(TM) SE Runtime Environment 18.9 (build 11.0.2+7-LTS)
Java HotSpot(TM) 64-Bit Server VM 18.9 (build 11.0.2+7-LTS, mixed mode)
ApplePressAndHoldEnabled = 1
Exception in thread "main" java.lang.RuntimeException: TEST FAILED: Holding a key down causes the key repeat instead of accent menu popup
	at HoldDownAccentKey.main(HoldDownAccentKey.java:144)

---------- BEGIN SOURCE ----------
import java.awt.AWTException;
import java.awt.Frame;
import java.awt.GraphicsEnvironment;
import java.awt.Robot;
import java.awt.TextField;
import java.awt.event.KeyEvent;
import java.awt.event.TextListener;
import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStreamReader;

/**
 * @test
 * @summary Regression test for: Press&Hold macOS feature doesn't work when holding down an accent key using Robot
 * @requires (jdk.version.major >= 8 & os.family == "mac")
 * @run main HoldDownAccentKey
 */

/*
 * Description: Tests that accents popup menu appears instead of the key repeat
 * when holding down an accent key with Java Robot if ApplePressAndHoldEnabled=1.
 *
 * Note: Test works with English keyboard layout.
 * Test requires macOS system property ApplePressAndHoldEnabled=1 (default value for macOS >= 10.7).
 * MacOS accessibility permission should also be granted for the application launching this test, so
 * Java Robot is able to access keyboard (use System Preferences -> Security&Privacy -> Privacy tab -> Accessibility).
 */

public class HoldDownAccentKey {

    private static final int SAMPLE_KEY = KeyEvent.VK_A;
    private static final int REPEAT_NUM = 10;

    private static final String SAMPLE_RESULT="à";
    private static final String SAMPLE_REPEAT = "aaaaaaaaaa1";

    private static final int PAUSE = 2000;

    private static volatile String result="";

    /*
     * Returns macOS major and minor version as an integer
     */
    private static int getMajorMinorMacOsVersion() {
        int version = 0;
        String versionProp = System.getProperty("os.version");
        if (versionProp != null && !versionProp.isEmpty()) {
            String[] versionComponents = versionProp.split("\\.");
            String majorMinor =  versionComponents[0];
            if (versionComponents.length > 1) {
                majorMinor += versionComponents[1];
            }
            try {
                version = Integer.parseInt(majorMinor);
            } catch (NumberFormatException nfexception) {
                // Do nothing
            }
        }
        return version;
    }

    /*
     * Returns ApplePressAndHoldEnabled system property value
     */
    private static String getApplePressAndHoldValue() throws IOException, InterruptedException {
        Process readDefaults = new ProcessBuilder("defaults", "read", "-g", "ApplePressAndHoldEnabled")
                .redirectError(ProcessBuilder.Redirect.INHERIT).start();
        readDefaults.waitFor();

        try (BufferedReader reader = new BufferedReader(new InputStreamReader(readDefaults.getInputStream()))) {
            return reader.readLine();
        }
    }

    /*
     * Checks that accents popup menu appears instead of the key repeat
     * when holding down an accent key with Java Robot if ApplePressAndHoldEnabled=1
     */
    public static void main(String[] args) throws AWTException, InterruptedException, IOException {
        if (GraphicsEnvironment.isHeadless()) {
            throw new RuntimeException("ERROR: Cannot execute the test in headless environment");
        }

        final int osVersion = getMajorMinorMacOsVersion();
        if (osVersion == 0) {
            throw new RuntimeException("ERROR: Cannot determine MacOS version");
        } else if (osVersion < 107) {
            System.out.println("TEST SKIPPED: No Press&Hold feature for Snow Leopard or lower MacOS version");
            return;
        }

        final String applePressAndHoldValue = getApplePressAndHoldValue();
        System.out.println("ApplePressAndHoldEnabled = " + applePressAndHoldValue);
        if (!"1".equals(applePressAndHoldValue)) {
            throw new RuntimeException("TEST ERROR: ApplePressAndHoldEnabled system property must be set to 1");
        }

        final Frame frame = new Frame("Test Frame");
        final TextField textField = new TextField();
        final TextListener textListener = (e -> result = textField.getText());

        try {
            textField.addTextListener(textListener);
            frame.add(textField);
            frame.setSize(400, 200);
            frame.setLocation(100, 100);
            frame.setVisible(true);

            Robot robot = new Robot();
            robot.setAutoDelay(50);
            robot.waitForIdle();

            // Hold down sample key so accents popup menu may appear
            for (int i = 0; i < REPEAT_NUM; i++) {
                robot.keyPress(SAMPLE_KEY);
            }
            robot.keyRelease(SAMPLE_KEY);

            // Select the first accent
            robot.keyPress(KeyEvent.VK_1);
            robot.keyRelease(KeyEvent.VK_1);

            Thread.sleep(PAUSE);
            robot.waitForIdle();

            if (SAMPLE_RESULT.equals(result)) {
                System.out.println("TEST PASSED");
            } else if (SAMPLE_REPEAT.equals(result)) {
                throw new RuntimeException("TEST FAILED: Holding a key down " +
                        "causes the key repeat instead of accent menu popup");
            } else {
                throw new RuntimeException("TEST ERROR: Unexpected input value: " + result);
            }

        } finally {
            textField.removeTextListener(textListener);
            frame.dispose();
            /* Waiting for EDT auto-shutdown */
            Thread.sleep(PAUSE);
        }
    }
}
---------- END SOURCE ----------

FREQUENCY : always



Comments
Feedback are accessible to the submitter only, so the link above will not be visible to everybody. I suggest to leave this bug open to track the progress if any.
30-08-2021

Seems to be a bug with Apple and not JDK
19-08-2021

Since I couldn't find the bug in the link in Sergey's comment, I have filed another bug report to Apple, with a short program that reproduces the bug. FB9537504 (Core Graphics CGEventCreateKeyboardEvent and CGEventPost not bringing up accent menu when ApplePressAndHoldEnabled is set to true) Basic information Please provide a descriptive title for your feedback: Core Graphics CGEventCreateKeyboardEvent and CGEventPost not bringing up accent menu when ApplePressAndHoldEnabled is set to true Which area are you seeing an issue with? ‌Something else not on this list What type of feedback are you reporting? Incorrect/Unexpected Behavior Description Please describe the issue: Using CGEventCreateKeyboardEvent and CGEventPost to mimic a held keypress does not mirror the behavior of an actual held keypress when ApplePressAndHoldEnabled is set to true. When the synthetic key is pressed and held, the accent menu will never come up. Please list the steps you took to reproduce the issue: Set ApplePressAndHoldEnabled to true using the command "defaults write -g ApplePressAndHoldEnabled -bool true". Compile and run sample test provided. Wait a couple seconds. What did you expect to happen? Expected behavior is that an accent menu will show up after a few seconds of holding down the keypress using CGEventPost. In the sample test, after "a" is posted, an accent menu should show up after a couple of seconds. What actually happened? The key that was pressed shows up, but the accent menu never shows up. "a" is posted and the program exits without the accent menu showing up. https://feedbackassistant.apple.com/feedback/9537504
19-08-2021

The bug for delays was fixed by the https://bugs.openjdk.java.net/browse/JDK-8242174 Two separate bugs were filed to apple: ======================= FB6798484: Documentation about CGEventCreateKeyboardEvent is incomplete Basic Information Please provide a descriptive title for your feedback: Documentation about CGEventCreateKeyboardEvent is incomplete Which area are you seeing an issue with? What type of issue are you reporting? Incorrect/Unexpected Behavior Description Please describe the issue and what steps we can take to reproduce it: The code below does not work by default(shift+z): CGEventRef event1, event2, event3, event4; event1 = CGEventCreateKeyboardEvent (NULL, (CGKeyCode)56, true); event2 = CGEventCreateKeyboardEvent (NULL, (CGKeyCode)6, true); event3 = CGEventCreateKeyboardEvent (NULL, (CGKeyCode)6, false); event4 = CGEventCreateKeyboardEvent (NULL, (CGKeyCode)56, false); CGEventPost(kCGHIDEventTap, event1); CGEventPost(kCGHIDEventTap, event2); CGEventPost(kCGHIDEventTap, event3); CGEventPost(kCGHIDEventTap, event4); It will type the lowcase "z" instead of uppercase "Z", as specified in the documentation: https://developer.apple.com/documentation/coregraphics/1456564-cgeventcreatekeyboardevent?language=objc According to the answer at https://feedbackassistant.apple.com/feedback/5415543 some delay should be added in between. Please update an example in the documentation. ======================= FB6798409: It is not possible to emulate human behaviour using CGEventPost and CGEventCreateKeyboardEvent Basic Information Please provide a descriptive title for your feedback: It is not possible to emulate human behaviour using CGEventPost and CGEventCreateKeyboardEvent Which area are you seeing an issue with? What type of issue are you reporting? Incorrect/Unexpected Behavior Description Please describe the issue and what steps we can take to reproduce it: Summary: The application and frameworks which tries to emulate human behavior cannot do that using CGEventPost and CGEventCreateKeyboardEvent. When the "ApplePressAndHoldEnabled" option is enabled it is expected that emulation of press+hold the key will show "accents choices" window. But it always repeats the key. This bug, for example, was found in the OpenJDK: https://bugs.openjdk.java.net/browse/JDK-8217964 SB Jul 23, 2019 at 3:44 PM See some details at https://feedbackassistant.apple.com/feedback/5415543
17-08-2021

Here is the report full log of that bugs: ======================= FB5415543: It is not possible to emulate human behaviour using CGEventPost and CGEventCreateKeyboardEvent 47839781 Basic Information Please provide a descriptive title for your feedback It is not possible to emulate human behaviour using CGEventPost and CGEventCreateKeyboardEvent Which product are you seeing an issue with? macOS + SDK Which area are you seeing an issue with? Description Please describe the issue and what steps we can take to reproduce it Area: Keyboards, mice and trackpads Summary: The application and frameworks which tries to emulate the human bihaviour cannot do that using CGEventPost and CGEventCreateKeyboardEvent. There are two main issues: - When the "ApplePressAndHoldEnabled" option is enabled it is expected that emulation of press+hold the key will show "accents choices" window. But it always repeat the key. This bug for example was found in the OpenJDK: https://bugs.openjdk.java.net/browse/JDK-8217964 - It is not possible to emulate the next sequence of press/release(shift+z): CGEventRef event1, event2, event3, event4; event1 = CGEventCreateKeyboardEvent (NULL, (CGKeyCode)56, true); event2 = CGEventCreateKeyboardEvent (NULL, (CGKeyCode)6, true); event3 = CGEventCreateKeyboardEvent (NULL, (CGKeyCode)6, false); event4 = CGEventCreateKeyboardEvent (NULL, (CGKeyCode)56, false); CGEventPost(kCGHIDEventTap, event1); CGEventPost(kCGHIDEventTap, event2); CGEventPost(kCGHIDEventTap, event3); CGEventPost(kCGHIDEventTap, event4); It will type the lowcase "z" instead of uppercase "Z", as specified in the documentation: https://developer.apple.com/documentation/coregraphics/1456564-cgeventcreatekeyboardevent?language=objc Configuration: macOS 10.13 or 10.14 Apple Feb 5, 2019 at 6:36 PM This feedback was created from radar ID 47839781 SB Feb 5, 2019 at 6:40 PM Same bug was found in "Xamarin": https://forums.xamarin.com/discussion/126313/simulate-sending-modifier-key-in-macos Apple Feb 25, 2019 at 4:20 PM Engineering has the following feedback for you: It’s possible, but you have to post and give time for the event source to update to match state. Try put some delays between each CGEventPost. ...and the creation of subsequent events needs to be after those postings - the events inherit their flags at creation time. Apple Closing per lack of response. SB May 16, 2019 at 3:27 AM It seems that the answer above is only about the second issue, this answer just prove that an example on the link below is incomplete: https://developer.apple.com/documentation/coregraphics/1456564-cgeventcreatekeyboardevent?language=objc But the first more important thing about "ApplePressAndHoldEnabled" was not answered. Apple May 31, 2019 at 8:20 PM For any remaining issues on the latest releases or betas, please file a new report.
17-08-2021

I asked a contact at Apple abiot this 2019 radr report and the response is ---------------------------------------------------------------- It was closed as behaving correctly back in 2019 after multiple attempts to contact the originator. """ It’s possible, but you have to post and give time for the event source to update to match state. Try put some delays between each CGEventPost. And the creation of subsequent events needs to be after those postings - the events inherit their flags at creation time. """ ---------------------------------------------------------------- So we need to look at this again from our end with this new info.
17-08-2021

It looks like a defect in the CGEventPost+CGEventCreateKeyboardEvent, I'll file a follow bug to the Apple(47839781).
06-02-2019

The root cause is change in: http://hg.openjdk.java.net/jdk9/jdk9/jdk/rev/16a8f15abd96 AXUIElementPostKeyboardEvent is replaced by CGEventCreateKeyboardEvent+CGEventPost
01-02-2019

Reported as a regression with JDK 11.0.2, Press&Hold macOS feature fails to work when holding down an accent key using Robot. Checked this for reported version with macOS 10.13.6 and confirm the issue. Results: ========== 8u201: Pass 8u202: Pass 9: Fail 10.0.2: Fail 11: Fail 11.0.2: Fail 12 ea b28: Fail To verify, run the attached test case with respective JDK versions. 8u201: ============= $ java -showversion HoldDownAccentKey java version "1.8.0_201" Java(TM) SE Runtime Environment (build 1.8.0_201-b09) Java HotSpot(TM) 64-Bit Server VM (build 25.201-b09, mixed mode) ApplePressAndHoldEnabled = 1 TEST PASSED 12 ea b28: ============= $ java -showversion HoldDownAccentKey openjdk version "12-ea" 2019-03-19 OpenJDK Runtime Environment (build 12-ea+28) OpenJDK 64-Bit Server VM (build 12-ea+28, mixed mode, sharing) ApplePressAndHoldEnabled = 1 Exception in thread "main" java.lang.RuntimeException: TEST FAILED: Holding a key down causes the key repeat instead of accent menu popup at HoldDownAccentKey.main(HoldDownAccentKey.java:145)
29-01-2019