JDK-8160893 : [macosx] JMenuItems in JPopupMenu are not accessible
  • Type: Bug
  • Component: client-libs
  • Affected Version: 9
  • Priority: P3
  • Status: Resolved
  • Resolution: Fixed
  • OS: os_x
  • Submitted: 2016-07-06
  • Updated: 2018-08-11
  • Resolved: 2016-10-07
The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.
JDK 8 JDK 9
8u152Fixed 9 b143Fixed
Related Reports
Duplicate :  
Relates :  
Relates :  
Description
Run the test listed below with either "Case 1" or "Case 2" commented.

"Case 1", JComboBox, pops up a list which is normally accessible with VO by navigating it with arrow keys. ���Case 2���, JPopupMenu, pops up the same list, but it is not accessible with arrow keys. Though, VO+"arrow key" seems to work for it.

The problem is likely the same for Swing menus which behave like ���Case 2���. Reproducible with SwingSet2.

(The difference b/w popups is that one of it is a non-focusable window, whereas the second one is a focusable window.)

public class Test {

    public static void main(String[] args) {
        EventQueue.invokeLater(() -> gui());
    }

    private static void gui() {
        JFrame f = new JFrame("frame");

        String[] items = { "One", "Two", "Three", "Four", "Five" };

        // case 1
        JComboBox<String> cb = new JComboBox<>(items);
        f.add(cb);

        // case 2
//        JList<String> l = new JList<>(items);
//        JPopupMenu m = new JPopupMenu();
//        m.add(l);
//        JButton b = new JButton("button");
//        b.addActionListener(e -> m.show(b, 0, 0));
//        f.add(b);

        f.pack();
        f.setLocationRelativeTo(null);
        f.setVisible(true);
    }
}
Comments
Re noreg-other, manual test is as follows: <SQE> - start VoiceOver - start Test1 (attached) - press space to activate the button and open the menu - press up/down arrow to select menu items; VO should announce changes - press space to choose an item; the button text should be updated - do the above three again but this time use enter to select an item - do those above three again but this time press Esc; the menu should close and the button text should not change - press control option space to open the menu - press control option up/down arrow to select items; VO should announce the changes Note: the blue background highlight will not update - press control option space to choose and item; the menu should close and the button text should update. </SQE>
26-09-2016

Swing���s JPopupMenu/MenuItem works OK; shift F10
23-09-2016

New webrev: http://cr.openjdk.java.net/~ptbrunet/JDK-8160893/webrev.01/ This one filters out side effects to combo boxes. Combo boxes need work but no sense having this fix complicate things for now.
22-09-2016

When using VoiceOver control+option+up/down arrow the user can hear the selection change but the blue background highlight is not visible. I investigated using the AccessibleSelection interface to select the popup menu's menu items as the selection changes. However AccessibleJPopupMenu does not implement the AccessibleSelection interface. Implementing that and using it from the Mac a11y code, i.e. sun.lwawt.macosx.CAccessibility /might/ solve the problem (but may not). This is a low priority issue so won't be fixed at this time.
20-09-2016

http://cr.openjdk.java.net/~ptbrunet/JDK-8160893/webrev.00/ This will fix the case of JPopupMenu/JMenuItem and JPopupMenu/Action but will not fix the case of JPopupMenu/JList.
14-09-2016

Suspending work for now. Summary follows. Last week Patti Hoa at Apple indicated that the code needs to fire kAXMenuOpenedNotification, kAXMenuClosedNotification, and kAXMenuItemSelectedNotification. Those have to be cast to NSString*, e.g. NSAccessibilityPostNotification(self, (NSString *)kAXMenuOpenedNotification); where self is the menu. This is the current Java hierarchy: frame window root pane layered pane panel popup menu list text This currently maps to the following as seen with the Accessibility Inspector AXApp AXWindow AXPopupButton AXList AXStaticText This is a native popup menu hierarchy: AXApp AXWindow AXPopupButton AXMenu AXMenuItem May need to remap popup/list/text to popup/menu/menuitem though no need to try that until it is seen how VO behaves with the above event changes. There is a problem firing the visibility event, i.e. kAXMenuOpenedNotification. The problem is that event happens before focus. The focus event causes construction of the Mac a11y parts (Java and native) needed prior to being able to handle the visibility event. As a result when the popup menu visibility event is fired there is not yet a handler to catch it. Although it makes sense that you'd want to fire visibility events prior to focus events, with the current design that doesn't work. Here is some background information: Current focus sequence: When CAccessibility is constructed it adds a property change listener to the keyboard focus manager. That handler posts focus to Mac. Then VO calls in to JavaComponentAccessibility.accessibilityFocusedUIElement. The call sequence is JavaComponentAccessibility.accessibilityFocusedUIElement upcall CAccessibility.getFocusOwner CAccessible.getCAccessible // creates the CAccessible Accessible.addNotificationListeners // adds a property change listener to AccessibleJPopupMenu [JavaComponentAccessibility createWithAccessible:...] // creates the native code, a JavaComponentAccessibility Something similar needs to happen for dealing with visibility events but those mechanisms don't exist. It appears that the only option is to have the focus event happen first, but it���s not clear how to make that happen. I tried adding RequestFocusInWindow and GrabFocus in the show method prior to setting visibility but nothing changed. I also tried requestFocusInWindow but that returned false. I tried grabFocus followed by a Thread.sleep for 5 seconds but that didn't change anything. This was at the top of JPopupMenu.show and prior to the call to setVisible(true).
19-08-2016

Opened 27747026 at bugreport.apple.com with this information: I'm trying to get VO to speak selection changes in a popup list. According to Accessibility Inspector the tree looks like this: AXApplication AXWindow:AXSystemDialog AXPopupMenu AXList AXStaticTest This is generated from a Java JList attached to a JPopupMenu which is shown when a JButton is activated. The list has 5 items, "one" ... "five". When the list appears VO does announce it: "system dialog, interact with popup button, 5 items, one". But when pressing up/down arrow VO doesn't announce the selection changes. Looking at the VO calls that come back in after a posting a NSAccessibilitySelectedChildrenChangedNotification for the list I only see - role called once on list, returns list - enabled called on label, only when moving from one to two, returns true - isIgnored called once on list, returns 0 accessibilityAttributeNames for the list are: AXParent, AXRole, AXRoleDescription, AXHelp, AXWindow, AXTopLevelUIElement, AXEnabled, AXSize, AXPosition, AXFocused, AXChildren, AXSelectedChildren, AXVisibleChildren accessibilityAttributeNames for the labels are: AXParent, AXRole, AXRoleDescription, AXHelp, AXWindow, AXTopLevelUIElement, AXEnabled, AXSize, AXPosition, AXFocused, AXTitle, AXValue, AXSelectedText, AXSelectedTextRange, AXNumberOfCharacters, AXVisibleCharacterRange, AXInsertionPointLineNumber, AXSelected, AXIndex I tried posting a NSAccessibilityLayoutChangedNotification on the list (with list in the userInfo dictionary) after the list became visible with no luck.
08-08-2016

Findings: Case 1 (updated, first test did not use the proper build) VO says to open list with CO space but that doesn���t work That doesn���t work for SwingSet2 combos either VO doesn���t say that prompt for SwingSet2 combos Typing text changes value but VO says nothing What works: down arrow opens the list, Shift CO downarrow to interact with list, CO up/down to change focus to new list item, CO space to select item Note: CO up/down doesn���t change highlight - but that���s how JLists work too Questions: - For case 1 VO says to use CO space but that doesn���t happen with SwingSet2. Why the difference? Case 1 fixes needed: - Open list with CO space? - Speak new value when value changes due to typing - With SwingSet2 and Case 1 often using the letter to switch selection doesn���t work Case 2 When JPopupMenu appears can arrow through it but VO says nothing when using up/down arrow CO up/down arrow can navigate but only to the first two items CO space will select an item (but only for the first two) Fixes needed: - VO should speak when using up/down arrow - CO up/down should be able to navigate past the first two items
22-07-2016

Ok, that list doesn't look critical. Being able to just navigate any lists with arrow keys would be great for a start...
22-07-2016

Thanks Anton, It turns out the build I tested with did not have 8145207 included. I updated the findings above. Can I assume the minor fixes for the JComboBox can be deferred for a while, i.e. P4 tbd_major - Open list with CO space - Speak new value when value changes due to typing - Often using the letter to switch selection doesn���t work (keyboard issue, not a11y issue)
21-07-2016

Pete, sure you can split it as you find it convenient (just add me to the watchers, please). I see from your comments that JComboBox didn't work for you. Did you test it with your latest fix 8145207? I was able to voice a combo with up/down arrows, with that fix... Ok, judging about priorities, if you can please fix JPopupMenu first. Thanks again!
21-07-2016

Anton, I am going to split this into two bugs, a new one for the JComboBox issues and this one for the JPopupMenu issues. Are the JComboBox fixes needed for your work? If so what is the priority?
20-07-2016

I've investigated it a little. When a list item is navigated, CAccessible.selectionChanged(ptr) is called for the list. This calls the following native stuff: postSelectionChanged -> NSAccessibilityPostNotification(self, NSAccessibilitySelectedChildrenChangedNotification); in response, Cocoa should call: [JavaComponentAccessibility accessibilitySelectedChildrenAttribute] with JAVA_AX_SELECTED_CHILDREN however, I don't see the response for the JPopupMenu list.
06-07-2016