JDK-6533175 : Pressing F10 when a modal dialog is visible hangs the Xserver, XToolkit
  • Type: Bug
  • Component: client-libs
  • Sub-Component: java.awt
  • Affected Version: 6,6u10
  • Priority: P2
  • Status: Closed
  • Resolution: Fixed
  • OS: solaris_2.5.1,solaris_10
  • CPU: x86,sparc
  • Submitted: 2007-03-10
  • Updated: 2011-03-07
  • Resolved: 2011-03-07
The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.
JDK 6 JDK 7
6u18Fixed 7 b14Fixed
Related Reports
Duplicate :  
JDK-6829862 - F10 pressed in JDialog produces NPE in EDT
Relates :  
JDK-6890456 - Introduce additonal check in XMenuBarPeer to block F10 key press for menus in blocked windows
Description
I have a frame with a menu added to the menubar. On clicking a button on the frame, I am opening a file dialog. When the file dialog is visible, I am pressing 'F10'. This has been dispatched to the frame and the menu comes up. But Xserver hangs after that and I have to kill the java process from another machine. It is incorrect that frame's menu responds to F10 when a modal dialog is visible. 

This is reproducible on SolSparc10-JDS as well as SuSE10 with JDK6 as well as JDK7-b09. I tried regressing this failure and found that the menu always responds to F10 on JDK5 but the Xserver started hanging only after JDK6-b40. Xserver behaves normally on b30. Lot of fixes have gone into b39 and b40 of JDK6. 

I have attached a sample test. Execute it and press the button. A file dialog would come up. Once the file dialog comes, press F10. Xserver will stop responding to user interactions.
Comments
EVALUATION F10 should not open menu in the parent Frame at all.
15-03-2007
EVALUATION The problem seems to be in the grab + modality. When a modal dialog is shown, we expect no blocked windows to receive focus and key events. However, F10 is handles separately, so it is retargeted from the dialog to its parent frame and a menu is shown. After this menu can't hidden as no ungrab event is generated (they all are blocked by the modal dialog), and at the same time all the events are grabbed by menu and the X server looks locked. To fix the problem we need to track how F10 is handled. After we find a frame with menubar that should receive this key event, we should first check if it is blocked and show menu only if it isn't.
13-03-2007