JDK-6940127 : New Deployment mixed code security dialog can hang in some instances
  • Type: Bug
  • Component: deploy
  • Sub-Component: webstart
  • Affected Version: 6u19
  • Priority: P2
  • Status: Resolved
  • Resolution: Cannot Reproduce
  • OS: solaris_8
  • CPU: x86
  • Submitted: 2010-04-01
  • Updated: 2013-06-25
  • Resolved: 2012-12-06
The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.
JDK 7
7u40Resolved
Related Reports
Relates :  
JDK-7192121 - Enable native mixed code dialogs by default
Relates :  
JDK-6896918 - Mixed ceritification scenarios are making browser/VM to hang
Relates :  
JDK-6809570 - Certificate prompt totally lock up when a webstart apps connect via SSL
Relates :  
JDK-7003455 - Security Warning and "More information" dialogs hang
Description
FULL PRODUCT VERSION :
java version "1.6.0_19"
Java(TM) SE Runtime Environment (build 1.6.0_19-b04)
Java HotSpot(TM) 64-Bit Server VM (build 16.2-b04, mixed mode)

ADDITIONAL OS VERSION INFORMATION :
all operation systems with the latest java 6_19

A DESCRIPTION OF THE PROBLEM :
Our application loads unsigned classes from jars in a different thread then the awt thread. Then the new JavaWebstart Mixed code Security dialog wants to show itself but that hangs our complete setup

stackdump where you can see the 2 AWT event threads (AWT-EventQueue-0 and AWT-EventQueue-1) some how being blocked.
and the pool-1-thread-1 thread is just waiting for an answer for its dialog.
(attached seperatly)



STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
you could try our test server:

http://test-servoy-02.servoy.com:9090/servoy-client/servoy_client.jnlp



EXPECTED VERSUS ACTUAL BEHAVIOR :
EXPECTED -
That i can just click on No or Yes in that dialog
ACTUAL -
Can click on anything, the security dialog seems to be hidden by another modal dialog that isnt there.


REPRODUCIBILITY :
This bug can be reproduced always.

CUSTOMER SUBMITTED WORKAROUND :
The only work around is by going to the java control panel and disable the mixed code verification completely..

But this is not really a option for all our deployments of our customers.
So for now the only real work around is NOT to install Java 1.6_19

SUPPORT :
YES

Release Regression From : 6u18
The above release value was the last known release where this 
bug was not reproducible. Since then there has been a regression.
Comments
there's a major changes related to mixed-code dialog committed to 7u12 b04: https://jbs.oracle.com/bugs/browse/JDK-7192121 (Enable native mixed code dialogs by default) So, deploy doesn't use Swing's dialogs anymore and current native implementation shows mixed-code dialog on the current thread (not EDT) and this change should resolve this bug. Unfortunately I can't verify this fix using the initial test case http://test-servoy-02.servoy.com:9090/servoy-client/servoy_client.jnlp it's not loading to me now (is it still a valid test?). Closing it as not reproducible for now, please feel free to reopen the bug if any mixed-code test still fails/hangs with 7u12 b04 or later.
06-12-2012
EVALUATION We don't currently know how to fix this problem, it will require extensive changes and will require core changes in AWT. Too risky to fix for JDK7, perhaps it can be handled as an RFE for 8?
24-02-2011
EVALUATION > I used the new API from AWT to disable sequenced event when showing security dialog in our deployment code, and it fixed the hang issue: > > webrev: > http://web-east.east.sun.com/www/webrevs/ngthomas/6940127/webrev-6940127/ > > test: http://web-east.east.sun.com/www/tests/1.6.0_21/6940127/hello.jnlp > > I noticed a side effect though. When running the above test, with the > security dialog and application window both displayed, I can still click > on the buttons in the application window. It will have no effect while > the security dialog is still there, but the event will be processed > right after the security dialog is disposed. Is this expected ? Reply from AWT team: No. It's not expected, but I currently don't see a way to fix it. Here is what's going: 1. EDT1 (application's AppContext dispatch thread) spawns a new thread and waits for it to complete via join(). 2. On this new thread, a call to untrusted code is performed. 3. With the fix, sequenced events are temporarily disabled. 4. The mixed code dialog is shown on EDT0 (system AppContext). 5. If I click on blocked dialog now, mouse events will be placed into event queue, but *will not be dispatched* as EDT1 is blocked. 6. The dialog is hidden. 7. Sequenced events are re-enabled. 8. EDT1 is unblocked. 9. All the events in the queue are dispatched. When a mouse event is dispatched, *no mixed code dialog* is shown, so they are not filtered.
24-08-2010
SUGGESTED FIX deploy: http://web-east.east.sun.com/www/webrevs/ngthomas/6940127/webrev-6940127/ j2se: http://javaweb.sfbay.sun.com/~aa153232/6940127/
18-06-2010
EVALUATION another possible workaround in our code is to disable the processing of the SequencedEvents in the EventQueue when plugin/webstart shows security dialog on non-EDT thread. This requires a new private API from the AWT team. Also, we need to evaluate what are the side effect of doing this, if any. Since customer have workaround the issue in thier production application already by using the right JAR manifest to prevent mixed code dialog to show, we do not think this is a showstopper for 6u21.
18-05-2010
WORK AROUND provide appropriate mixed code attributes (trusted-library) in the JAR manifests to prevent mixed code dialog to show. http://java.sun.com/javase/6/docs/technotes/guides/jweb/mixed_code.html
18-05-2010
EVALUATION Evaluation from Igor: Problem: following app hangs with 6u19+ (100% reproducible at least on windows) http://demo.servoy.com/servoy-client/servoy_client.jnlp application code does something "potentially insecure" and we are trying to show modal warning dialog. However, buttons in the dialog can not be clicked. Stack trace attached too. My understanding of deadlock: 1) TaskExecutor[0] (application) thread performs suspicios action. This cause attempt to show warning. TaskExecutor is action is likely to be triggered by EventQueue-0 (we are not sure as we do not have source code). However, we know EventQueue-0 is blocked in the app code and will continue once TaskExecutor[0] is done (if deadlock resolved) 2) Security dialogs are shown in the secure context (app context and thread group), so TaskExecutor[0] call JavawsSysRun.delegate() that will wait for "Javaws Secure Thread" to show dialog. "Javaws secure thread" will open new modal dialog (default modality) in the secure context using EventQueue-1 for events and stop waiting for user decision 3) When mouse is clicked on button in the mixed code dialog event is delivered to EventQueue-1 and added to the end of the queue (this is proven by dumping queue content). However, it is never processed and this does not happen because thread is stuck in wait(). 4) Event is stuck because there is SequenceEvent earlier in the EventQueue-1 that can not be dispatched correctly. You can see this in the stack trace for the EventQueue-1, we stuck in the sequenceEvent.dispatch(). This SequenceEvent is indeed the event of grabbing focus from the frame and passing it to dialog. Unfortunately we can not get confirmation that focus was stolen (as EventQueue-0 has to process this and it is blocked) and therefore stuck. I.e. the deadlock is EventQueue-0 => (waits for) TaskExecutor[0] => Javaws Secure Thread => EventQueue-1 =>EventQueue-0 I am not sure how this can be resolved. The only idea i have at this point is to have mixed code dialogs to be native (or it will cause same problem with focus too?). Any other ideas? Can this be improved on AWT side? Can something be implemented in the deployment code to workaround this? One more data point - to prove the concept we added code to ignore all sequence event and hang is not reproducible. So, it seems theory could be correct. AWT team suggestion on fix: the problem described is exactly the same as described in 6809570: Certificate prompt totally lock up when a webstart apps connect via SSL To fix it correctly, we (AWT) should provide an API to start a new message pump on EDT, and then Plugin/WebStart to start using it when showing any dialogs (security or mixed code ones).
04-05-2010