JDK-4260109 : If we pass "-Djava.security.debug=all" in Plugin control panel, browser freezes
  • Type: Bug
  • Component: deploy
  • Sub-Component: plugin
  • Affected Version: 1.3.0
  • Priority: P4
  • Status: Closed
  • Resolution: Duplicate
  • OS: generic,linux
  • CPU: generic,x86
  • Submitted: 1999-08-05
  • Updated: 2001-03-14
  • Resolved: 2001-02-13
Related Reports
Duplicate :  
JDK-4380275 - Browser hangs when runtime parameters are specified in the Plug-in control panel
Relates :  
JDK-4316821 - 1.1.only TextArea Bug 4080391 should not have been closed : OS Win95, 98, NT40
Relates :  
JDK-4080391 - TextArea.setText() doesn't work for large Strings.
Description
When a user passes the "-Djava.security.debug=all" in the java runtime arguments filed of the Java_Plug-in Control panel and try to run an plugins demo applet then the browser feezes.

Configurations:
Platform : Windows95/98/NT
Browser's: Internet Explorer and also with Netscape.
kestrelbuild: With "M" build.

To reporduce :

Step1:
	Install the kestrel "M" build.

Step2:
 	Set the run time parameters to the following string in the 
	java_Plugin control panel.

	"-Djava.security.debug=All"

Step3:

	Now try to run an plugin demo applet on internet explorer or netscape.
	Then browser freezes.This is not suppose to happen.
Comments
SUGGESTED FIX Name: dsC76792 Date: 10/25/99 ###@###.### The first two problems can be fixed in AWT code: 1.Set text limit for textarea and textfield to the maximum possible value (0xFFFF on win9x and 0xFFFFFFFF on winNT). 2.Change AwtTextArea::CountNewLines to allocate memory in the heap instead of on the stack. Context diffs are as follows: --- awt_TextArea.cpp Fri Oct 22 19:59:00 1999 *************** *** 110,115 **** --- 110,120 ---- c->UpdateBackground(env, target); c->SendMessage(EM_SETMARGINS, EC_LEFTMARGIN | EC_RIGHTMARGIN, MAKELPARAM(1, 1)); + /* + * Fix for BugTraq Id 4260109. + * Set the text limit to the maximum. + */ + c->SendMessage(EM_LIMITTEXT); } } catch (...) { *************** *** 131,142 **** if (jStr == NULL) { return nNewlines; } ! WCHAR* string = TO_WSTRING(jStr); ! for (int i = 0; i < maxlen && i < env->GetStringLength(jStr); i++) { if (string[i] == L'\n') { nNewlines++; } } return nNewlines; } --- 136,156 ---- if (jStr == NULL) { return nNewlines; } ! /* ! * Fix for BugTraq Id 4260109. ! * Don't use TO_WSTRING since it allocates memory on the stack ! * causing stack overflow when the text is very long. ! */ ! int length = env->GetStringLength(jStr) + 1; ! WCHAR *string = new WCHAR[length]; ! env->GetStringRegion(jStr, 0, length - 1, string); ! string[length-1] = '\0'; ! for (int i = 0; i < maxlen && i < length - 1; i++) { if (string[i] == L'\n') { nNewlines++; } } + delete[] string; return nNewlines; } --- awt_TextField.cpp Fri Oct 22 20:01:12 1999 *************** *** 77,82 **** --- 77,87 ---- c->UpdateBackground(env, target); c->SendMessage(EM_SETMARGINS, EC_LEFTMARGIN | EC_RIGHTMARGIN, MAKELPARAM(1, 1)); + /* + * Fix for BugTraq Id 4260109. + * Set the text limit to the maximum. + */ + c->SendMessage(EM_LIMITTEXT); } } catch (...) { I think we cannot fix the third problem (deadlock) in AWT code, since -Djava.security.debug=all switch allows client code to be called on the toolkit thread - the case that we assume should never happen. So the one who writes the code that can be executed on the toolkit thread should take care that it won't deadlock. In this case i would suggest to modify the plugin code so that it creates an intermediate buffer to which all threads will dump their output inside a synchronized block and a helper thread that will stream the data from the buffer to the textarea outside the synchronized block. Also the console window should monitor the text length of the contained text area and before it exceeds some fixed limit it should wipe the required portion of text from the beginnning. It is required since the textarea content length is physically limited on Windows. I tried to implement this idea in the plugin code and it seems to fix the problem. Tested using IE 5 and Netscape 4.7 on WinNT and IE 5 on Win95. Tentative diffs: --- ext/plugin/java/src/sun/plugin/DebugOutputStream.java Mon Oct 25 17:47:13 1999 *************** *** 34,40 **** import java.io.*; ! public class DebugOutputStream extends OutputStream { private FileOutputStream traceFile; /** --- 34,40 ---- import java.io.*; ! public class DebugOutputStream extends ByteArrayOutputStream implements Runnable { private FileOutputStream traceFile; /** *************** *** 58,77 **** } catch (IOException e) { e.printStackTrace(); } ! count=0; } - /** Flush the internal buffer */ - private void flushBuffer() throws IOException { - if (count > 0) { - // forward to the console if open - if (console != null) { - trace(buf, 0, count); - } - count = 0; - } - } - private void trace(byte b[], int off, int len) { if (console != null) { console.append(new String(b, off, len)); --- 58,69 ---- } catch (IOException e) { e.printStackTrace(); } ! Thread thread = new Thread(this, "Console Writer"); ! thread.setDaemon(false); ! thread.setPriority(Thread.NORM_PRIORITY + 2); ! thread.start(); } private void trace(byte b[], int off, int len) { if (console != null) { console.append(new String(b, off, len)); *************** *** 88,102 **** * Writes the specified byte to this buffered output stream. * * @param b the byte to be written. - * @exception IOException if an I/O error occurs. */ ! public synchronized void write(int b) throws IOException { ! if (count >= buf.length) { ! flushBuffer(); ! } ! buf[count++] = (byte)b; ! if (b == '\n' || b == '\r') ! flushBuffer(); } /** --- 80,89 ---- * Writes the specified byte to this buffered output stream. * * @param b the byte to be written. */ ! public synchronized void write(int b) { ! super.write(b); ! this.notify(); } /** *************** *** 113,151 **** * @param b the data. * @param off the start offset in the data. * @param len the number of bytes to write. - * @exception IOException if an I/O error occurs. */ ! public synchronized void write(byte b[], int off, int len) throws IOException { ! if (len >= buf.length) { ! /* If the request length exceeds the size of the output buffer, ! flush the output buffer and then write the data directly. ! In this way buffered streams will cascade harmlessly. */ ! flushBuffer(); ! trace(b, off, len); ! return; ! } ! if (len > buf.length - count) { ! flushBuffer(); ! } ! System.arraycopy(b, off, buf, count, len); ! count += len; ! ! if (b[len-1] == '\n' || b[len-1] == '\r') ! flushBuffer(); } /** - * Flushes this buffered output stream. This forces any buffered - * output bytes to be written out to the underlying output stream. - * - * @exception IOException if an I/O error occurs. - * @see java.io.FilterOutputStream#out - */ - public synchronized void flush() throws IOException { - flushBuffer(); - } - - /** * <p> * Method to provide an alternative tracing facility beside * the console window. Messages could be traced for example --- 100,112 ---- * @param b the data. * @param off the start offset in the data. * @param len the number of bytes to write. */ ! public synchronized void write(byte b[], int off, int len) { ! super.write(b, off, len); ! this.notify(); } /** * <p> * Method to provide an alternative tracing facility beside * the console window. Messages could be traced for example *************** *** 156,162 **** */ protected void printDebug(String out) {} ! private int count; ! private byte[] buf = new byte[80]; private ConsoleWindow console = null; } --- 117,136 ---- */ protected void printDebug(String out) {} ! public void run() { ! while (true) { ! byte[] buf = null; ! synchronized (this) { ! try { ! this.wait(); ! } catch (InterruptedException e) { ! } ! buf = this.toByteArray(); ! this.reset(); ! } ! this.trace(buf, 0, buf.length); ! } ! } ! private ConsoleWindow console = null; } --- ext/plugin/java/src/sun/plugin/ConsoleWindow.java Mon Oct 25 19:06:22 1999 *************** *** 95,105 **** */ public void append(String text) { synchronized (textArea) { ! textArea.append(text); }; } - public void windowActivated(WindowEvent e) {}; public void windowClosed(WindowEvent e) {}; public void windowClosing(WindowEvent e) { dispose(); }; --- 95,109 ---- */ public void append(String text) { synchronized (textArea) { ! int diff = textArea.getText().length() + ! text.length() - TEXT_LIMIT; ! if (diff > 0) { ! textArea.replaceRange("", 0, diff); ! } ! textArea.append(text); }; } public void windowActivated(WindowEvent e) {}; public void windowClosed(WindowEvent e) {}; public void windowClosing(WindowEvent e) { dispose(); }; *************** *** 109,113 **** --- 113,118 ---- public void windowOpened(WindowEvent e) {}; private TextArea textArea; + private static final int TEXT_LIMIT = 0x3FFF; } ======================================================================
11-06-2004
EVALUATION As of 1.3 FCS build "K" this doesn't seem to occur for Netscape 4.08 or 4.7 (except for one case below), but does occur under MSIE5. All testing was done under NT4. In Netscape the applet just takes forever (minutes) to start, but the Java Console's "Clear" button still responds. The hang doesn't occur if Java Console not enabled. The "all" option actually encompasses a number of options (see sun.security.util.Debug). Various options may or may not cause a hang. For example, the "jar" option doesn't cause a hang. The "access" options has suboptions, e.g. "failure", "stack", and "domain". If "access,stack" or "access,domain" are used, then MSIE hangs. These options seem to enable debugging in java.security.AccessControlContext.checkPermission() and java.security.AccessController.checkPermission(). Looking at native stacks didn't yield anything obvious. At first I thought it was some interaction between native AccessController code and AWT TextArea code, but I couldn't get symbols from the VM because I was running HotSpot. When I switched to classic VM to get symbols the threads appeared to be in different places when I stopped the process. So that analysis seemed a little inconclusive. Further, trying to attach the java debugger didn't work because so much memory was being consumed (doesn't happen if not running in debug). One strange thing I just noticed is that Netscape appears to also hang when "access,stack" is used sometimes when the Java Plugin control panel also visible, but not always. In this case the control panel itself also seems to hang until the Netscape process is killed. This seems intermittent. ryang@eng 1999-10-12 This appears to be a race condition in awt_TextComponent::GetText(), where I caught it hanging while the message pump thread remained idle. This doesn't appear to have anything to do with java.security.debug or the plug-in's console, but instead is a case of a text component not being able to handle a lot of text being written to it. This is definitely a Windows-only problem. thomas.ball@Eng 1999-10-19 Name: dsC76792 Date: 10/25/99 ###@###.### The plugin has three nested failures: 1. After the plugin starts it writes a stream of debug output to the textarea in Java Console and then at some point new text doesn't appear anymore. The cause is that the text limit for the textarea is exhausted. Currently we don't set this limit and it defaults to ~0x7500 symbols. 2. When the first problem is fixed by extending the text limit the browser crashes with stack overflow at some time after the plugin starts. The cause if that AwtTextArea::CountNewLines passes all the text contained in the textarea to TO_WSTRING for conversion. TO_WSTRING allocates memory for the text on the stack causing stack overflow when the text is very long. 3. When the second problem is fixed by changing CountNewLines to allocate memory in the heap the plugin works fine while the console window is active. Once any other window becomes active the plugin hangs and the whole browser hangs. The cause is that the plugin deadlocks. When the java console is enabled the System.err and System.out are redirected to the textarea in the console window. When the plugin is started with -Djava.security.debug=all option it dumps stack to System.err on every call to java.security.AccessController.checkPermission. This method is called very often on almost all threads. One of possible deadlock conditions: 1.checkPermission() is called on the main thread. It starts dumping stack to System.err and calls Throwable.printStackTrace() that enters a block synchronized on System.err. 2.checkPermission() is called on the toolkit thread. It starts dumping stack to System.err, calls Throwable.printStackTrace() and waits to enter the same block synchronized on System.err. 3.The main thread tries to put the generated stack trace to the underlying native edit control of textarea by calling SendMessage() and since the toolkit thread is blocked the main thread is blocked. 4.Since this moment any thread that calls checkPermission() will be blocked. To investigate this problem i used a test case that simulates plugin behavior. It reassigns System.err and System.out to the instance of DebugOutputStream (stolen from sun.plugin package and slightly truncated) that writes all the data into a textarea the same way as the plugin does. After that it starts appletviewer with a sample applet. This test behaves the same way as the plugin: while the java console frame is active it works fine. When you try to switch to another window and back to the console (maybe several times) the test deadlocks. When this test locks up the thread dump is as follows: Full thread dump Classic VM (das, native threads): "AWT-Windows" (TID:0x15b7bd8, sys_thread_t:0x7e1920, state:MW, native ID:0xc c) prio=5 at java.lang.Throwable.printStackTrace(Throwable.java:187) at java.lang.Thread.dumpStack(Thread.java:992) at java.security.AccessController.checkPermission(AccessController.java: 393) at java.lang.SecurityManager.checkPermission(SecurityManager.java:551) at java.lang.SecurityManager.checkLink(SecurityManager.java:843) at java.lang.Runtime.loadLibrary0(Runtime.java:742) at java.lang.System.loadLibrary(System.java:817) at sun.security.action.LoadLibraryAction.run(LoadLibraryAction.java:57) at java.security.AccessController.doPrivileged(Native Method) at java.awt.event.NativeLibLoader.loadLibraries(NativeLibLoader.java:45) at java.awt.event.MouseEvent.<clinit>(MouseEvent.java:206) at sun.awt.windows.WToolkit.eventLoop(Native Method) at sun.awt.windows.WToolkit.run(WToolkit.java:190) at java.lang.Thread.run(Thread.java:488) "SunToolkit.PostEventQueue-0" (TID:0x15b4e78, sys_thread_t:0x7d3c30, state:C W, native ID:0xc1) prio=5 at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Object.java:424) at sun.awt.PostEventQueue.run(SunToolkit.java:500) "AWT-EventQueue-0" (TID:0x15b4ea0, sys_thread_t:0x7d23b0, state:MW, native I D:0xc5) prio=6 at java.lang.Throwable.printStackTrace(Throwable.java:187) at java.lang.Thread.dumpStack(Thread.java:992) at java.security.AccessController.checkPermission(AccessController.java: 393) at java.lang.SecurityManager.checkPermission(SecurityManager.java:551) at java.lang.SecurityManager.checkPropertyAccess(SecurityManager.java:12 84) at java.lang.System.getProperty(System.java:557) at sun.security.action.GetPropertyAction.run(GetPropertyAction.java:73) at java.security.AccessController.doPrivileged(Native Method) at sun.java2d.loops.RasterOutputManager.<clinit>(RasterOutputManager.jav a:197) at sun.awt.windows.WGraphics.<clinit>(WGraphics.java:84) at sun.awt.windows.WComponentPeer.getGraphics(WComponentPeer.java:285) at java.awt.Component.getGraphics(Component.java:1838) at sun.awt.RepaintArea.paint(RepaintArea.java:261) at sun.awt.windows.WComponentPeer.handleEvent(WComponentPeer.java:197) at java.awt.Component.dispatchEventImpl(Component.java:2653) at java.awt.Container.dispatchEventImpl(Container.java:1211) at java.awt.Window.dispatchEventImpl(Window.java:886) at java.awt.Component.dispatchEvent(Component.java:2486) at java.awt.EventQueue.dispatchEvent(EventQueue.java:320) at java.awt.EventDispatchThread.pumpOneEvent(EventDispatchThread.java:10 7) at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:97) at java.awt.EventDispatchThread.run(EventDispatchThread.java:88) "Finalizer" (TID:0x1599528, sys_thread_t:0x779580, state:CW, native ID:0xc6) prio=8 at java.lang.Object.wait(Native Method) at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:112) at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:127) at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:166) "Reference Handler" (TID:0x1599300, sys_thread_t:0x7787c0, state:CW, native ID:0xb5) prio=10 at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Object.java:424) at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:114) "Signal dispatcher" (TID:0x1599330, sys_thread_t:0x778f90, state:R, native I D:0xba) prio=5 "main" (TID:0x15991a0, sys_thread_t:0x761ed0, state:R, native ID:0xbd) prio= 5 at sun.awt.windows.WTextAreaPeer.insertText(Native Method) at java.awt.TextArea.insertText(TextArea.java:260) at java.awt.TextArea.appendText(TextArea.java:281) at java.awt.TextArea.append(TextArea.java:272) at DebugOutputStream.trace(FullTest.java:66) at DebugOutputStream.flushBuffer(FullTest.java:58) at DebugOutputStream.write(FullTest.java:118) at java.io.PrintStream.write(PrintStream.java:226) at java.io.OutputStreamWriter.flushBuffer(OutputStreamWriter.java:227) at java.io.PrintStream.newLine(PrintStream.java:292) at java.io.PrintStream.println(PrintStream.java:534) at java.lang.Throwable.printStackTrace0(Native Method) at java.lang.Throwable.printStackTrace(Throwable.java:185) at java.lang.Thread.dumpStack(Thread.java:992) at java.security.AccessControlContext.checkPermission(AccessControlConte xt.java:234) at java.security.AccessController.checkPermission(AccessController.java: 403) at java.lang.SecurityManager.checkPermission(SecurityManager.java:551) at java.lang.SecurityManager.checkRead(SecurityManager.java:896) at java.io.FileInputStream.<init>(FileInputStream.java:65) at sun.net.www.protocol.file.FileURLConnection.connect(FileURLConnection .java:73) at sun.net.www.protocol.file.FileURLConnection.getInputStream(FileURLCon nection.java:137) at sun.applet.AppletViewer.parse(AppletViewer.java:1002) at sun.applet.AppletViewer.parse(AppletViewer.java:973) at sun.applet.Main.run(Main.java:144) at sun.applet.Main.main(Main.java:86) at FullTest.main(FullTest.java:32) Monitor Cache Dump: sun.net.www.protocol.file.FileURLConnection@15AF150/1637620: owner "main" (0 x761ed0) 1 entry java.lang.Runtime@159E870/15ECB60: owner "AWT-Windows" (0x7e1920) 1 entry java.awt.TextArea@15B7038/15F1D08: owner "main" (0x761ed0) 2 entries sun.awt.PostEventQueue@15B4E78/1631788: <unowned> Waiting to be notified: "SunToolkit.PostEventQueue-0" (0x7d3c30) java.lang.ref.ReferenceQueue$Lock@1599540/15CF338: <unowned> Waiting to be notified: "Finalizer" (0x779580) java.io.PrintStream@15B71C8/15F1EB8: owner "main" (0x761ed0) 5 entries Waiting to enter: "AWT-Windows" (0x7e1920) "AWT-EventQueue-0" (0x7d23b0) sun.awt.windows.WFramePeer@15B6FD8/16048C8: owner "AWT-EventQueue-0" (0x7d23 b0) 1 entry java.lang.ref.Reference$Lock@1599310/15CEF10: <unowned> Waiting to be notified: "Reference Handler" (0x7787c0) DebugOutputStream@15B7148/15F1E50: owner "main" (0x761ed0) 1 entry Registered Monitor Dump: utf8 hash table: <unowned> JNI pinning lock: <unowned> JNI global reference lock: <unowned> BinClass lock: <unowned> Class linking lock: <unowned> System class loader lock: <unowned> Code rewrite lock: <unowned> Heap lock: <unowned> Monitor cache lock: owner "Signal dispatcher" (0x778f90) 1 entry Thread queue lock: owner "Signal dispatcher" (0x778f90) 1 entry Monitor registry: owner "Signal dispatcher" (0x778f90) 1 entry ====================================================================== Let's fix this for Merlin or Ladybird jerome.dochez@Eng 1999-10-27 AWT problem will be fixed in Kestrel as of build 'O'. Reassigning to plugin team. lara.bunni@Eng 1999-11-04 AWT fix integrated (see Suggested Fix for details). eric.hawkes@eng 1999-11-05
04-11-1999