User heavy load of 1000 HTTPS threads WebLogic 12.1.1 managed server running 
with full debug logging jstack reports: 

Deadlock Detection: 

Found one Java-level deadlock: 
============================= 

"[STUCK] ExecuteThread: '2' for queue: 'weblogic.kernel.Default 
(self-tuning)'": 
  waiting to lock Monitor@0x000000010521cfe8 (Object@0x00000006065d4320, a 
java/io/PrintStream), 
  which is held by "[ACTIVE] ExecuteThread: '4' for queue: 
'weblogic.kernel.Default (self-tuning) 
'" 
"[ACTIVE] ExecuteThread: '4' for queue: 'weblogic.kernel.Default 
(self-tuning)'": 
  waiting to lock Monitor@0x0000000105219418 (Object@0x0000000606664aa8, a 
sun/security/pkcs11/Se 
ssionManager), 
  which is held by "[STUCK] ExecuteThread: '2' for queue: 
'weblogic.kernel.Default (self-tuning)' 
" 

Found a total of 1 deadlock. 

----------------- t@1 ----------------- 
0xffffffff7eae75fc      __lwp_wait + 0x4 
0xffffffff7eade8d4      _thrp_join + 0x3c 
0xffffffff7eadea40      thr_join + 0x14 
0xffffffff75513408      ContinueInNewThread0 + 0x30 
0xffffffff7550617c      ContinueInNewThread + 0x94 
0xffffffff75513484      JVMInit + 0x28 
0xffffffff75501c70      JLI_Launch + 0x2d0 
0x0000000100000a00      main + 0x90 
0x000000010000093c      _start + 0x17c 
----------------- t@2 ----------------- 
0xffffffff7eae76b0      ___lwp_cond_wait + 0x4 
0xffffffff74df2b00      void os::PlatformEvent::park() + 0x100 
0xffffffff74dd2944      void ObjectMonitor::wait(long,bool,Thread*) + 0x32c 
0xffffffff74eff338      void ObjectSynchronizer::wait(Handle,long,Thread*) + 
0x160 
0xffffffff745b3794      JVM_MonitorWait + 0x2d0 
0xffffffff70410d6c      <interpreter> method entry point (kind = native) 
0xffffffff70410d10      <interpreter> method entry point (kind = native) 
0xffffffff70406f44      <interpreter> return entry points 
0xffffffff70406f44      <interpreter> return entry points 
0xffffffff70406f44      <interpreter> return entry points 
0xffffffff70406cb4      <interpreter> return entry points 
0xffffffff7040024c      <StubRoutines> 
0xffffffff74524700      void 
JavaCalls::call_helper(JavaValue*,methodHandle*,JavaCallArguments*,T 
hread*) + 0x2a0 
0xffffffff74531a14      void 
JavaCalls::call(JavaValue*,methodHandle,JavaCallArguments*,Thread*) 
+ 0x3c 
0xffffffff745cf20c      jni_CallStaticVoidMethod + 0x630 
0xffffffff755023ec      JavaMain + 0x74c 
0xffffffff7eae2694      _lwp_start 
Locked ownable synchronizers: 
    - None 
----------------- t@3 ----------------- 
0xffffffff7eae76b0      ___lwp_cond_wait + 0x4 
0xffffffff74df2b00      void os::PlatformEvent::park() + 0x100 
0xffffffff74da0ebc      int Monitor::IWait(Thread*,long) + 0xa4 
0xffffffff74da1dd0      bool Monitor::wait(bool,long,bool) + 0x350 
0xffffffff745231f8      GCTask*GCTaskManager::get_task(unsigned) + 0xa8 
0xffffffff7461e598      void GCTaskThread::run() + 0x200 
0xffffffff74de765c      java_start + 0x35c 
0xffffffff7eae2694      _lwp_start 
----------------- t@4 ----------------- 
0xffffffff7eae76b0      ___lwp_cond_wait + 0x4 
0xffffffff74df2b00      void os::PlatformEvent::park() + 0x100 
0xffffffff74da0ebc      int Monitor::IWait(Thread*,long) + 0xa4 
0xffffffff74da1dd0      bool Monitor::wait(bool,long,bool) + 0x350 
0xffffffff745231f8      GCTask*GCTaskManager::get_task(unsigned) + 0xa8 
0xffffffff7461e598      void GCTaskThread::run() + 0x200 
0xffffffff74de765c      java_start + 0x35c 
0xffffffff7eae2694      _lwp_start 


The error does not occur if the debug options are off or if the SunPKCS11 
provider is removed from java.security. 

JAVA_OPTIONS="${SAVE_JAVA_OPTIONS} 
-Dweblogic.security.SSL.ignoreHostnameVerification=true -Dweblogic.nodem 
anager.sslHostNameVerificationEnabled=false   
-Dweblogic.security.SSL.enforceConstraints=off  -Dssl.debug=true 
-Dweblogic.StdoutDebugEnabled=true 
-Djavax.net.debug=ssl,handshake,record,keygen,session,defaultctx,sslctx,ses 
sioncache,keymanager,trustmanager -Dcom.oracle.security.ucrypto.debug=true   
-Djava.security.debug=all -Djsse.S 
SLEngine.acceptLargeFragments=true -Dweblogic.ssl.JSSEEnabled=true   
-Dweblogic.management.allowPasswordEcho=tr 
ue" 
 
Hang always occurs after this info is printed by verbose logging in JDK : 
==== 
Server write IV: 
0000: 8B DF 9B 59 E3 E5 3C 59                            ...Y..<Y 
Generating a random fake premaster secret 
[ACTIVE] ExecuteThread: '1' for queue: 'weblogic.kernel.Default 
(self-tuning)', WRITE: TLSv1 Change Cipher Spec, length = 1 
CONNECTION KEYGEN: 
Client Nonce: 
0000: 54 1C 81 96 90 75 72 F8   38 72 56 BA 31 BB 1F 12  T....ur.8rV.1... 
0010: 9F D9 80 B0 3F 79 57 14   ED D3 80 AE 77 8D 36 1D  ....?yW.....w.6. 
Server Nonce: 
0000: 54 1C 81 9E 20 13 2A 9A   A7 A6 B9 FD 2A 14 3C 92  T... .*.....*.<. 
0010: B5 E2 65 11 D8 6E BB 6D   18 D2 DC 9E 9E 0F 26 33  ..e..n.m......&3 
Master Secret: 
 
==== 
# At this point the WLS managed server is hung. 

I suspect the "synchronized (System.out)" code found in 
sun/security/ssl/Handshaker.java is not good to have.