JDK-8211909 : JDWP Transport Listener: dt_socket thread crash
  • Type: Bug
  • Component: hotspot
  • Sub-Component: jvmti
  • Affected Version: 8,10.0.2,12
  • Priority: P3
  • Status: Resolved
  • Resolution: Fixed
  • OS: windows_7
  • CPU: x86_64
  • Submitted: 2018-10-08
  • Updated: 2019-07-29
  • Resolved: 2018-10-16
The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.
JDK 11 JDK 12 JDK 8
11.0.2Fixed 12 b16Fixed 8u202Fixed
Related Reports
Duplicate :  
Duplicate :  
Relates :  
Description
ADDITIONAL SYSTEM INFORMATION :
OS: Windows 7 , 64 bit Build 7601 (6.1.7601.23807)

CPU:total 4 (4 cores per cpu, 1 threads per core) family 6 model 60 stepping 3, cmov, cx8, fxsr, mmx, sse, sse2, sse3, ssse3, sse4.1, sse4.2, popcnt, avx, avx2, aes, clmul, erms, rtm, lzcnt, tsc, tscinvbit, bmi1, bmi2

Memory: 4k page, physical 33455432k(15060212k free), swap 66920964k(49306188k free)

vm_info: Java HotSpot(TM) 64-Bit Server VM (25.121-b13) for windows-amd64 JRE (1.8.0_121-b13), built on Dec 12 2016 18:21:36 by "java_re" with MS VC++ 10.0 (VS2010)

----

The same problem was happening on some earlier versions of JVM 8. I do not remember having this problem when working with JVM 7.

A DESCRIPTION OF THE PROBLEM :
Often when debugging an application (from Eclipse), the JVM runtime causes EXCEPTION_ACCESS_VIOLATION caused by NULL pointer dereference.

Current thread (0x0000000026cf9000):  JavaThread "JDWP Transport Listener: dt_socket" daemon [_thread_in_vm, id=23256, stack(0x00000000274c0000,0x00000000275c0000)]

Windows Minidump available. There doesn't seem to be a place to attach the dump to this report, so the minidump is accessible at

http://server.gamagio.com/oracle-jdk/1/jvm_crash.zip

I don't have debug symbols for jvm.dll, so the info from WinDbg isn't very helpful, except it shows a null pointer dereference:
0:012> .ecxr
*** ERROR: Symbol file could not be found.  Defaulted to export symbols for jvm.dll - 
rax=0000000000000010 rbx=00000000256afe80 rcx=0000000000000000
rdx=000000000000002f rsi=0000000000000000 rdi=0000000026cf9000
rip=00000000550d8637 rsp=00000000275bf238 rbp=000000000000002f
 r8=0000000000000002  r9=000000071d23ab38 r10=0000000000000000
r11=00000000275bf160 r12=00000000256af598 r13=0000000000000030
r14=0000000000000000 r15=000000071d23ab38
iopl=0         nv up ei pl zr na po nc
cs=0033  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00010246
jvm!JVM_GetManagementExt+0x443a7:
00000000`550d8637 488b0408        mov     rax,qword ptr [rax+rcx] ds:00000000`00000010=????????????????
0:012> kPn
  *** Stack trace for last set context - .thread/.cxr resets it
 # Child-SP          RetAddr           Call Site
00 00000000`275bf238 00000000`5521b2c5 jvm!JVM_GetManagementExt+0x443a7
01 00000000`275bf240 00000000`553a864e jvm!JVM_GetThreadStateNames+0xd2f5
*** ERROR: Symbol file could not be found.  Defaulted to export symbols for jdwp.dll - 
02 00000000`275bf340 00000000`6fb6806c jvm!JVM_FindSignal+0xa27de
03 00000000`275bf3a0 00000000`6fb6f168 jdwp+0x806c
04 00000000`275bf410 00000000`6fb7f395 jdwp!Agent_OnLoad+0xdb8
05 00000000`275bf650 00000000`6fb7f56a jdwp!Agent_OnLoad+0x10fe5
06 00000000`275bf680 00000000`5522d548 jdwp!Agent_OnLoad+0x111ba
07 00000000`275bf6b0 00000000`552b4e1f jvm!JVM_GetThreadStateNames+0x1f578
08 00000000`275bf6e0 00000000`5530c69a jvm!JVM_ResolveClass+0x5ee7f
09 00000000`275bf760 00000000`6fc01d9f jvm!JVM_FindSignal+0x682a
0a 00000000`275bfa20 00000000`6fc01e3b msvcr100!_callthreadstartex(void)+0x17 [f:\dd\vctools\crt_bld\self_64_amd64\crt\src\threadex.c @ 314] 
0b 00000000`275bfa50 00000000`76f459cd msvcr100!_threadstartex(
			void * ptd = 0x00000000`256aa4f0)+0x7f [f:\dd\vctools\crt_bld\self_64_amd64\crt\src\threadex.c @ 292] 
0c 00000000`275bfa80 00000000`7717a561 kernel32!BaseThreadInitThunk+0xd
0d 00000000`275bfab0 00000000`00000000 ntdll!RtlUserThreadStart+0x1d
----

The crash happens often enough - I typically get a few every day. They always happen at different times, there doesn't seem to be a pattern. My Java code does not contain any weird things like JNI calls from C++ - just Java code.

ERROR MESSAGES/STACK TRACES THAT OCCUR :
#
# A fatal error has been detected by the Java Runtime Environment:
#
#  EXCEPTION_ACCESS_VIOLATION (0xc0000005) at pc=0x00000000550d8637, pid=21508, tid=0x0000000000005ad8
#
# JRE version: Java(TM) SE Runtime Environment (8.0_121-b13) (build 1.8.0_121-b13)
# Java VM: Java HotSpot(TM) 64-Bit Server VM (25.121-b13 mixed mode windows-amd64 compressed oops)
# Problematic frame:
# V  [jvm.dll+0x68637]
#
# Core dump written. Default location: D:\Blimey!\Server\Source\Hydra\hs_err_pid21508.mdmp
#
# If you would like to submit a bug report, please visit:
#   http://bugreport.java.com/bugreport/crash.jsp
#

---------------  T H R E A D  ---------------

Current thread (0x0000000026cf9000):  JavaThread "JDWP Transport Listener: dt_socket" daemon [_thread_in_vm, id=23256, stack(0x00000000274c0000,0x00000000275c0000)]

siginfo: ExceptionCode=0xc0000005, reading address 0x0000000000000010

Registers:
RAX=0x0000000000000010, RBX=0x00000000256afe80, RCX=0x0000000000000000, RDX=0x000000000000002f
RSP=0x00000000275bf238, RBP=0x000000000000002f, RSI=0x0000000000000000, RDI=0x0000000026cf9000
R8 =0x0000000000000002, R9 =0x000000071d23ab38, R10=0x0000000000000000, R11=0x00000000275bf160
R12=0x00000000256af598, R13=0x0000000000000030, R14=0x0000000000000000, R15=0x000000071d23ab38
RIP=0x00000000550d8637, EFLAGS=0x0000000000010246

Top of Stack: (sp=0x00000000275bf238)
0x00000000275bf238:   000000005521b2c5 00000000256afe80
0x00000000275bf248:   0000000000000030 0000000000000004
0x00000000275bf258:   000000005528f221 0000000000000000
0x00000000275bf268:   00000000000003d8 00000000256af420
0x00000000275bf278:   0000000000000000 00000000256af7f8
0x00000000275bf288:   0000000026cf9000 00000000256af420
0x00000000275bf298:   0000000025691360 00000000256af410
0x00000000275bf2a8:   00000000004ae330 0000000026cf9000
0x00000000275bf2b8:   00000000256913a0 00000000256afe70
0x00000000275bf2c8:   00000000256afe80 00000000256aff58
0x00000000275bf2d8:   00000000000000d8 00000000275bf700
0x00000000275bf2e8:   00000000551ae06f 00000000256af7f8
0x00000000275bf2f8:   0000000000000000 0000000000000000
0x00000000275bf308:   00000000256af7f8 000000002a48dd70
0x00000000275bf318:   00000000275bf420 00000000004aa520
0x00000000275bf328:   00000000275bf3d0 0000000026cf9000 

Instructions: (pc=0x00000000550d8637)
0x00000000550d8617:   05 c4 38 73 00 83 f8 ff 74 03 89 14 08 f3 c3 cc
0x00000000550d8627:   cc cc cc cc cc cc cc cc cc 48 63 05 4d 9d 78 00
0x00000000550d8637:   48 8b 04 08 c3 cc cc cc cc 48 63 05 3d 9d 78 00
0x00000000550d8647:   48 89 14 08 c3 cc cc cc cc 48 63 05 29 9d 78 00 


Register to memory mapping:

RAX=0x0000000000000010 is an unknown value
RBX=0x00000000256afe80 is an unknown value
RCX=0x0000000000000000 is an unknown value
RDX=0x000000000000002f is an unknown value
RSP=0x00000000275bf238 is pointing into the stack for thread: 0x0000000026cf9000
RBP=0x000000000000002f is an unknown value
RSI=0x0000000000000000 is an unknown value
RDI=0x0000000026cf9000 is a thread
R8 =0x0000000000000002 is an unknown value
R9 =0x000000071d23ab38 is an oop
[Ljava.lang.Thread; 
 - klass: 'java/lang/Thread'[]
 - length: 64
R10=0x0000000000000000 is an unknown value
R11=0x00000000275bf160 is pointing into the stack for thread: 0x0000000026cf9000
R12=0x00000000256af598 is an unknown value
R13=0x0000000000000030 is an unknown value
R14=0x0000000000000000 is an unknown value
R15=0x000000071d23ab38 is an oop
[Ljava.lang.Thread; 
 - klass: 'java/lang/Thread'[]
 - length: 64


Stack: [0x00000000274c0000,0x00000000275c0000],  sp=0x00000000275bf238,  free space=1020k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
V  [jvm.dll+0x68637]


---------------  P R O C E S S  ---------------

Java Threads: ( => current thread )
  0x0000000045e2b800 JavaThread "LEADERBOARD michal:public (175)" [_thread_blocked, id=22296, stack(0x0000000032000000,0x0000000032100000)]
  0x00000000293ec000 JavaThread "DISCOVERY michal:private (56)" [_thread_blocked, id=22972, stack(0x00000000441a0000,0x00000000442a0000)]
  0x000000002d0ee800 JavaThread "PROFILE michal:public (38)" [_thread_blocked, id=11596, stack(0x0000000033850000,0x0000000033950000)]
  0x00000000296c5800 JavaThread "LEADERBOARD michal:public (174)" [_thread_blocked, id=23320, stack(0x000000002f6d0000,0x000000002f7d0000)]
  0x00000000296c7000 JavaThread "DISCOVERY michal:private (55)" [_thread_blocked, id=6944, stack(0x0000000031ba0000,0x0000000031ca0000)]
  0x0000000045e26000 JavaThread "PROFILE michal:public (37)" [_thread_blocked, id=11648, stack(0x00000000438c0000,0x00000000439c0000)]
  0x000000002dcdb000 JavaThread "DISCOVERY michal:private (54)" [_thread_blocked, id=13940, stack(0x0000000031420000,0x0000000031520000)]
  0x0000000045e24800 JavaThread "PROFILE michal:public (36)" [_thread_blocked, id=9952, stack(0x0000000040960000,0x0000000040a60000)]
  0x0000000036dfc000 JavaThread "DISCOVERY michal:private (53)" [_thread_blocked, id=9052, stack(0x000000002f5a0000,0x000000002f6a0000)]
  0x000000002dcd4800 JavaThread "PROFILE michal:public (35)" [_thread_blocked, id=15104, stack(0x0000000035af0000,0x0000000035bf0000)]
  0x000000002dcdc800 JavaThread "FRONT_END michal:public (171)" [_thread_blocked, id=9300, stack(0x0000000040490000,0x0000000040590000)]
  0x000000002dcc6800 JavaThread "DISCOVERY michal:private (52)" [_thread_blocked, id=3332, stack(0x0000000032670000,0x0000000032770000)]
  0x0000000045e27000 JavaThread "FRIEND michal:public (15)" [_thread_blocked, id=8700, stack(0x0000000043f10000,0x0000000044010000)]
  0x0000000045e23000 JavaThread "FRONT_END michal:public (170)" [_thread_blocked, id=17356, stack(0x0000000043d40000,0x0000000043e40000)]
  0x0000000045e1b800 JavaThread "FRONT_END michal:public (169)" [_thread_blocked, id=21952, stack(0x00000000437c0000,0x00000000438c0000)]
  0x0000000045e1f800 JavaThread "FRONT_END michal:public (168)" [_thread_blocked, id=348, stack(0x0000000043510000,0x0000000043610000)]
  0x0000000045e1d000 JavaThread "PROFILE michal:public (34)" [_thread_blocked, id=14904, stack(0x0000000043660000,0x0000000043760000)]
  0x0000000045e1c800 JavaThread "FRONT_END michal:public (167)" [_thread_blocked, id=22988, stack(0x00000000430c0000,0x00000000431c0000)]
  0x0000000045e1b000 JavaThread "PROFILE michal:public (33)" [_thread_blocked, id=18328, stack(0x0000000043200000,0x0000000043300000)]
  0x000000002d0f9000 JavaThread "FRONT_END michal:public (166)" [_thread_blocked, id=19752, stack(0x0000000042fb0000,0x00000000430b0000)]
  0x000000002d0fc000 JavaThread "FRONT_END michal:public (165)" [_thread_blocked, id=6108, stack(0x00000000428c0000,0x00000000429c0000)]
  0x000000002d0f6000 JavaThread "PROFILE michal:public (32)" [_thread_blocked, id=20804, stack(0x0000000042e20000,0x0000000042f20000)]
  0x000000002d0f4800 JavaThread "FRONT_END michal:public (164)" [_thread_blocked, id=17692, stack(0x0000000042b20000,0x0000000042c20000)]
  0x000000002d0fb000 JavaThread "FRONT_END michal:public (163)" [_thread_blocked, id=20012, stack(0x0000000042720000,0x0000000042820000)]
  0x000000002d0fa800 JavaThread "ACCOUNT michal:public (8)" [_thread_blocked, id=20352, stack(0x0000000041e50000,0x0000000041f50000)]
  0x000000002d0f8000 JavaThread "FRONT_END michal:public (162)" [_thread_blocked, id=22464, stack(0x00000000425a0000,0x00000000426a0000)]
  0x000000002d0f6800 JavaThread "FRONT_END michal:public (161)" [_thread_blocked, id=13244, stack(0x0000000042330000,0x0000000042430000)]
  0x000000002d0f3800 JavaThread "ACCOUNT michal:public (7)" [_thread_blocked, id=19852, stack(0x0000000042430000,0x0000000042530000)]
  0x000000002d0f0800 JavaThread "GAME_LOGGER michal:public (12)" [_thread_blocked, id=15956, stack(0x0000000041a90000,0x0000000041b90000)]
  0x000000002d0f1800 JavaThread "FRONT_END michal:public (160)" [_thread_blocked, id=5584, stack(0x0000000042220000,0x0000000042320000)]
  0x000000002d0f3000 JavaThread "FRONT_END michal:public (159)" [_thread_blocked, id=23200, stack(0x0000000042080000,0x0000000042180000)]
  0x000000002d0ed000 JavaThread "PROFILE michal:public (31)" [_thread_blocked, id=22652, stack(0x0000000041f50000,0x0000000042050000)]
  0x000000002d0f2000 JavaThread "FRONT_END michal:public (158)" [_thread_blocked, id=17488, stack(0x0000000041850000,0x0000000041950000)]
  0x000000002d0f0000 JavaThread "FRONT_END michal:public (157)" [_thread_blocked, id=2524, stack(0x000000003cc50000,0x000000003cd50000)]
  0x000000002d0ef000 JavaThread "GAME_LOGGER michal:public (11)" [_thread_blocked, id=8352, stack(0x0000000041990000,0x0000000041a90000)]
  0x0000000036e02000 JavaThread "FRIEND michal:public (14)" [_thread_blocked, id=9940, stack(0x00000000416f0000,0x00000000417f0000)]
  0x0000000036df6000 JavaThread "FRONT_END michal:public (156)" [_thread_blocked, id=23548, stack(0x0000000041380000,0x0000000041480000)]
  0x0000000036df2000 JavaThread "GAME_LOGGER michal:public (10)" [_thread_blocked, id=6316, stack(0x000000003fc40000,0x000000003fd40000)]
  0x0000000036df6800 JavaThread "FRONT_END michal:private (23)" [_thread_blocked, id=19004, stack(0x0000000040fb0000,0x00000000410b0000)]
  0x0000000036df7800 JavaThread "FRONT_END michal:private (22)" [_thread_blocked, id=11068, stack(0x0000000040df0000,0x0000000040ef0000)]
  0x0000000036df9000 JavaThread "LOGIN michal:private (17)" [_thread_blocked, id=1048, stack(0x0000000040c40000,0x0000000040d40000)]
  0x0000000036df5000 JavaThread "ACCOUNT michal:private (16)" [_thread_blocked, id=7832, stack(0x0000000040a60000,0x0000000040b60000)]
  0x0000000036e01000 JavaThread "LEADERBOARD michal:private (27)" [_thread_blocked, id=4652, stack(0x0000000040850000,0x0000000040950000)]
  0x0000000036dff000 JavaThread "WEB_SERVANT michal:private (10)" [_thread_blocked, id=17996, stack(0x0000000040050000,0x0000000040150000)]
  0x0000000036dfb000 JavaThread "FRIEND michal:private (9)" [_thread_blocked, id=22976, stack(0x0000000040720000,0x0000000040820000)]
  0x0000000036dfe000 JavaThread "PLAYER_MESSAGING michal:private (16)" [_thread_blocked, id=17076, stack(0x0000000040590000,0x0000000040690000)]
  0x0000000036e02800 JavaThread "DISCOVERY michal:private (51)" [_thread_blocked, id=6208, stack(0x0000000040390000,0x0000000040490000)]
  0x0000000036e04000 JavaThread "PROFILE michal:private (14)" [_thread_blocked, id=9980, stack(0x0000000040280000,0x0000000040380000)]
  0x0000000036e00800 JavaThread "SERVANT michal:private (15)" [_thread_blocked, id=1480, stack(0x0000000040160000,0x0000000040260000)]
  0x0000000036e05000 JavaThread "GAME_LOGGER michal:private (17)" [_thread_blocked, id=15048, stack(0x000000003e810000,0x000000003e910000)]
  0x0000000036dff800 JavaThread "BOX_MONITOR michal:private (20)" [_thread_blocked, id=15016, stack(0x000000003ff30000,0x0000000040030000)]
  0x0000000036e03800 JavaThread "CHAT michal:private (18)" [_thread_blocked, id=14344, stack(0x000000003fd80000,0x000000003fe80000)]
  0x0000000036dfd800 JavaThread "LOGIN michal:private (16)" [_thread_blocked, id=13128, stack(0x000000003fb10000,0x000000003fc10000)]
  0x000000002b872000 JavaThread "FRONT_END michal:private (21)" [_thread_blocked, id=9356, stack(0x000000003f490000,0x000000003f590000)]
  0x000000002b870800 JavaThread "GAME_LOGGER michal:private (16)" [_thread_blocked, id=18304, stack(0x000000003f6d0000,0x000000003f7d0000)]
  0x000000002b873000 JavaThread "LOGIN michal:public (10)" [_thread_blocked, id=20212, stack(0x000000003f9c0000,0x000000003fac0000)]
  0x00000000293f5800 JavaThread "DISCOVERY michal:public (6)" [_thread_blocked, id=21428, stack(0x000000003f800000,0x000000003f900000)]
  0x00000000293f5000 JavaThread "GAME_LOGGER michal:public (9)" [_thread_blocked, id=10720, stack(0x000000003f5d0000,0x000000003f6d0000)]
  0x00000000293f4000 JavaThread "GAME_LOGGER michal:public (8)" [_thread_blocked, id=21588, stack(0x000000003f360000,0x000000003f460000)]
  0x00000000293f6800 JavaThread "GAME_LOGGER michal:private (15)" [_thread_blocked, id=21480, stack(0x000000003e330000,0x000000003e430000)]
  0x00000000293f3800 JavaThread "DISCOVERY michal:private (50)" [_thread_blocked, id=17512, stack(0x0000000035890000,0x0000000035990000)]
  0x000000002b874800 JavaThread "LOGIN michal:private (15)" [_thread_blocked, id=15804, stack(0x000000003ec50000,0x000000003ed50000)]
  0x000000002b871800 JavaThread "FRONT_END michal:private (20)" [_thread_blocked, id=22356, stack(0x000000003f040000,0x000000003f140000)]
  0x000000002b877000 JavaThread "WEB_SERVANT michal:private (9)" [_thread_blocked, id=16740, stack(0x000000003ef40000,0x000000003f040000)]
  0x000000002b87a800 JavaThread "FRIEND michal:private (8)" [_thread_blocked, id=5956, stack(0x000000003eac0000,0x000000003ebc0000)]
  0x000000002b87d000 JavaThread "PLAYER_MESSAGING michal:private (15)" [_thread_blocked, id=18744, stack(0x000000003e910000,0x000000003ea10000)]
  0x000000002b87e800 JavaThread "ACCOUNT michal:private (15)" [_thread_blocked, id=5804, stack(0x000000003e710000,0x000000003e810000)]
  0x000000002b875800 JavaThread "LEADERBOARD michal:private (26)" [_thread_blocked, id=9824, stack(0x000000003e5d0000,0x000000003e6d0000)]
  0x000000002b880000 JavaThread "PROFILE michal:private (13)" [_thread_blocked, id=12708, stack(0x000000003c030000,0x000000003c130000)]
  0x000000002b87c000 JavaThread "SERVANT michal:private (14)" [_thread_blocked, id=5236, stack(0x000000003e430000,0x000000003e530000)]
  0x000000002b87f000 JavaThread "GAME_LOGGER michal:private (14)" [_thread_blocked, id=6952, stack(0x000000003e210000,0x000000003e310000)]
  0x000000002b87d800 JavaThread "DISCOVERY michal:private (49)" [_thread_blocked, id=21100, stack(0x000000003e060000,0x000000003e160000)]
  0x000000002b87b800 JavaThread "BOX_MONITOR michal:private (19)" [_thread_blocked, id=4780, stack(0x000000003de70000,0x000000003df70000)]
  0x000000002b890800 JavaThread "GAME_LOGGER michal:private (13)" [_thread_blocked, id=1468, stack(0x000000003d000000,0x000000003d100000)]
  0x000000002b88e000 JavaThread "CHAT michal:private (17)" [_thread_blocked, id=12476, stack(0x000000003dce0000,0x000000003dde0000)]
  0x000000002b891000 JavaThread "FRONT_END michal:public (155)" [_thread_blocked, id=13132, stack(0x000000003d200000,0x000000003d300000)]
  0x000000002b888000 JavaThread "FRONT_END michal:public (154)" [_thread_blocked, id=8756, stack(0x000000003ca40000,0x000000003cb40000)]
  0x000000002b88f000 JavaThread "LEADERBOARD michal:public (173)" [_thread_blocked, id=10784, stack(0x000000003daf0000,0x000000003dbf0000)]
  0x000000002b883800 JavaThread "FRONT_END michal:public (153)" [_thread_blocked, id=21360, stack(0x000000003d920000,0x000000003da20000)]
  0x000000002b889800 JavaThread "FRONT_END michal:public (152)" [_thread_blocked, id=7324, stack(0x000000003d770000,0x000000003d870000)]
  0x000000002b882000 JavaThread "LEADERBOARD michal:public (172)" [_thread_blocked, id=12348, stack(0x000000003d480000,0x000000003d580000)]
  0x000000002b884800 JavaThread "FRONT_END michal:public (151)" [_thread_blocked, id=19868, stack(0x000000003d310000,0x000000003d410000)]
  0x000000002b88c800 JavaThread "FRONT_END michal:public (150)" [_thread_blocked, id=15236, stack(0x000000003d100000,0x000000003d200000)]
  0x000000002b88a800 JavaThread "LEADERBOARD michal:public (171)" [_thread_blocked, id=7408, stack(0x000000003cef0000,0x000000003cff0000)]
  0x000000002b887800 JavaThread "FRONT_END michal:public (149)" [_thread_blocked, id=8092, stack(0x000000003c4b0000,0x000000003c5b0000)]
  0x0000000028e90800 JavaThread "FRONT_END michal:public (148)" [_thread_blocked, id=10296, stack(0x000000003c2f0000,0x000000003c3f0000)]
  0x0000000028e87800 JavaThread "LEADERBOARD michal:public (170)" [_thread_blocked, id=17420, stack(0x000000003bf10000,0x000000003c010000)]
  0x0000000028e8b800 JavaThread "FRONT_END michal:public (147)" [_thread_blocked, id=18880, stack(0x000000003bd80000,0x000000003be80000)]
  0x0000000028e90000 JavaThread "FRONT_END michal:public (146)" [_thread_blocked, id=16012, stack(0x000000003ba10000,0x000000003bb10000)]
  0x0000000028e93000 JavaThread "LEADERBOARD michal:public (169)" [_thread_blocked, id=23104, stack(0x000000003b270000,0x000000003b370000)]
  0x0000000028e8f000 JavaThread "FRONT_END michal:public (145)" [_thread_blocked, id=18152, stack(0x000000003ab70000,0x000000003ac70000)]
  0x0000000028e96000 JavaThread "FRONT_END michal:public (144)" [_thread_blocked, id=12016, stack(0x000000003a390000,0x000000003a490000)]
  0x0000000028e95000 JavaThread "LEADERBOARD michal:public (168)" [_thread_blocked, id=18200, stack(0x000000003b510000,0x000000003b610000)]
  0x0000000028e8d800 JavaThread "FRONT_END michal:public (143)" [_thread_blocked, id=19336, stack(0x000000003b3b0000,0x000000003b4b0000)]
  0x0000000028e8a800 JavaThread "FRONT_END michal:public (142)" [_thread_blocked, id=15488, stack(0x0000000035710000,0x0000000035810000)]
  0x0000000028e94800 JavaThread "LEADERBOARD michal:private (25)" [_thread_blocked, id=18168, stack(0x000000003b150000,0x000000003b250000)]
  0x0000000028e91800 JavaThread "FRIEND michal:public (13)" [_thread_blocked, id=17036, stack(0x000000003af70000,0x000000003b070000)]
  0x0000000028e92000 JavaThread "LEADERBOARD michal:public (167)" [_thread_blocked, id=12568, stack(0x0000000032d10000,0x0000000032e10000)]
  0x0000000028e88800 JavaThread "FRONT_END michal:public (141)" [_thread_blocked, id=11432, stack(0x000000003ae40000,0x000000003af40000)]
  0x0000000028e87000 JavaThread "FRONT_END michal:public (140)" [_thread_blocked, id=1972, stack(0x000000003aca0000,0x000000003ada0000)]
  0x0000000028e8c000 JavaThread "LEADERBOARD michal:public (166)" [_thread_blocked, id=20512, stack(0x0000000034f10000,0x0000000035010000)]
  0x0000000028e89000 JavaThread "FRONT_END michal:public (139)" [_thread_blocked, id=13864, stack(0x000000003aa50000,0x000000003ab50000)]
  0x0000000028e8d000 JavaThread "FRONT_END michal:public (138)" [_thread_blocked, id=5364, stack(0x0000000039e30000,0x0000000039f30000)]
  0x0000000028e8a000 JavaThread "LEADERBOARD michal:public (165)" [_thread_blocked, id=21524, stack(0x00000000396a0000,0x00000000397a0000)]
  0x000000002dcd7000 JavaThread "FRONT_END michal:public (137)" [_thread_blocked, id=22856, stack(0x0000000039d10000,0x0000000039e10000)]
  0x000000002dcc7000 JavaThread "FRONT_END michal:public (136)" [_thread_blocked, id=3252, stack(0x0000000039910000,0x0000000039a10000)]
  0x000000002dcce800 JavaThread "LEADERBOARD michal:public (164)" [_thread_blocked, id=11036, stack(0x00000000397b0000,0x00000000398b0000)]
  0x000000002dcc5800 JavaThread "FRONT_END michal:public (119)" [_thread_blocked, id=22060, stack(0x0000000033df0000,0x0000000033ef0000)]
  0x000000002dcc4000 JavaThread "FRONT_END michal:public (118)" [_thread_blocked, id=21292, stack(0x00000000342c0000,0x00000000343c0000)]
  0x000000002dcd8000 JavaThread "FRONT_END michal:public (117)" [_thread_blocked, id=12292, stack(0x0000000033600000,0x0000000033700000)]
  0x000000002dcd8800 JavaThread "FRONT_END michal:public (116)" [_thread_blocked, id=14640, stack(0x00000000334e0000,0x00000000335e0000)]
  0x000000002dcca000 JavaThread "FRONT_END michal:public (115)" [_thread_blocked, id=7988, stack(0x00000000323a0000,0x00000000324a0000)]
  0x000000002d0ed800 JavaThread "FRONT_END michal:public (114)" [_thread_blocked, id=12464, stack(0x0000000032260000,0x0000000032360000)]
  0x000000002d0f7800 JavaThread "FRONT_END michal:public (113)" [_thread_blocked, id=7280, stack(0x0000000031e70000,0x0000000031f70000)]
  0x000000002dcdf800 JavaThread "FRONT_END michal:public (112)" [_thread_blocked, id=21580, stack(0x000000002e500000,0x000000002e600000)]
  0x000000002dce1800 JavaThread "FRONT_END michal:public (111)" [_thread_blocked, id=7184, stack(0x0000000031760000,0x0000000031860000)]
  0x0000000036df3000 JavaThread "FRONT_END michal:public (110)" [_thread_blocked, id=22956, stack(0x0000000036530000,0x0000000036630000)]
  0x0000000036df8000 JavaThread "FRONT_END michal:public (109)" [_thread_blocked, id=20844, stack(0x0000000035d40000,0x0000000035e40000)]
  0x000000002dcd4000 JavaThread "FRONT_END michal:public (107)" [_thread_blocked, id=18804, stack(0x0000000032f20000,0x0000000033020000)]
  0x00000000293f2800 JavaThread "WEB_SERVANT michal:scheduler (16)" [_thread_blocked, id=3892, stack(0x00000000602e0000,0x00000000603e0000)]
  0x00000000293f2000 JavaThread "CHAT michal:scheduler (16)" [_thread_blocked, id=18904, stack(0x000000005fd80000,0x000000005fe80000)]
  0x00000000293f1000 JavaThread "FRONT_END michal:scheduler (16)" [_thread_in_native, id=23368, stack(0x00000000601e0000,0x00000000602e0000)]
  0x00000000293f0800 JavaThread "SERVANT michal:scheduler (16)" [_thread_blocked, id=2416, stack(0x000000005ffc0000,0x00000000600c0000)]
  0x00000000293ef800 JavaThread "LEADERBOARD michal:scheduler (16)" [_thread_in_native, id=6236, stack(0x000000005fe80000,0x000000005ff80000)]
  0x00000000293ef000 JavaThread "LOGIN michal:scheduler (16)" [_thread_blocked, id=21584, stack(0x000000005f040000,0x000000005f140000)]
  0x00000000293ee000 JavaThread "PLAYER_MESSAGING michal:scheduler (16)" [_thread_blocked, id=20760, stack(0x000000005d1f0000,0x000000005d2f0000)]
  0x00000000293ed800 JavaThread "CHAT michal:scheduler (15)" [_thread_blocked, id=8580, stack(0x000000005fc50000,0x000000005fd50000)]
  0x00000000293ec800 JavaThread "FRIEND michal:scheduler (16)" [_thread_blocked, id=10804, stack(0x000000005f4a0000,0x000000005f5a0000)]
  0x00000000293eb000 JavaThread "PROFILE michal:scheduler (16)" [_thread_blocked, id=9360, stack(0x000000005fa20000,0x000000005fb20000)]
  0x00000000293ea800 JavaThread "FRONT_END michal:scheduler (15)" [_thread_blocked, id=9324, stack(0x000000005f830000,0x000000005f930000)]
  0x00000000293e9800 JavaThread "WEB_SERVANT michal:scheduler (15)" [_thread_blocked, id=20944, stack(0x000000005f5d0000,0x000000005f6d0000)]
  0x00000000293e9000 JavaThread "CONNECTOR michal:scheduler (16)" [_thread_blocked, id=23044, stack(0x000000005f250000,0x000000005f350000)]
  0x00000000293e8000 JavaThread "SERVANT michal:scheduler (15)" [_thread_blocked, id=20628, stack(0x000000005f390000,0x000000005f490000)]
  0x00000000293e7800 JavaThread "LEADERBOARD michal:scheduler (15)" [_thread_blocked, id=19848, stack(0x000000005ea60000,0x000000005eb60000)]
  0x00000000293e6800 JavaThread "LOGIN michal:scheduler (15)" [_thread_blocked, id=5860, stack(0x000000005f140000,0x000000005f240000)]
  0x00000000293e6000 JavaThread "ACCOUNT michal:scheduler (16)" [_thread_blocked, id=476, stack(0x000000005ce20000,0x000000005cf20000)]
  0x00000000293e5000 JavaThread "PLAYER_MESSAGING michal:scheduler (15)" [_thread_blocked, id=11060, stack(0x000000005dc50000,0x000000005dd50000)]
  0x00000000293e4800 JavaThread "CHAT michal:scheduler (14)" [_thread_blocked, id=11008, stack(0x000000005ef40000,0x000000005f040000)]
  0x00000000293e3800 JavaThread "FRIEND michal:scheduler (15)" [_thread_blocked, id=6240, stack(0x000000005ed70000,0x000000005ee70000)]
  0x00000000293e3000 JavaThread "PROFILE michal:scheduler (15)" [_thread_blocked, id=3156, stack(0x000000005e030000,0x000000005e130000)]
  0x00000000293e2000 JavaThread "DISCOVERY michal:scheduler (16)" [_thread_blocked, id=20952, stack(0x000000005eb90000,0x000000005ec90000)]
  0x00000000293e1800 JavaThread "FRONT_END michal:scheduler (14)" [_thread_blocked, id=13624, stack(0x000000005e960000,0x000000005ea60000)]
  0x00000000293e0800 JavaThread "WEB_SERVANT michal:scheduler (14)" [_thread_blocked, id=15136, stack(0x000000005e7c0000,0x000000005e8c0000)]
  0x00000000293e0000 JavaThread "BOX_MONITOR michal:scheduler (16)" [_thread_blocked, id=4968, stack(0x000000005e670000,0x000000005e770000)]
  0x00000000293df000 JavaThread "CONNECTOR michal:scheduler (15)" [_thread_blocked, id=10908, stack(0x000000005da20000,0x000000005db20000)]
  0x00000000293de800 JavaThread "SERVANT michal:scheduler (14)" [_thread_blocked, id=21964, stack(0x000000005e4c0000,0x000000005e5c0000)]
  0x00000000293dd800 JavaThread "LEADERBOARD michal:scheduler (14)" [_thread_blocked, id=6828, stack(0x000000005e380000,0x000000005e480000)]
  0x00000000293dd000 JavaThread "LOGIN michal:scheduler (14)" [_thread_blocked, id=19488, stack(0x000000005e230000,0x000000005e330000)]
  0x00000000293dc000 JavaThread "ACCOUNT michal:scheduler (15)" [_thread_blocked, id=23396, stack(0x000000005e130000,0x000000005e230000)]
  0x00000000293db800 JavaThread "PLAYER_MESSAGING michal:scheduler (14)" [_thread_blocked, id=4596, stack(0x000000005df30000,0x000000005e030000)]
  0x00000000293da800 JavaThread "CHAT michal:scheduler (13)" [_thread_blocked, id=13260, stack(0x000000005dd60000,0x000000005de60000)]
  0x00000000293da000 JavaThread "FRIEND michal:scheduler (14)" [_thread_blocked, id=23340, stack(0x000000005db40000,0x000000005dc40000)]
  0x00000000293d9000 JavaThread "DISCOVERY michal:scheduler (15)" [_thread_in_native, id=18640, stack(0x000000005d8d0000,0x000000005d9d0000)]
  0x00000000293d8800 JavaThread "PROFILE michal:scheduler (14)" [_thread_blocked, id=21420, stack(0x000000005d7a0000,0x000000005d8a0000)]
  0x00000000293d7800 JavaThread "FRONT_END michal:scheduler (13)" [_thread_blocked, id=6104, stack(0x000000005d540000,0x000000005d640000)]
  0x000000002aa85800 JavaThread "WEB_SERVANT michal:scheduler (13)" [_thread_blocked, id=3280, stack(0x000000005d650000,0x000000005d750000)]
  0x000000002aa84800 JavaThread "BOX_MONITOR michal:scheduler (15)" [_thread_blocked, id=8224, stack(0x000000005d440000,0x000000005d540000)]
  0x000000002aa83800 JavaThread "CONNECTOR michal:scheduler (14)" [_thread_blocked, id=19248, stack(0x000000005cbe0000,0x000000005cce0000)]
  0x000000002aa83000 JavaThread "SERVANT michal:scheduler (13)" [_thread_blocked, id=9644, stack(0x000000005d300000,0x000000005d400000)]
  0x000000002aa82000 JavaThread "LEADERBOARD michal:scheduler (13)" [_thread_blocked, id=20576, stack(0x000000005d0f0000,0x000000005d1f0000)]
  0x000000002aa81800 JavaThread "LOGIN michal:scheduler (13)" [_thread_blocked, id=18780, stack(0x000000005cf30000,0x000000005d030000)]
  0x000000002aa80800 JavaThread "ACCOUNT michal:scheduler (14)" [_thread_blocked, id=12924, stack(0x000000005cd20000,0x000000005ce20000)]
  0x000000002aa80000 JavaThread "PLAYER_MESSAGING michal:scheduler (13)" [_thread_blocked, id=23364, stack(0x000000005ca70000,0x000000005cb70000)]
  0x000000002aa7f000 JavaThread "CHAT michal:scheduler (12)" [_thread_blocked, id=3300, stack(0x000000005c7d0000,0x000000005c8d0000)]
  0x000000002aa7e800 JavaThread "FRIEND michal:scheduler (13)" [_thread_in_native, id=21540, stack(0x000000005c5a0000,0x000000005c6a0000)]
  0x000000002aa7d800 JavaThread "DISCOVERY michal:scheduler (14)" [_thread_blocked, id=19160, stack(0x000000005c8e0000,0x000000005c9e0000)]
  0x000000002aa7d000 JavaThread "PROFILE michal:scheduler (13)" [_thread_in_native, id=4820, stack(0x000000005be30000,0x000000005bf30000)]
  0x000000002aa7c000 JavaThread "FRONT_END michal:scheduler (12)" [_thread_blocked, id=14320, stack(0x000000005c6c0000,0x000000005c7c0000)]
  0x000000002aa7b800 JavaThread "WEB_SERVANT michal:scheduler (12)" [_thread_in_native, id=12080, stack(0x000000005c490000,0x000000005c590000)]
  0x000000002aa7a800 JavaThread "BOX_MONITOR michal:scheduler (14)" [_thread_in_native, id=11356, stack(0x000000005c2f0000,0x000000005c3f0000)]
  0x000000002aa7a000 JavaThread "CONNECTOR michal:scheduler (13)" [_thread_blocked, id=5228, stack(0x000000005c150000,0x000000005c250000)]
  0x000000002aa79000 JavaThread "SERVANT michal:scheduler (12)" [_thread_in_native, id=20868, stack(0x000000005bfb0000,0x000000005c0b0000)]
  0x000000002aa78800 JavaThread "LEADERBOARD michal:scheduler (12)" [_thread_blocked, id=1508, stack(0x000000005bc30000,0x000000005bd30000)]
  0x000000002aa77800 JavaThread "LOGIN michal:scheduler (12)" [_thread_blocked, id=11928, stack(0x000000005bd30000,0x000000005be30000)]
  0x000000002aa77000 JavaThread "ACCOUNT michal:scheduler (13)" [_thread_blocked, id=5500, stack(0x000000005ba80000,0x000000005bb80000)]
  0x000000002aa76000 JavaThread "PLAYER_MESSAGING michal:scheduler (12)" [_thread_blocked, id=20024, stack(0x000000005b750000,0x000000005b850000)]
  0x000000002aa75800 JavaThread "CHAT michal:scheduler (11)" [_thread_blocked, id=4028, stack(0x000000005b8c0000,0x000000005b9c0000)]
  0x000000002aa74800 JavaThread "FRIEND michal:scheduler (12)" [_thread_blocked, id=23424, stack(0x000000005b640000,0x000000005b740000)]
  0x000000002aa74000 JavaThread "DISCOVERY michal:scheduler (13)" [_thread_blocked, id=21008, stack(0x000000005b320000,0x000000005b420000)]
  0x000000002aa73000 JavaThread "PROFILE michal:scheduler (12)" [_thread_blocked, id=22344, stack(0x000000005b460000,0x000000005b560000)]
  0x000000002aa72800 JavaThread "FRONT_END michal:scheduler (11)" [_thread_blocked, id=5408, stack(0x000000005b170000,0x000000005b270000)]
  0x000000002aa71800 JavaThread "WEB_SERVANT michal:scheduler (11)" [_thread_blocked, id=22576, stack(0x000000005b070000,0x000000005b170000)]
  0x000000002aa71000 JavaThread "BOX_MONITOR michal:scheduler (13)" [_thread_blocked, id=10912, stack(0x0000000059360000,0x0000000059460000)]
  0x000000002aa70000 JavaThread "CONNECTOR michal:scheduler (12)" [_thread_blocked, id=23156, stack(0x000000005af00000,0x000000005b000000)]
  0x000000002aa6f800 JavaThread "SERVANT michal:scheduler (11)" [_thread_blocked, id=1960, stack(0x000000005adb0000,0x000000005aeb0000)]
  0x000000002aa6e800 JavaThread "LEADERBOARD michal:scheduler (11)" [_thread_blocked, id=15356, stack(0x000000005abb0000,0x000000005acb0000)]
  0x000000002aa6e000 JavaThread "LOGIN michal:scheduler (11)" [_thread_blocked, id=5436, stack(0x000000005a910000,0x000000005aa10000)]
  0x000000002aa6d000 JavaThread "ACCOUNT michal:scheduler (12)" [_thread_blocked, id=13620, stack(0x000000005acb0000,0x000000005adb0000)]
  0x000000002aa6c800 JavaThread "PLAYER_MESSAGING michal:scheduler (11)" [_thread_in_native, id=13092, stack(0x000000005aa70000,0x000000005ab70000)]
  0x000000002aa6b800 JavaThread "CHAT michal:scheduler (10)" [_thread_blocked, id=9380, stack(0x000000005a0f0000,0x000000005a1f0000)]
  0x000000002aa6b000 JavaThread "FRIEND michal:scheduler (11)" [_thread_blocked, id=17548, stack(0x000000005a7f0000,0x000000005a8f0000)]
  0x000000002aa6a000 JavaThread "DISCOVERY michal:scheduler (12)" [_thread_blocked, id=16916, stack(0x0000000058a70000,0x0000000058b70000)]
  0x000000002aa69800 JavaThread "PROFILE michal:scheduler (11)" [_thread_blocked, id=19344, stack(0x000000005a670000,0x000000005a770000)]
  0x000000002aa68800 JavaThread "FRONT_END michal:scheduler (10)" [_thread_blocked, id=18548, stack(0x0000000059e80000,0x0000000059f80000)]
  0x000000002aa68000 JavaThread "WEB_SERVANT michal:scheduler (10)" [_thread_blocked, id=7616, stack(0x000000005a530000,0x000000005a630000)]
  0x000000002aa67000 JavaThread "BOX_MONITOR michal:scheduler (12)" [_thread_blocked, id=19036, stack(0x000000005a380000,0x000000005a480000)]
  0x000000002aa66800 JavaThread "CONNECTOR michal:scheduler (11)" [_thread_blocked, id=20784, stack(0x000000005a200000,0x000000005a300000)]
  0x000000002ddf3000 JavaThread "CONNECTOR michal:scheduler (10)" [_thread_blocked, id=10404, stack(0x0000000059fa0000,0x000000005a0a0000)]
  0x000000002ddf2000 JavaThread "SERVANT michal:scheduler (10)" [_thread_blocked, id=7380, stack(0x0000000059d80000,0x0000000059e80000)]
  0x000000002ddf1800 JavaThread "GAME_LOGGER michal:scheduler (16)" [_thread_blocked, id=18312, stack(0x0000000059b90000,0x0000000059c90000)]
  0x000000002ddf0800 JavaThread "GAME_LOGGER michal:scheduler (15)" [_thread_blocked, id=13292, stack(0x0000000059830000,0x0000000059930000)]
  0x000000002ddf0000 JavaThread "LEADERBOARD michal:scheduler (10)" [_thread_blocked, id=2356, stack(0x00000000599a0000,0x0000000059aa0000)]
  0x000000002ddef000 JavaThread "LOGIN michal:scheduler (10)" [_thread_blocked, id=21820, stack(0x0000000058fb0000,0x00000000590b0000)]
  0x000000002ddee800 JavaThread "ACCOUNT michal:scheduler (11)" [_thread_blocked, id=9880, stack(0x0000000059730000,0x0000000059830000)]
  0x000000002dded800 JavaThread "PLAYER_MESSAGING michal:scheduler (10)" [_thread_blocked, id=12320, stack(0x0000000059610000,0x0000000059710000)]
  0x000000002dded000 JavaThread "CHAT michal:scheduler (9)" [_thread_in_native, id=16284, stack(0x0000000059470000,0x0000000059570000)]
  0x000000002ddec000 JavaThread "FRIEND michal:scheduler (10)" [_thread_blocked, id=6288, stack(0x0000000059260000,0x0000000059360000)]
  0x000000002ddeb800 JavaThread "DISCOVERY michal:scheduler (11)" [_thread_blocked, id=10648, stack(0x0000000056a50000,0x0000000056b50000)]
  0x000000002ddea800 JavaThread "PROFILE michal:scheduler (10)" [_thread_blocked, id=8032, stack(0x0000000059100000,0x0000000059200000)]
  0x000000002ddea000 JavaThread "FRONT_END michal:scheduler (9)" [_thread_blocked, id=16072, stack(0x0000000058e90000,0x0000000058f90000)]
  0x000000002dde9000 JavaThread "WEB_SERVANT michal:scheduler (9)" [_thread_blocked, id=1176, stack(0x0000000058ca0000,0x0000000058da0000)]
  0x000000002dde8800 JavaThread "BOX_MONITOR michal:scheduler (11)" [_thread_blocked, id=22516, stack(0x0000000058ba0000,0x0000000058ca0000)]
  0x000000002dde7800 JavaThread "CONNECTOR michal:scheduler (9)" [_thread_blocked, id=23192, stack(0x0000000058970000,0x0000000058a70000)]
  0x000000002dde7000 JavaThread "SERVANT michal:scheduler (9)" [_thread_blocked, id=11500, stack(0x0000000058820000,0x0000000058920000)]
  0x000000002dde6000 JavaThread "GAME_LOGGER michal:scheduler (14)" [_thread_blocked, id=19396, stack(0x0000000057640000,0x0000000057740000)]
  0x000000002dde5800 JavaThread "GAME_LOGGER michal:scheduler (13)" [_thread_blocked, id=22016, stack(0x00000000586d0000,0x00000000587d0000)]
  0x000000002dde4800 JavaThread "LEADERBOARD michal:scheduler (9)" [_thread_blocked, id=7588, stack(0x0000000058500000,0x0000000058600000)]
  0x000000002dde4000 JavaThread "LOGIN michal:scheduler (9)" [_thread_blocked, id=12372, stack(0x00000000583f0000,0x00000000584f0000)]
  0x0000000036e11000 JavaThread "ACCOUNT michal:scheduler (10)" [_thread_blocked, id=11468, stack(0x0000000058250000,0x0000000058350000)]
  0x0000000036e10800 JavaThread "PLAYER_MESSAGING michal:scheduler (9)" [_thread_blocked, id=20064, stack(0x0000000058150000,0x0000000058250000)]
  0x0000000036e0f800 JavaThread "CHAT michal:scheduler (8)" [_thread_blocked, id=18824, stack(0x0000000057ff0000,0x00000000580f0000)]
  0x0000000036e0f000 JavaThread "FRIEND michal:scheduler (9)" [_thread_blocked, id=14696, stack(0x0000000057e40000,0x0000000057f40000)]
  0x0000000036e0e000 JavaThread "DISCOVERY michal:scheduler (10)" [_thread_blocked, id=17096, stack(0x0000000057c50000,0x0000000057d50000)]
  0x0000000036e0d800 JavaThread "PROFILE michal:scheduler (9)" [_thread_blocked, id=22200, stack(0x0000000057af0000,0x0000000057bf0000)]
  0x0000000036e0c800 JavaThread "FRONT_END michal:scheduler (8)" [_thread_blocked, id=21060, stack(0x00000000577d0000,0x00000000578d0000)]
  0x0000000036e0c000 JavaThread "WEB_SERVANT michal:scheduler (8)" [_thread_blocked, id=17204, stack(0x0000000056cf0000,0x0000000056df0000)]
  0x0000000036e0b000 JavaThread "BOX_MONITOR michal:scheduler (10)" [_thread_blocked, id=16064, stack(0x0000000056b50000,0x0000000056c50000)]
  0x0000000036e0a800 JavaThread "CONNECTOR michal:scheduler (8)" [_thread_in_native, id=4788, stack(0x0000000057980000,0x0000000057a80000)]
  0x0000000036e09800 JavaThread "SERVANT michal:scheduler (8)" [_thread_blocked, id=8560, stack(0x00000000573a0000,0x00000000574a0000)]
  0x0000000036e09000 JavaThread "LOGIN michal:scheduler (8)" [_thread_blocked, id=8152, stack(0x0000000057530000,0x0000000057630000)]
  0x0000000036e08000 JavaThread "LEADERBOARD michal:scheduler (8)" [_thread_blocked, id=5612, stack(0x0000000057270000,0x0000000057370000)]
  0x0000000036e07800 JavaThread "GAME_LOGGER michal:scheduler (12)" [_thread_in_native, id=14172, stack(0x0000000056850000,0x0000000056950000)]
  0x0000000036e06800 JavaThread "GAME_LOGGER michal:scheduler (11)" [_thread_blocked, id=5324, stack(0x0000000056ec0000,0x0000000056fc0000)]
  0x0000000036e06000 JavaThread "ACCOUNT michal:scheduler (9)" [_thread_blocked, id=14536, stack(0x0000000056950000,0x0000000056a50000)]
  0x0000000036dfa800 JavaThread "FRIEND michal:scheduler (8)" [_thread_blocked, id=17444, stack(0x0000000054e50000,0x0000000054f50000)]
  0x0000000036df9800 JavaThread "FRIEND michal:scheduler (7)" [_thread_blocked, id=20152, stack(0x0000000054cb0000,0x0000000054db0000)]
  0x0000000036df4800 JavaThread "PROFILE michal:scheduler (8)" [_thread_blocked, id=12904, stack(0x00000000542a0000,0x00000000543a0000)]
  0x0000000036df3800 JavaThread "PROFILE michal:scheduler (7)" [_thread_blocked, id=18108, stack(0x0000000054120000,0x0000000054220000)]
  0x0000000045e22800 JavaThread "PLAYER_MESSAGING michal:scheduler (8)" [_thread_blocked, id=5880, stack(0x0000000051090000,0x0000000051190000)]
  0x0000000045e21800 JavaThread "CHAT michal:scheduler (7)" [_thread_blocked, id=5624, stack(0x0000000052830000,0x0000000052930000)]
  0x0000000045e20000 JavaThread "WEB_SERVANT michal:scheduler (7)" [_thread_blocked, id=8624, stack(0x0000000052560000,0x0000000052660000)]
  0x0000000045e1e800 JavaThread "WEB_SERVANT michal:scheduler (6)" [_thread_blocked, id=13756, stack(0x00000000521f0000,0x00000000522f0000)]
  0x0000000045e1a000 JavaThread "FRIEND michal:scheduler (6)" [_thread_blocked, id=11964, stack(0x0000000050690000,0x0000000050790000)]
  0x0000000045e19800 JavaThread "DISCOVERY michal:scheduler (9)" [_thread_blocked, id=11084, stack(0x00000000504c0000,0x00000000505c0000)]
  0x0000000045e18800 JavaThread "PROFILE michal:scheduler (6)" [_thread_blocked, id=18176, stack(0x000000004d600000,0x000000004d700000)]
  0x0000000045e18000 JavaThread "FRONT_END michal:scheduler (7)" [_thread_blocked, id=23132, stack(0x000000004d420000,0x000000004d520000)]
  0x0000000045e17000 JavaThread "WEB_SERVANT michal:scheduler (5)" [_thread_blocked, id=20508, stack(0x0000000031320000,0x0000000031420000)]
  0x0000000045e16800 JavaThread "BOX_MONITOR michal:scheduler (9)" [_thread_blocked, id=22368, stack(0x00000000311d0000,0x00000000312d0000)]
  0x0000000045e15800 JavaThread "CONNECTOR michal:scheduler (7)" [_thread_blocked, id=12624, stack(0x0000000031560000,0x0000000031660000)]
  0x0000000045e15000 JavaThread "SERVANT michal:scheduler (7)" [_thread_blocked, id=5644, stack(0x0000000050cb0000,0x0000000050db0000)]
  0x0000000045e14000 JavaThread "GAME_LOGGER michal:scheduler (10)" [_thread_blocked, id=18520, stack(0x000000004fea0000,0x000000004ffa0000)]
  0x0000000045e13800 JavaThread "GAME_LOGGER michal:scheduler (9)" [_thread_blocked, id=9004, stack(0x000000004fc80000,0x000000004fd80000)]
  0x0000000045e12800 JavaThread "LEADERBOARD michal:scheduler (7)" [_thread_blocked, id=18700, stack(0x0000000050a70000,0x0000000050b70000)]
  0x0000000045e12000 JavaThread "LOGIN michal:scheduler (7)" [_thread_blocked, id=16984, stack(0x0000000050910000,0x0000000050a10000)]
  0x0000000045e11000 JavaThread "ACCOUNT michal:scheduler (8)" [_thread_blocked, id=12056, stack(0x000000004c640000,0x000000004c740000)]
  0x0000000045e0f800 JavaThread "PLAYER_MESSAGING michal:scheduler (7)" [_thread_blocked, id=22520, stack(0x0000000050100000,0x0000000050200000)]
  0x0000000028e93800 JavaThread "CHAT michal:scheduler (6)" [_thread_blocked, id=6148, stack(0x000000004fb10000,0x000000004fc10000)]
  0x0000000028e8e800 JavaThread "FRIEND michal:scheduler (5)" [_thread_blocked, id=12688, stack(0x000000004f1e0000,0x000000004f2e0000)]
  0x000000002b87a000 JavaThread "FRIEND michal:scheduler (4)" [_thread_blocked, id=18308, stack(0x000000004d2c0000,0x000000004d3c0000)]
  0x000000002b879000 JavaThread "FRIEND michal:scheduler (3)" [_thread_blocked, id=1312, stack(0x000000004d0c0000,0x000000004d1c0000)]
  0x000000002b878800 JavaThread "FRIEND michal:scheduler (2)" [_thread_blocked, id=8184, stack(0x000000004d1c0000,0x000000004d2c0000)]
  0x000000002b877800 JavaThread "FRIEND michal:scheduler (1)" [_thread_blocked, id=22004, stack(0x000000004cf20000,0x000000004d020000)]
  0x000000002b876000 JavaThread "DISCOVERY michal:scheduler (8)" [_thread_blocked, id=20656, stack(0x000000004ce00000,0x000000004cf00000)]
  0x000000002b874000 JavaThread "FRIEND michal:network" [_thread_in_native, id=15764, stack(0x000000004ca10000,0x000000004cb10000)]
  0x000000002b88f800 JavaThread "PROFILE michal:scheduler (4)" [_thread_blocked, id=22496, stack(0x000000004bfc0000,0x000000004c0c0000)]
  0x000000002b88d800 JavaThread "PROFILE michal:scheduler (5)" [_thread_blocked, id=19948, stack(0x000000004bdf0000,0x000000004bef0000)]
  0x000000002b88c000 JavaThread "PROFILE michal:scheduler (3)" [_thread_blocked, id=18008, stack(0x000000004bb80000,0x000000004bc80000)]
  0x000000002b88b000 JavaThread "PROFILE michal:scheduler (2)" [_thread_blocked, id=17984, stack(0x000000004b950000,0x000000004ba50000)]
  0x000000002b889000 JavaThread "LEADERBOARD michal:scheduler (6)" [_thread_blocked, id=8324, stack(0x000000004b470000,0x000000004b570000)]
  0x000000002b886800 JavaThread "FRONT_END michal:scheduler (6)" [_thread_blocked, id=1852, stack(0x000000004afe0000,0x000000004b0e0000)]
  0x000000002b886000 JavaThread "LEADERBOARD michal:scheduler (5)" [_thread_blocked, id=21320, stack(0x000000004b0e0000,0x000000004b1e0000)]
  0x000000002b885000 JavaThread "GAME_LOGGER michal:scheduler (8)" [_thread_blocked, id=22708, stack(0x000000004aeb0000,0x000000004afb0000)]
  0x000000002b883000 JavaThread "GAME_LOGGER michal:scheduler (7)" [_thread_blocked, id=12136, stack(0x000000004a5b0000,0x000000004a6b0000)]
  0x000000002d0f9800 JavaThread "PROFILE michal:scheduler (1)" [_thread_blocked, id=15472, stack(0x000000004a4b0000,0x000000004a5b0000)]
  0x000000002d0f5000 JavaThread "PROFILE michal:network" [_thread_in_native, id=17980, stack(0x0000000049e70000,0x0000000049f70000)]
  0x000000002c34f000 JavaThread "WEB_SERVANT michal:scheduler (4)" [_thread_blocked, id=21500, stack(0x0000000047cc0000,0x0000000047dc0000)]
  0x000000002c34e000 JavaThread "WEB_SERVANT michal:scheduler (3)" [_thread_blocked, id=19956, stack(0x0000000047ac0000,0x0000000047bc0000)]
  0x000000002c34c800 JavaThread "WEB_SERVANT michal:scheduler (2)" [_thread_blocked, id=2868, stack(0x0000000047990000,0x0000000047a90000)]
  0x000000002c34c000 JavaThread "WEB_SERVANT michal:scheduler (1)" [_thread_blocked, id=22780, stack(0x0000000047780000,0x0000000047880000)]
  0x000000002c34b000 JavaThread "WEB_SERVANT michal:web (10)-acceptor-0@1977df46-publicPort@24be2d9c{SSL,[ssl, http/1.1]}{0.0.0.0:7711}" [_thread_in_native, id=7716, stack(0x0000000047610000,0x0000000047710000)]
  0x000000002c34a800 JavaThread "WEB_SERVANT michal:web (9)" [_thread_in_native, id=13400, stack(0x0000000047460000,0x0000000047560000)]
  0x000000002dd8c000 JavaThread "WEB_SERVANT michal:web (8)" [_thread_in_native, id=5352, stack(0x0000000047270000,0x0000000047370000)]
  0x000000002dd8b000 JavaThread "WEB_SERVANT michal:web (7)-lowPrioritySelector" [_thread_blocked, id=17540, stack(0x00000000470b0000,0x00000000471b0000)]
  0x000000002dd8a800 JavaThread "WEB_SERVANT michal:web (6)-lowPrioritySelector" [_thread_blocked, id=3364, stack(0x0000000046e50000,0x0000000046f50000)]
  0x000000002dd89800 JavaThread "WEB_SERVANT michal:web (5)-acceptor-0@311c2533-trustedPort@40f33492{SSL,[ssl, http/1.1]}{0.0.0.0:7708}" [_thread_in_native, id=7424, stack(0x000000002e040000,0x000000002e140000)]
  0x000000002dd89000 JavaThread "WEB_SERVANT michal:web (4)" [_thread_in_native, id=2588, stack(0x00000000458a0000,0x00000000459a0000)]
  0x000000002dd88000 JavaThread "WEB_SERVANT michal:web (3)-lowPrioritySelector" [_thread_blocked, id=15828, stack(0x0000000046c20000,0x0000000046d20000)]
  0x000000002dd87800 JavaThread "WEB_SERVANT michal:web (2)-lowPrioritySelector" [_thread_blocked, id=19080, stack(0x0000000046ab0000,0x0000000046bb0000)]
  0x000000002dd86800 JavaThread "WEB_SERVANT michal:web (1)" [_thread_in_native, id=18600, stack(0x0000000045600000,0x0000000045700000)]
  0x000000002dd86000 JavaThread "BOX_MONITOR michal:scheduler (8)" [_thread_blocked, id=7404, stack(0x0000000045790000,0x0000000045890000)]
  0x000000002dd85000 JavaThread "CONNECTOR michal:scheduler (6)" [_thread_blocked, id=16288, stack(0x0000000044e90000,0x0000000044f90000)]
  0x000000002dcf4000 JavaThread "PLAYER_MESSAGING michal:scheduler (6)" [_thread_blocked, id=16348, stack(0x0000000044f90000,0x0000000045090000)]
  0x000000002dcf2000 JavaThread "ACCOUNT michal:scheduler (7)" [_thread_blocked, id=14256, stack(0x00000000443e0000,0x00000000444e0000)]
  0x000000002dcf1000 JavaThread "ACCOUNT michal:scheduler (6)" [_thread_in_native, id=12108, stack(0x0000000044990000,0x0000000044a90000)]
  0x000000002dced800 JavaThread "PLAYER_MESSAGING michal:scheduler (5)" [_thread_blocked, id=13200, stack(0x0000000044090000,0x0000000044190000)]
  0x000000002dce9800 JavaThread "LOGIN michal:scheduler (6)" [_thread_blocked, id=13008, stack(0x0000000043a40000,0x0000000043b40000)]
  0x000000002dce9000 JavaThread "CHAT michal:scheduler (5)" [_thread_blocked, id=12988, stack(0x0000000043b90000,0x0000000043c90000)]
  0x000000002d4f0000 JavaThread "BOX_MONITOR michal:scheduler (7)" [_thread_blocked, id=15456, stack(0x00000000433b0000,0x00000000434b0000)]
  0x000000002d4eb800 JavaThread "DISCOVERY michal:scheduler (7)" [_thread_blocked, id=23148, stack(0x0000000042d00000,0x0000000042e00000)]
  0x000000002d4ea000 JavaThread "LOGIN michal:scheduler (5)" [_thread_blocked, id=3636, stack(0x00000000429d0000,0x0000000042ad0000)]
  0x000000002d4e4000 JavaThread "CHAT michal:scheduler (4)" [_thread_blocked, id=17920, stack(0x0000000041d20000,0x0000000041e20000)]
  0x000000002d4e2800 JavaThread "DISCOVERY michal:scheduler (6)" [_thread_blocked, id=15800, stack(0x0000000041b90000,0x0000000041c90000)]
  0x000000002dd73000 JavaThread "BOX_MONITOR michal:scheduler (6)" [_thread_blocked, id=6600, stack(0x0000000041510000,0x0000000041610000)]
  0x0000000036d20000 JavaThread "FRONT_END michal:scheduler (5)" [_thread_blocked, id=14068, stack(0x000000003f1a0000,0x000000003f2a0000)]
  0x0000000036d1e000 JavaThread "FRONT_END michal:scheduler (4)" [_thread_blocked, id=7236, stack(0x000000003eda0000,0x000000003eea0000)]
  0x000000002dd4e800 JavaThread "SERVANT michal:scheduler (6)" [_thread_blocked, id=1696, stack(0x000000003d5c0000,0x000000003d6c0000)]
  0x000000002ddfa800 JavaThread "SERVANT michal:scheduler (5)" [_thread_blocked, id=14124, stack(0x000000003cd50000,0x000000003ce50000)]
  0x000000002ddfa000 JavaThread "SERVANT michal:scheduler (4)" [_thread_blocked, id=18228, stack(0x000000003cb50000,0x000000003cc50000)]
  0x000000002ddf9000 JavaThread "WEB_SERVANT michal:public (4)-lowPrioritySelector" [_thread_blocked, id=10116, stack(0x000000003bc00000,0x000000003bd00000)]
  0x000000002ddf8800 JavaThread "WEB_SERVANT michal:public (3)" [_thread_in_native, id=18696, stack(0x000000003c8f0000,0x000000003c9f0000)]
  0x000000002ddf7800 JavaThread "WEB_SERVANT michal:public (2)-lowPrioritySelector" [_thread_in_native, id=23460, stack(0x000000003c7b0000,0x000000003c8b0000)]
  0x000000002ddf7000 JavaThread "WEB_SERVANT michal:public (1)" [_thread_blocked, id=18140, stack(0x000000003c630000,0x000000003c730000)]
  0x000000002ddf5800 JavaThread "WEB_SERVANT michal:network" [_thread_in_native, id=3412, stack(0x000000003c130000,0x000000003c230000)]
  0x000000002ddb6800 JavaThread "SERVANT michal:scheduler (3)" [_thread_blocked, id=18644, stack(0x0000000039aa0000,0x0000000039ba0000)]
  0x000000002ddb5800 JavaThread "SERVANT michal:scheduler (2)" [_thread_blocked, id=18464, stack(0x000000003b900000,0x000000003ba00000)]
  0x000000002ddb5000 JavaThread "SERVANT michal:scheduler (1)" [_thread_blocked, id=11224, stack(0x000000003b7c0000,0x000000003b8c0000)]
  0x000000002ddb4000 JavaThread "SERVANT michal:network" [_thread_in_native, id=14148, stack(0x000000003b620000,0x000000003b720000)]
  0x000000002d789000 JavaThread "GAME_LOGGER michal:scheduler (6)" [_thread_blocked, id=19172, stack(0x000000003a900000,0x000000003aa00000)]
  0x000000002d788800 JavaThread "GAME_LOGGER michal:scheduler (5)" [_thread_blocked, id=7908, stack(0x000000003a760000,0x000000003a860000)]
  0x000000002d787800 JavaThread "GAME_LOGGER michal:scheduler (4)" [_thread_blocked, id=17908, stack(0x000000003a620000,0x000000003a720000)]
  0x000000002d787000 JavaThread "GAME_LOGGER michal:scheduler (3)" [_thread_blocked, id=2220, stack(0x000000003a4a0000,0x000000003a5a0000)]
  0x000000002d786000 JavaThread "GAME_LOGGER michal:scheduler (2)" [_thread_blocked, id=20304, stack(0x000000003a290000,0x000000003a390000)]
  0x000000002d785800 JavaThread "GAME_LOGGER michal:scheduler (1)" [_thread_blocked, id=10552, stack(0x000000003a130000,0x000000003a230000)]
  0x000000002d784000 JavaThread "GAME_LOGGER michal:network" [_thread_in_native, id=21212, stack(0x0000000039fb0000,0x000000003a0b0000)]
  0x000000002d782800 JavaThread "LEADERBOARD michal:scheduler (4)" [_thread_blocked, id=8820, stack(0x0000000039ba0000,0x0000000039ca0000)]
  0x000000002dda7000 JavaThread "LEADERBOARD michal:scheduler (3)" [_thread_blocked, id=10508, stack(0x0000000038fa0000,0x00000000390a0000)]
  0x000000002dda6000 JavaThread "LEADERBOARD michal:scheduler (2)" [_thread_blocked, id=14744, stack(0x0000000038dc0000,0x0000000038ec0000)]
  0x000000002d1e4800 JavaThread "LEADERBOARD michal:scheduler (1)" [_thread_blocked, id=8288, stack(0x0000000038b00000,0x0000000038c00000)]
  0x000000002d1e4000 JavaThread "LEADERBOARD michal:network" [_thread_in_native, id=9692, stack(0x00000000389a0000,0x0000000038aa0000)]
  0x000000002cfeb800 JavaThread "LOGIN michal:scheduler (4)" [_thread_in_native, id=20036, stack(0x0000000037eb0000,0x0000000037fb0000)]
  0x000000002cfea000 JavaThread "LOGIN michal:scheduler (3)" [_thread_blocked, id=19272, stack(0x0000000037c50000,0x0000000037d50000)]
  0x000000002cfe9800 JavaThread "LOGIN michal:scheduler (2)" [_thread_blocked, id=12552, stack(0x0000000037ab0000,0x0000000037bb0000)]
  0x000000002cfe8800 JavaThread "LOGIN michal:public (4)-lowPrioritySelector" [_thread_blocked, id=22548, stack(0x00000000367e0000,0x00000000368e0000)]
  0x000000002cfe8000 JavaThread "LOGIN michal:public (3)" [_thread_in_native, id=23264, stack(0x0000000036660000,0x0000000036760000)]
  0x000000002cfe7000 JavaThread "LOGIN michal:public (2)-lowPrioritySelector" [_thread_blocked, id=7680, stack(0x0000000036280000,0x0000000036380000)]
  0x000000002cfe6800 JavaThread "LOGIN michal:public (1)" [_thread_in_native, id=5892, stack(0x0000000035e90000,0x0000000035f90000)]
  0x000000002cfe5800 JavaThread "LOGIN michal:scheduler (1)" [_thread_blocked, id=21064, stack(0x00000000363f0000,0x00000000364f0000)]
  0x000000002dde1000 JavaThread "LOGIN michal:network" [_thread_in_native, id=22924, stack(0x00000000355e0000,0x00000000356e0000)]
  0x000000002dddf800 JavaThread "ACCOUNT michal:scheduler (5)" [_thread_blocked, id=5780, stack(0x0000000034cb0000,0x0000000034db0000)]
  0x000000002dddc000 JavaThread "ACCOUNT michal:scheduler (4)" [_thread_blocked, id=9548, stack(0x0000000035360000,0x0000000035460000)]
  0x000000002d30b000 JavaThread "ACCOUNT michal:scheduler (3)" [_thread_blocked, id=15492, stack(0x00000000351b0000,0x00000000352b0000)]
  0x000000002d30a000 JavaThread "ACCOUNT michal:scheduler (2)" [_thread_blocked, id=19020, stack(0x0000000035010000,0x0000000035110000)]
  0x000000002d309800 JavaThread "ACCOUNT michal:scheduler (1)" [_thread_blocked, id=21740, stack(0x0000000034e10000,0x0000000034f10000)]
  0x000000002d308800 JavaThread "ACCOUNT michal:network" [_thread_in_native, id=17384, stack(0x0000000033f00000,0x0000000034000000)]
  0x000000002d306800 JavaThread "PLAYER_MESSAGING michal:scheduler (4)" [_thread_blocked, id=7948, stack(0x0000000034a20000,0x0000000034b20000)]
  0x000000002d304000 JavaThread "PLAYER_MESSAGING michal:scheduler (3)" [_thread_blocked, id=9748, stack(0x0000000034640000,0x0000000034740000)]
  0x000000002d2f1800 JavaThread "PLAYER_MESSAGING michal:scheduler (2)" [_thread_blocked, id=17492, stack(0x0000000033c90000,0x0000000033d90000)]
  0x000000002d2f1000 JavaThread "PLAYER_MESSAGING michal:scheduler (1)" [_thread_blocked, id=5700, stack(0x0000000033240000,0x0000000033340000)]
  0x000000002d2f0000 JavaThread "PLAYER_MESSAGING michal:network" [_thread_in_native, id=4104, stack(0x0000000033b40000,0x0000000033c40000)]
  0x00000000296ca800 JavaThread "CHAT michal:scheduler (3)" [_thread_blocked, id=3964, stack(0x0000000033730000,0x0000000033830000)]
  0x00000000296c9000 JavaThread "CHAT michal:scheduler (2)" [_thread_blocked, id=3116, stack(0x0000000032c10000,0x0000000032d10000)]
  0x00000000296c8800 JavaThread "CHAT michal:scheduler (1)" [_thread_blocked, id=15024, stack(0x0000000033340000,0x0000000033440000)]
  0x00000000296c7800 JavaThread "DISCOVERY michal:scheduler (5)" [_thread_blocked, id=8388, stack(0x0000000033130000,0x0000000033230000)]
  0x00000000296c4800 JavaThread "CHAT michal:network" [_thread_in_native, id=20828, stack(0x0000000032b10000,0x0000000032c10000)]
  0x000000002c3b3800 JavaThread "DISCOVERY michal:scheduler (4)" [_thread_blocked, id=9240, stack(0x0000000032840000,0x0000000032940000)]
  0x000000002c3b1000 JavaThread "DISCOVERY michal:scheduler (3)" [_thread_blocked, id=6520, stack(0x0000000032560000,0x0000000032660000)]
  0x000000002c3af800 JavaThread "DISCOVERY michal:scheduler (2)" [_thread_blocked, id=12012, stack(0x0000000032130000,0x0000000032230000)]
  0x000000002c3af000 JavaThread "DISCOVERY michal:scheduler (1)" [_thread_blocked, id=19416, stack(0x0000000031d70000,0x0000000031e70000)]
  0x000000002b8b6800 JavaThread "FRONT_END michal:scheduler (3)" [_thread_blocked, id=17812, stack(0x00000000318f0000,0x00000000319f0000)]
  0x000000002b8b6000 JavaThread "DISCOVERY michal:network" [_thread_in_native, id=9784, stack(0x0000000031a00000,0x0000000031b00000)]
  0x000000002b8b4800 JavaThread "FRONT_END michal:scheduler (2)" [_thread_blocked, id=3760, stack(0x000000002e2f0000,0x000000002e3f0000)]
  0x000000002b8b3800 JavaThread "FRONT_END michal:scheduler (1)" [_thread_blocked, id=11204, stack(0x0000000031660000,0x0000000031760000)]
  0x000000002b8b3000 JavaThread "FRONT_END michal:web (6)" [_thread_blocked, id=12764, stack(0x0000000030d60000,0x0000000030e60000)]
  0x000000002b8b2000 JavaThread "FRONT_END michal:web (5)-acceptor-0@3900020d-default@37c7eb5d{SSL,[ssl, http/1.1]}{0.0.0.0:7712}" [_thread_in_native, id=22584, stack(0x0000000030c30000,0x0000000030d30000)]
  0x000000002b8b1800 JavaThread "FRONT_END michal:web (4)-lowPrioritySelector" [_thread_blocked, id=2852, stack(0x0000000030760000,0x0000000030860000)]
  0x000000002b8b0800 JavaThread "FRONT_END michal:web (3)" [_thread_in_native, id=5504, stack(0x000000002e160000,0x000000002e260000)]
  0x000000002b8b0000 JavaThread "FRONT_END michal:web (2)-lowPrioritySelector" [_thread_blocked, id=10056, stack(0x0000000030af0000,0x0000000030bf0000)]
  0x000000002d17a000 JavaThread "BOX_MONITOR michal:scheduler (5)" [_thread_blocked, id=22104, stack(0x0000000030ff0000,0x00000000310f0000)]
  0x000000002d179800 JavaThread "FRONT_END michal:web (1)" [_thread_in_native, id=15568, stack(0x0000000030ed0000,0x0000000030fd0000)]
  0x000000002d178800 JavaThread "FRONT_END michal:network" [_thread_in_native, id=5200, stack(0x000000002f8c0000,0x000000002f9c0000)]
  0x000000002d175800 JavaThread "BOX_MONITOR michal:scheduler (4)" [_thread_blocked, id=21180, stack(0x000000002c7d0000,0x000000002c8d0000)]
  0x000000002d175000 JavaThread "BOX_MONITOR michal:scheduler (3)" [_thread_blocked, id=14588, stack(0x000000002efc0000,0x000000002f0c0000)]
  0x000000002d174000 JavaThread "BOX_MONITOR michal:scheduler (2)" [_thread_blocked, id=8340, stack(0x000000002f3b0000,0x000000002f4b0000)]
  0x000000002d173800 JavaThread "BOX_MONITOR michal:scheduler (1)" [_thread_blocked, id=21756, stack(0x000000002f2b0000,0x000000002f3b0000)]
  0x0000000029450800 JavaThread "BOX_MONITOR michal:network" [_thread_in_native, id=17804, stack(0x000000002f0e0000,0x000000002f1e0000)]
  0x0000000029450000 JavaThread "CONNECTOR michal:scheduler (5)" [_thread_blocked, id=14956, stack(0x000000002ed80000,0x000000002ee80000)]
  0x000000002944f000 JavaThread "CONNECTOR michal:scheduler (4)" [_thread_blocked, id=1800, stack(0x000000002ee80000,0x000000002ef80000)]
  0x000000002944e800 JavaThread "CONNECTOR michal:scheduler (3)" [_thread_blocked, id=16392, stack(0x000000002ec20000,0x000000002ed20000)]
  0x000000002944d800 JavaThread "CONNECTOR michal:scheduler (2)" [_thread_blocked, id=10456, stack(0x000000002eb10000,0x000000002ec10000)]
  0x000000002944d000 JavaThread "CONNECTOR michal:public (4)-lowPrioritySelector" [_thread_blocked, id=13712, stack(0x0000000027a00000,0x0000000027b00000)]
  0x000000002944c000 JavaThread "CONNECTOR michal:public (3)" [_thread_in_native, id=22736, stack(0x000000002e7a0000,0x000000002e8a0000)]
  0x000000002944b800 JavaThread "CONNECTOR michal:public (2)-lowPrioritySelector" [_thread_blocked, id=3080, stack(0x000000002e600000,0x000000002e700000)]
  0x000000002944a800 JavaThread "CONNECTOR michal:public (1)" [_thread_in_native, id=12496, stack(0x000000002e3f0000,0x000000002e4f0000)]
  0x000000002944a000 JavaThread "CONNECTOR michal:scheduler (1)" [_thread_blocked, id=10040, stack(0x000000002c6d0000,0x000000002c7d0000)]
  0x0000000029fe7000 JavaThread "Abandoned connection cleanup thread" daemon [_thread_blocked, id=11332, stack(0x000000002b1c0000,0x000000002b2c0000)]
  0x000000002be5b000 JavaThread "CONNECTOR michal:network" [_thread_in_native, id=20788, stack(0x0000000029af0000,0x0000000029bf0000)]
  0x000000002bdb1800 JavaThread "Web (3)-acceptor-0@7648b318-default@13e39c73{HTTP/1.1,[http/1.1]}{0.0.0.0:7703}" [_thread_in_native, id=5360, stack(0x000000002cb70000,0x000000002cc70000)]
  0x000000002bdb0800 JavaThread "Web (2)-lowPrioritySelector" [_thread_blocked, id=14496, stack(0x000000002abd0000,0x000000002acd0000)]
  0x000000002a516800 JavaThread "Web (1)" [_thread_in_native, id=7576, stack(0x000000002c930000,0x000000002ca30000)]
  0x0000000028a7f000 JavaThread "Analyze" [_thread_blocked, id=12296, stack(0x0000000029990000,0x0000000029a90000)]
  0x0000000028d69800 JavaThread "Send" [_thread_blocked, id=3256, stack(0x0000000027c30000,0x0000000027d30000)]
  0x0000000026e75800 JavaThread "Service Thread" daemon [_thread_blocked, id=14664, stack(0x0000000027ea0000,0x0000000027fa0000)]
  0x0000000026ef0800 JavaThread "C1 CompilerThread2" daemon [_thread_blocked, id=14828, stack(0x0000000027d50000,0x0000000027e50000)]
  0x0000000026eef800 JavaThread "C2 CompilerThread1" daemon [_thread_blocked, id=5072, stack(0x0000000027700000,0x0000000027800000)]
  0x0000000026ee5000 JavaThread "C2 CompilerThread0" daemon [_thread_blocked, id=11588, stack(0x0000000027b00000,0x0000000027c00000)]
  0x0000000026d02800 JavaThread "JDWP Command Reader" daemon [_thread_in_native, id=17648, stack(0x0000000027900000,0x0000000027a00000)]
  0x0000000026cff000 JavaThread "JDWP Event Helper Thread" daemon [_thread_blocked, id=7872, stack(0x0000000027230000,0x0000000027330000)]
=>0x0000000026cf9000 JavaThread "JDWP Transport Listener: dt_socket" daemon [_thread_in_vm, id=23256, stack(0x00000000274c0000,0x00000000275c0000)]
  0x00000000256a9800 JavaThread "Attach Listener" daemon [_thread_blocked, id=11464, stack(0x0000000026bf0000,0x0000000026cf0000)]
  0x00000000256a6000 JavaThread "Signal Dispatcher" daemon [_thread_blocked, id=11512, stack(0x0000000026780000,0x0000000026880000)]
  0x000000002568c000 JavaThread "Finalizer" daemon [_thread_blocked, id=21252, stack(0x0000000026a10000,0x0000000026b10000)]
  0x0000000025644800 JavaThread "Reference Handler" daemon [_thread_blocked, id=12860, stack(0x0000000026890000,0x0000000026990000)]
  0x0000000002109800 JavaThread "main" [_thread_blocked, id=20488, stack(0x0000000002470000,0x0000000002570000)]

Other Threads:
  0x000000002563d000 VMThread [stack: 0x0000000026660000,0x0000000026760000] [id=21396]
  0x0000000026f79800 WatcherThread [stack: 0x0000000028040000,0x0000000028140000] [id=8244]

VM state:not at safepoint (normal execution)

VM Mutex/Monitor currently owned by a thread:  ([mutex/lock_event])
[0x00000000004ae330] Threads_lock - owner thread: 0x0000000026cf9000

Heap:
 PSYoungGen      total 284672K, used 175043K [0x0000000715d80000, 0x0000000732e80000, 0x00000007c0000000)
  eden space 263168K, 63% used [0x0000000715d80000,0x00000007201f2c90,0x0000000725e80000)
  from space 21504K, 30% used [0x0000000725e80000,0x00000007264fe108,0x0000000727380000)
  to   space 21504K, 0% used [0x0000000731980000,0x0000000731980000,0x0000000732e80000)
 ParOldGen       total 185856K, used 7147K [0x00000005c1800000, 0x00000005ccd80000, 0x0000000715d80000)
  object space 185856K, 3% used [0x00000005c1800000,0x00000005c1efadb0,0x00000005ccd80000)
 Metaspace       used 30395K, capacity 30648K, committed 30976K, reserved 1077248K
  class space    used 3441K, capacity 3524K, committed 3584K, reserved 1048576K

Card table byte_map: [0x0000000011db0000,0x0000000012db0000] byte_map_base: 0x000000000efa4000

Marking Bits: (ParMarkBitMap*) 0x000000005588b6d0
 Begin Bits: [0x0000000013d70000, 0x000000001bd10000)
 End Bits:   [0x000000001bd10000, 0x0000000023cb0000)

Polling page: 0x00000000001b0000

CodeCache: size=245760Kb used=11697Kb max_used=12283Kb free=234062Kb
 bounds [0x00000000029f0000, 0x0000000003610000, 0x00000000119f0000]
 total_blobs=3551 nmethods=3090 adapters=380
 compilation: enabled

Compilation events (10 events):
Event: 374.797 Thread 0x0000000026ef0800 3620   !   3       com.mysql.jdbc.ConnectionImpl::execSQL (427 bytes)
Event: 374.799 Thread 0x0000000026ef0800 nmethod 3620 0x000000000312ab50 code [0x000000000312afc0, 0x000000000312e8b8]
Event: 374.799 Thread 0x0000000026ef0800 3623       3       java.util.ArrayList::toArray (12 bytes)
Event: 374.799 Thread 0x0000000026ef0800 nmethod 3623 0x0000000003404290 code [0x0000000003404400, 0x00000000034046a8]
Event: 374.799 Thread 0x0000000026ef0800 3621   !   3       com.mysql.jdbc.MysqlIO::sqlQueryDirect (1778 bytes)
Event: 374.811 Thread 0x0000000026ef0800 nmethod 3621 0x0000000002b86c90 code [0x0000000002b87e80, 0x0000000002b99078]
Event: 374.811 Thread 0x0000000026ef0800 3622       3       com.mysql.jdbc.MysqlIO::readAllResults (206 bytes)
Event: 374.812 Thread 0x0000000026ef0800 nmethod 3622 0x0000000002cbf6d0 code [0x0000000002cbf940, 0x0000000002cc0b08]
Event: 374.858 Thread 0x0000000026eef800 3624       4       com.mwl.nynp.NynpConnectionManager::networkTick (9 bytes)
Event: 374.859 Thread 0x0000000026eef800 nmethod 3624 0x0000000003322e90 code [0x0000000003322fe0, 0x0000000003323078]

GC Heap History (8 events):
Event: 1.683 GC heap before
{Heap before GC invocations=1 (full 0):
 PSYoungGen      total 153088K, used 131584K [0x0000000715d80000, 0x0000000720800000, 0x00000007c0000000)
  eden space 131584K, 100% used [0x0000000715d80000,0x000000071de00000,0x000000071de00000)
  from space 21504K, 0% used [0x000000071f300000,0x000000071f300000,0x0000000720800000)
  to   space 21504K, 0% used [0x000000071de00000,0x000000071de00000,0x000000071f300000)
 ParOldGen       total 349696K, used 0K [0x00000005c1800000, 0x00000005d6d80000, 0x0000000715d80000)
  object space 349696K, 0% used [0x00000005c1800000,0x00000005c1800000,0x00000005d6d80000)
 Metaspace       used 17359K, capacity 17608K, committed 17792K, reserved 1064960K
  class space    used 1987K, capacity 2106K, committed 2176K, reserved 1048576K
Event: 1.714 GC heap after
Heap after GC invocations=1 (full 0):
 PSYoungGen      total 153088K, used 5343K [0x0000000715d80000, 0x0000000728880000, 0x00000007c0000000)
  eden space 131584K, 0% used [0x0000000715d80000,0x0000000715d80000,0x000000071de00000)
  from space 21504K, 24% used [0x000000071de00000,0x000000071e337ce0,0x000000071f300000)
  to   space 21504K, 0% used [0x0000000727380000,0x0000000727380000,0x0000000728880000)
 ParOldGen       total 349696K, used 24K [0x00000005c1800000, 0x00000005d6d80000, 0x0000000715d80000)
  object space 349696K, 0% used [0x00000005c1800000,0x00000005c1806000,0x00000005d6d80000)
 Metaspace       used 17359K, capacity 17608K, committed 17792K, reserved 1064960K
  class space    used 1987K, capacity 2106K, committed 2176K, reserved 1048576K
}
Event: 2.112 GC heap before
{Heap before GC invocations=2 (full 0):
 PSYoungGen      total 153088K, used 51348K [0x0000000715d80000, 0x0000000728880000, 0x00000007c0000000)
  eden space 131584K, 34% used [0x0000000715d80000,0x0000000718a6d540,0x000000071de00000)
  from space 21504K, 24% used [0x000000071de00000,0x000000071e337ce0,0x000000071f300000)
  to   space 21504K, 0% used [0x0000000727380000,0x0000000727380000,0x0000000728880000)
 ParOldGen       total 349696K, used 24K [0x00000005c1800000, 0x00000005d6d80000, 0x0000000715d80000)
  object space 349696K, 0% used [0x00000005c1800000,0x00000005c1806000,0x00000005d6d80000)
 Metaspace       used 20783K, capacity 21038K, committed 21248K, reserved 1069056K
  class space    used 2358K, capacity 2459K, committed 2560K, reserved 1048576K
Event: 2.127 GC heap after
Heap after GC invocations=2 (full 0):
 PSYoungGen      total 284672K, used 7328K [0x0000000715d80000, 0x0000000728880000, 0x00000007c0000000)
  eden space 263168K, 0% used [0x0000000715d80000,0x0000000715d80000,0x0000000725e80000)
  from space 21504K, 34% used [0x0000000727380000,0x0000000727aa81b0,0x0000000728880000)
  to   space 21504K, 0% used [0x0000000725e80000,0x0000000725e80000,0x0000000727380000)
 ParOldGen       total 349696K, used 32K [0x00000005c1800000, 0x00000005d6d80000, 0x0000000715d80000)
  object space 349696K, 0% used [0x00000005c1800000,0x00000005c1808000,0x00000005d6d80000)
 Metaspace       used 20783K, capacity 21038K, committed 21248K, reserved 1069056K
  class space    used 2358K, capacity 2459K, committed 2560K, reserved 1048576K
}
Event: 2.127 GC heap before
{Heap before GC invocations=3 (full 1):
 PSYoungGen      total 284672K, used 7328K [0x0000000715d80000, 0x0000000728880000, 0x00000007c0000000)
  eden space 263168K, 0% used [0x0000000715d80000,0x0000000715d80000,0x0000000725e80000)
  from space 21504K, 34% used [0x0000000727380000,0x0000000727aa81b0,0x0000000728880000)
  to   space 21504K, 0% used [0x0000000725e80000,0x0000000725e80000,0x0000000727380000)
 ParOldGen       total 349696K, used 32K [0x00000005c1800000, 0x00000005d6d80000, 0x0000000715d80000)
  object space 349696K, 0% used [0x00000005c1800000,0x00000005c1808000,0x00000005d6d80000)
 Metaspace       used 20783K, capacity 21038K, committed 21248K, reserved 1069056K
  class space    used 2358K, capacity 2459K, committed 2560K, reserved 1048576K
Event: 2.157 GC heap after
Heap after GC invocations=3 (full 1):
 PSYoungGen      total 284672K, used 0K [0x0000000715d80000, 0x0000000728880000, 0x00000007c0000000)
  eden space 263168K, 0% used [0x0000000715d80000,0x0000000715d80000,0x0000000725e80000)
  from space 21504K, 0% used [0x0000000727380000,0x0000000727380000,0x0000000728880000)
  to   space 21504K, 0% used [0x0000000725e80000,0x0000000725e80000,0x0000000727380000)
 ParOldGen       total 185856K, used 7139K [0x00000005c1800000, 0x00000005ccd80000, 0x0000000715d80000)
  object space 185856K, 3% used [0x00000005c1800000,0x00000005c1ef8db0,0x00000005ccd80000)
 Metaspace       used 20783K, capacity 21038K, committed 21248K, reserved 1069056K
  class space    used 2358K, capacity 2459K, committed 2560K, reserved 1048576K
}
Event: 3.240 GC heap before
{Heap before GC invocations=4 (full 1):
 PSYoungGen      total 284672K, used 263168K [0x0000000715d80000, 0x0000000728880000, 0x00000007c0000000)
  eden space 263168K, 100% used [0x0000000715d80000,0x0000000725e80000,0x0000000725e80000)
  from space 21504K, 0% used [0x0000000727380000,0x0000000727380000,0x0000000728880000)
  to   space 21504K, 0% used [0x0000000725e80000,0x0000000725e80000,0x0000000727380000)
 ParOldGen       total 185856K, used 7139K [0x00000005c1800000, 0x00000005ccd80000, 0x0000000715d80000)
  object space 185856K, 3% used [0x00000005c1800000,0x00000005c1ef8db0,0x00000005ccd80000)
 Metaspace       used 28367K, capacity 28652K, committed 29056K, reserved 1075200K
  class space    used 3248K, capacity 3330K, committed 3456K, reserved 1048576K
Event: 3.270 GC heap after
Heap after GC invocations=4 (full 1):
 PSYoungGen      total 284672K, used 6648K [0x0000000715d80000, 0x0000000732e80000, 0x00000007c0000000)
  eden space 263168K, 0% used [0x0000000715d80000,0x0000000715d80000,0x0000000725e80000)
  from space 21504K, 30% used [0x0000000725e80000,0x00000007264fe108,0x0000000727380000)
  to   space 21504K, 0% used [0x0000000731980000,0x0000000731980000,0x0000000732e80000)
 ParOldGen       total 185856K, used 7147K [0x00000005c1800000, 0x00000005ccd80000, 0x0000000715d80000)
  object space 185856K, 3% used [0x00000005c1800000,0x00000005c1efadb0,0x00000005ccd80000)
 Metaspace       used 28367K, capacity 28652K, committed 29056K, reserved 1075200K
  class space    used 3248K, capacity 3330K, committed 3456K, reserved 1048576K
}

Deoptimization events (10 events):
Event: 24.956 Thread 0x000000002b874000 Uncommon trap: reason=unstable_if action=reinterpret pc=0x00000000035412cc method=com.mwl.nynp.NynpMessageFormat.getMessagePrefixSize(Lcom/mwl/nynp/NynpMessageFormat$Prefix;Lcom/mwl/nynp/NynpMessageHeader;)I @ 136
Event: 24.976 Thread 0x0000000029155800 Uncommon trap: reason=unhandled action=none pc=0x00000000032fb778 method=java.lang.Integer.parseInt(Ljava/lang/String;I)I @ 204
Event: 36.126 Thread 0x000000002dcce000 Uncommon trap: reason=unhandled action=none pc=0x00000000032fb778 method=java.lang.Integer.parseInt(Ljava/lang/String;I)I @ 204
Event: 36.132 Thread 0x000000002dcce000 Uncommon trap: reason=unstable_if action=reinterpret pc=0x0000000003414eac method=java.lang.Math.abs(I)I @ 1
Event: 42.136 Thread 0x000000002dcd0000 Uncommon trap: reason=unstable_if action=reinterpret pc=0x00000000034ee86c method=java.util.concurrent.locks.ReentrantReadWriteLock$Sync.tryReleaseShared(I)Z @ 17
Event: 276.968 Thread 0x0000000029450000 Uncommon trap: reason=unstable_if action=reinterpret pc=0x0000000002cf338c method=java.util.concurrent.locks.AbstractQueuedSynchronizer.findNodeFromTail(Ljava/util/concurrent/locks/AbstractQueuedSynchronizer$Node;)Z @ 7
Event: 301.595 Thread 0x000000002944d800 Uncommon trap: reason=unstable_if action=reinterpret pc=0x0000000002f6a048 method=java.util.HashMap$HashIterator.<init>(Ljava/util/HashMap;)V @ 45
Event: 302.402 Thread 0x000000002aa80800 Uncommon trap: reason=unstable_if action=reinterpret pc=0x0000000003581ce0 method=java.util.HashMap$HashIterator.<init>(Ljava/util/HashMap;)V @ 45
Event: 302.510 Thread 0x000000002dde4000 Uncommon trap: reason=unstable_if action=reinterpret pc=0x00000000032beaa8 method=java.util.HashMap$HashIterator.<init>(Ljava/util/HashMap;)V @ 45
Event: 302.966 Thread 0x000000002dde7800 Uncommon trap: reason=unstable_if action=reinterpret pc=0x0000000002c5a3ac method=com.mwl.utils.SlidingAverage.addRecentValue(I)V @ 64

Internal exceptions (10 events):
Event: 0.922 Thread 0x0000000002109800 Exception <a 'java/lang/ExceptionInInitializerError'> (0x000000071929d558) thrown at [C:\re\workspace\8-2-build-windows-amd64-cygwin\jdk8u121\8372\hotspot\src\share\vm\prims\jni.cpp, line 709]
Event: 0.995 Thread 0x0000000002109800 Exception <a 'java/lang/NoSuchFieldError': method resolution failed> (0x00000007198cc6c8) thrown at [C:\re\workspace\8-2-build-windows-amd64-cygwin\jdk8u121\8372\hotspot\src\share\vm\prims\methodHandles.cpp, line 1146]
Event: 0.998 Thread 0x0000000002109800 Exception <a 'java/lang/NoSuchFieldError': method resolution failed> (0x00000007198da288) thrown at [C:\re\workspace\8-2-build-windows-amd64-cygwin\jdk8u121\8372\hotspot\src\share\vm\prims\methodHandles.cpp, line 1146]
Event: 1.593 Thread 0x0000000002109800 Implicit null exception at 0x0000000002c87f19 to 0x0000000002c88461
Event: 2.086 Thread 0x0000000002109800 Exception <a 'java/lang/ClassNotFoundException': sun/net/www/protocol/d/Handler> (0x00000007186b6590) thrown at [C:\re\workspace\8-2-build-windows-amd64-cygwin\jdk8u121\8372\hotspot\src\share\vm\classfile\systemDictionary.cpp, line 210]
Event: 2.088 Thread 0x0000000002109800 Exception <a 'java/lang/ClassNotFoundException': sun/net/www/protocol/d/Handler> (0x00000007186c0670) thrown at [C:\re\workspace\8-2-build-windows-amd64-cygwin\jdk8u121\8372\hotspot\src\share\vm\classfile\systemDictionary.cpp, line 210]
Event: 2.730 Thread 0x0000000002109800 Exception <a 'java/lang/ClassNotFoundException': sun/net/www/protocol/d/Handler> (0x000000071c5f11a0) thrown at [C:\re\workspace\8-2-build-windows-amd64-cygwin\jdk8u121\8372\hotspot\src\share\vm\classfile\systemDictionary.cpp, line 210]
Event: 2.731 Thread 0x0000000002109800 Exception <a 'java/lang/ClassNotFoundException': sun/net/www/protocol/d/Handler> (0x000000071c5f6448) thrown at [C:\re\workspace\8-2-build-windows-amd64-cygwin\jdk8u121\8372\hotspot\src\share\vm\classfile\systemDictionary.cpp, line 210]
Event: 2.733 Thread 0x0000000002109800 Exception <a 'sun/nio/fs/WindowsException'> (0x000000071c606870) thrown at [C:\re\workspace\8-2-build-windows-amd64-cygwin\jdk8u121\8372\hotspot\src\share\vm\prims\jni.cpp, line 709]
Event: 4.584 Thread 0x0000000028d69800 Implicit null exception at 0x000000000332f7f5 to 0x000000000332f9f6

Events (10 events):
Event: 375.028 Thread 0x000000002dccf800 Thread exited: 0x000000002dccf800
Event: 375.029 Thread 0x000000002dce1000 Thread exited: 0x000000002dce1000
Event: 375.029 Thread 0x000000002dcd1800 Thread exited: 0x000000002dcd1800
Event: 375.030 Thread 0x000000002dcde000 Thread exited: 0x000000002dcde000
Event: 375.031 Thread 0x000000002dcc2800 Thread exited: 0x000000002dcc2800
Event: 375.031 Thread 0x000000002dcd1000 Thread exited: 0x000000002dcd1000
Event: 375.032 Thread 0x000000002dccb800 Thread exited: 0x000000002dccb800
Event: 375.032 Thread 0x000000002dcd3000 Thread exited: 0x000000002dcd3000
Event: 375.033 Thread 0x000000002dcdd000 Thread exited: 0x000000002dcdd000
Event: 375.034 Thread 0x000000002dcc5000 Thread exited: 0x000000002dcc5000


Dynamic libraries:
0x000000013f100000 - 0x000000013f137000 	C:\Program Files\Java\jre1.8.0_121\bin\javaw.exe
0x0000000077150000 - 0x00000000772fa000 	C:\Windows\SYSTEM32\ntdll.dll
0x0000000076f30000 - 0x000000007704f000 	C:\Windows\system32\kernel32.dll
0x000007fefd280000 - 0x000007fefd2ea000 	C:\Windows\system32\KERNELBASE.dll
0x000007feff380000 - 0x000007feff45b000 	C:\Windows\system32\ADVAPI32.dll
0x000007fefeb90000 - 0x000007fefec2f000 	C:\Windows\system32\msvcrt.dll
0x000007feff1c0000 - 0x000007feff1df000 	C:\Windows\SYSTEM32\sechost.dll
0x000007fefefc0000 - 0x000007feff0ed000 	C:\Windows\system32\RPCRT4.dll
0x0000000077050000 - 0x000000007714a000 	C:\Windows\system32\USER32.dll
0x000007feff310000 - 0x000007feff377000 	C:\Windows\system32\GDI32.dll
0x000007fefe8e0000 - 0x000007fefe8ee000 	C:\Windows\system32\LPK.dll
0x000007feff0f0000 - 0x000007feff1bb000 	C:\Windows\system32\USP10.dll
0x000007fefb9a0000 - 0x000007fefbb94000 	C:\Windows\WinSxS\amd64_microsoft.windows.common-controls_6595b64144ccf1df_6.0.7601.18837_none_fa3b1e3d17594757\COMCTL32.dll
0x000007feff290000 - 0x000007feff301000 	C:\Windows\system32\SHLWAPI.dll
0x000007fefd990000 - 0x000007fefd9be000 	C:\Windows\system32\IMM32.DLL
0x000007fefee10000 - 0x000007fefef19000 	C:\Windows\system32\MSCTF.dll
0x000000006fbe0000 - 0x000000006fcb2000 	C:\Program Files\Java\jre1.8.0_121\bin\msvcr100.dll
0x0000000055070000 - 0x000000005590b000 	C:\Program Files\Java\jre1.8.0_121\bin\server\jvm.dll
0x000007fef7440000 - 0x000007fef7449000 	C:\Windows\system32\WSOCK32.dll
0x000007feff240000 - 0x000007feff28d000 	C:\Windows\system32\WS2_32.dll
0x000007fefe8d0000 - 0x000007fefe8d8000 	C:\Windows\system32\NSI.dll
0x000007fefb2b0000 - 0x000007fefb2eb000 	C:\Windows\system32\WINMM.dll
0x000007fefc060000 - 0x000007fefc06c000 	C:\Windows\system32\VERSION.dll
0x0000000077320000 - 0x0000000077327000 	C:\Windows\system32\PSAPI.DLL
0x00000000694a0000 - 0x00000000694af000 	C:\Program Files\Java\jre1.8.0_121\bin\verify.dll
0x0000000069440000 - 0x0000000069469000 	C:\Program Files\Java\jre1.8.0_121\bin\java.dll
0x000000006fb60000 - 0x000000006fb95000 	C:\Program Files\Java\jre1.8.0_121\bin\jdwp.dll
0x000000006fbd0000 - 0x000000006fbd8000 	C:\Program Files\Java\jre1.8.0_121\bin\npt.dll
0x000000006fba0000 - 0x000000006fbc3000 	C:\Program Files\Java\jre1.8.0_121\bin\instrument.dll
0x00000000693f0000 - 0x0000000069406000 	C:\Program Files\Java\jre1.8.0_121\bin\zip.dll
0x000007fefdaa0000 - 0x000007fefe82b000 	C:\Windows\system32\SHELL32.dll
0x000007fefe990000 - 0x000007fefeb8c000 	C:\Windows\system32\ole32.dll
0x000007fefcfa0000 - 0x000007fefcfaf000 	C:\Windows\system32\profapi.dll
0x000000006fb40000 - 0x000000006fb49000 	C:\Program Files\Java\jre1.8.0_121\bin\dt_socket.dll
0x000007fefb5a0000 - 0x000007fefb5b5000 	C:\Windows\system32\NLAapi.dll
0x000007fef6b80000 - 0x000007fef6b95000 	C:\Windows\system32\napinsp.dll
0x000007fef6b60000 - 0x000007fef6b79000 	C:\Windows\system32\pnrpnsp.dll
0x000007fefc730000 - 0x000007fefc785000 	C:\Windows\System32\mswsock.dll
0x000007fefc5b0000 - 0x000007fefc60b000 	C:\Windows\system32\DNSAPI.dll
0x000007fef6df0000 - 0x000007fef6dfb000 	C:\Windows\System32\winrnr.dll
0x000007fefab00000 - 0x000007fefab27000 	C:\Windows\system32\IPHLPAPI.DLL
0x000007fefaaf0000 - 0x000007fefaafb000 	C:\Windows\system32\WINNSI.DLL
0x000007fef95f0000 - 0x000007fef9643000 	C:\Windows\System32\fwpuclnt.dll
0x000007fef78a0000 - 0x000007fef78a8000 	C:\Windows\system32\rasadhlp.dll
0x000007fefc130000 - 0x000007fefc137000 	C:\Windows\System32\wshtcpip.dll
0x00000000679a0000 - 0x00000000679ba000 	C:\Program Files\Java\jre1.8.0_121\bin\net.dll
0x000007fefc720000 - 0x000007fefc727000 	C:\Windows\System32\wship6.dll
0x000007fef9650000 - 0x000007fef9661000 	C:\Windows\system32\dhcpcsvc6.DLL
0x000007fef95d0000 - 0x000007fef95e8000 	C:\Windows\system32\dhcpcsvc.DLL
0x0000000067980000 - 0x0000000067991000 	C:\Program Files\Java\jre1.8.0_121\bin\nio.dll
0x0000000067970000 - 0x000000006797d000 	C:\Program Files\Java\jre1.8.0_121\bin\management.dll
0x000007fefc790000 - 0x000007fefc7a8000 	C:\Windows\system32\CRYPTSP.dll
0x000007fefc490000 - 0x000007fefc4d7000 	C:\Windows\system32\rsaenh.dll
0x000007fefd170000 - 0x000007fefd18e000 	C:\Windows\system32\USERENV.dll
0x000007fefce00000 - 0x000007fefce0f000 	C:\Windows\system32\CRYPTBASE.dll
0x00000000671a0000 - 0x00000000671c4000 	C:\Program Files\Java\jre1.8.0_121\bin\sunec.dll
0x00000000693e0000 - 0x00000000693eb000 	C:\Program Files\Java\jre1.8.0_121\bin\sunmscapi.dll
0x000007fefcfc0000 - 0x000007fefd12d000 	C:\Windows\system32\CRYPT32.dll
0x000007fefcfb0000 - 0x000007fefcfbf000 	C:\Windows\system32\MSASN1.dll
0x000007fef6ea0000 - 0x000007fef6eee000 	C:\Windows\system32\pdh.dll
0x000007fefc030000 - 0x000007fefc05d000 	C:\Windows\system32\ntmarta.dll
0x000007feff1e0000 - 0x000007feff232000 	C:\Windows\system32\WLDAP32.dll
0x000007fef4c90000 - 0x000007fef4c9e000 	C:\Windows\System32\perfproc.dll
0x000007fef4ca0000 - 0x000007fef4cab000 	C:\Windows\System32\perfos.dll
0x000007fef4dd0000 - 0x000007fef4ef5000 	C:\Windows\system32\DBGHELP.DLL

VM Arguments:
jvm_args: -agentlib:jdwp=transport=dt_socket,suspend=y,address=localhost:59501 -ea -XX:+CreateMinidumpOnCrash -javaagent:C:\Users\Michal\.eclipse\org.eclipse.platform_4.8.0_920333535_win32_win32_x86_64\configuration\org.eclipse.osgi\418\0\.cp\lib\javaagent-shaded.jar -Dfile.encoding=UTF-8 -Xbootclasspath:C:\Program Files\Java\jre1.8.0_121\lib\resources.jar;C:\Program Files\Java\jre1.8.0_121\lib\rt.jar;C:\Program Files\Java\jre1.8.0_121\lib\jsse.jar;C:\Program Files\Java\jre1.8.0_121\lib\jce.jar;C:\Program Files\Java\jre1.8.0_121\lib\charsets.jar;C:\Program Files\Java\jre1.8.0_121\lib\jfr.jar 
java_command: com.mwl.hydra.HydraServer -config D:/Blimey!/Server/Source/Hydra/config/local/cluster.conf -spawnAll -name michal -controlPort 7703 -mailRecipient michal, rehacek@gmail.com -clusterName michal -boxName dev-michal -hostName orbis -fqdn orbis.michal-rehacek.cz -home /home/hydra -branch trunk -buildVersion 91823
java_class_path (initial): C:\Program Files\Java\jre1.8.0_121\lib\resources.jar;C:\Program Files\Java\jre1.8.0_121\lib\rt.jar;C:\Program Files\Java\jre1.8.0_121\lib\jsse.jar;C:\Program Files\Java\jre1.8.0_121\lib\jce.jar;C:\Program Files\Java\jre1.8.0_121\lib\charsets.jar;C:\Program Files\Java\jre1.8.0_121\lib\jfr.jar;D:\Blimey!\Server\Source\Hydra\bin;D:\Blimey!\Server\Source\Common\bin;D:\Blimey!\Server\Source\3rdParty\bin;D:\Blimey!\Server\Source\HydraCommon\bin;C:\Users\Michal\.eclipse\org.eclipse.platform_4.8.0_920333535_win32_win32_x86_64\configuration\org.eclipse.osgi\418\0\.cp\lib\javaagent-shaded.jar
Launcher Type: SUN_STANDARD

Environment Variables:
JAVA_HOME=C:\Program Files\Java\jdk1.8.0_72
PATH=C:/Program Files/Java/jre1.8.0_121/bin/server;C:/Program Files/Java/jre1.8.0_121/bin;C:/Program Files/Java/jre1.8.0_121/lib/amd64;C:\Program Files (x86)\SCE\ORBIS\Tools\Target Manager Server\bin;C:\Program Files (x86)\SCE\ORBIS SDKs\4.500\host_tools\bin;C:\Program Files (x86)\SCE\Common\SceVSI-VS14\bin;C:\Program Files (x86)\SCE\Common\SceVSI-VS11\bin;C:\Program Files (x86)\SCE\ORBIS\Tools\Publishing Tools\bin;C:\ProgramData\Oracle\Java\javapath;C:\Program Files (x86)\NVIDIA Corporation\PhysX\Common;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Program Files (x86)\Utilities;C:\Program Files (x86)\Subversion\bin;C:\Windows\System32\WindowsPowerShell\v1.0\;C:\Program Files\Microsoft Network Monitor 3\;C:\Program Files (x86)\QuickTime\QTSystem\;C:\Program Files (x86)\XAMPP\mysql\bin;C:\Program Files (x86)\XAMPP\php;C:\Program Files\Perl64\site\bin;C:\Program Files\Perl64\bin;C:\Program Files\Microsoft Windows Performance Toolkit\;C:\Program Files\Mercurial;C:\Program Files\Maven\bin;C:\Program Files\Microsoft SQL Server\110\Tools\Binn\;C:\Program Files (x86)\NVIDIA Corporation\DDS Utilities;C:\Program Files\Git\cmd;C:\Program Files (x86)\Perforce\DVCS\;C:\Program Files\Perforce;C:\Program Files (x86)\Skype\Phone\;C:\Program Files\Perforce\;C:\Program Files\Eclipse;
USERNAME=Michal
OS=Windows_NT
PROCESSOR_IDENTIFIER=Intel64 Family 6 Model 60 Stepping 3, GenuineIntel



---------------  S Y S T E M  ---------------

OS: Windows 7 , 64 bit Build 7601 (6.1.7601.23807)

CPU:total 4 (4 cores per cpu, 1 threads per core) family 6 model 60 stepping 3, cmov, cx8, fxsr, mmx, sse, sse2, sse3, ssse3, sse4.1, sse4.2, popcnt, avx, avx2, aes, clmul, erms, rtm, lzcnt, tsc, tscinvbit, bmi1, bmi2

Memory: 4k page, physical 33455432k(15060212k free), swap 66920964k(49306188k free)

vm_info: Java HotSpot(TM) 64-Bit Server VM (25.121-b13) for windows-amd64 JRE (1.8.0_121-b13), built on Dec 12 2016 18:21:36 by "java_re" with MS VC++ 10.0 (VS2010)

time: Mon Oct 08 17:09:59 2018
elapsed time: 376 seconds (0d 0h 6m 16s)



STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
Just debugging often enough from Eclipse. It will crash the JVM runtime sooner or later for me.


---------- BEGIN SOURCE ----------
The actual project contains 3537 java files. I really hope the minidump will be enough to investigate the problem.
---------- END SOURCE ----------

FREQUENCY : often



Comments
Fix Request The code in question uses the wrong synchronization to ensure it sees a safe and consistent snapshot of the threads and groups within a threadgroup, such that we can try to access invalid thread oops and crash. This has been demonstrated in JDK 8u upwards. The fix applies the correct synchronization by locking the ThreadGroup object's monitor. The risk is very low as we simply apply the correct synchronization. Test coverage in this area is fairly minimal. There are tests of the basic functionality but not of a dynamic environment where threads are being added and removed concurrently such that the synchronization problem is exposed. The submitter of the bug report has confirmed that the patch in JDK 12 addressed their problem. The patch applies cleanly to JDK11u
17-10-2018

Looks like all three uses of Threads_lock that you removed date back to the original JVM/TI delta: $ sgv src/share/vm/prims/jvmtiEnv.cpp | grep '{ MutexLocker mu(Threads_lock);' 1.168 1.1 { MutexLocker mu(Threads_lock); 1.1 { MutexLocker mu(Threads_lock); 1.1 { MutexLocker mu(Threads_lock); 3358 lines No id keywords (cm7) $ sp -r1.1 src/share/vm/prims/jvmtiEnv.cpp src/share/vm/prims/SCCS/s.jvmtiEnv.cpp: D 1.1 03/01/28 20:52:06 rfield 1 0 03130/00000/00000 MRs: COMMENTS: date and time created 03/01/28 20:52:06 by rfield
15-10-2018

The 8u181 crash doesn't show a full stack but decodes to: Stack: [0x0000000027970000,0x0000000027a70000], sp=0x0000000027a6ef18, free space=1019k Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code) V [jvm.dll+00069197] = 00068190+ 7 ?thread@java_lang_Thread@@SAPEAVJavaThread@@PEAVoopDesc@@@Z 0000000008069190 f javaClasses.obj which maps to this call in GetThreadGroupChildren: JavaThread *javathread = java_lang_Thread::thread(thread_obj); so we do seem to have hit on the problem code.
11-10-2018

Moving to JVMTI. The GetThreadGroupChildren function is completely broken. It contains this: { // Cannot allow thread or group counts to change. MutexLocker mu(Threads_lock); nthreads = java_lang_ThreadGroup::nthreads(group_hdl()); ngroups = java_lang_ThreadGroup::ngroups(group_hdl()); But the Threads_lock has no affect on how the ThreadGroup's internal state is updated. The threads array in the ThreadGroup can grow or shrink while we hold the Threads_lock. The only way to ensure atomic access to the ThreadGroup internal state is to synchronize on the ThreadGroup object, as that is how atomicity is achieved at the Java level.
11-10-2018

Decoded stack trace: --------------- T H R E A D --------------- Current thread (0x00000000316b2000): JavaThread "JDWP Transport Listener: dt_socket" daemon [_thread_in_vm, id=18408, stack(0x00000000323d0000,0x00000000324d0000)] _threads_hazard_ptr=0x0000000033812c60 Stack: [0x00000000323d0000,0x00000000324d0000], sp=0x00000000324cf4a0, free space=1021k Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code) V [jvm.dll+004f27c1] = 004f17a0+ 33 ?cv_oop_to_JavaThread@JvmtiExport@@SA?AW4jvmtiError@@PEAVThreadsList@@PEAVoopDesc@@PEAPEAVJavaThread@@@Z 00000001804f27a0 f jvmtiExport.obj V [jvm.dll+004e75eb] = 004e63e0+ 523 ?GetThreadGroupChildren@JvmtiEnv@@QEAA?AW4jvmtiError@@PEAV_jobject@@PEAJPEAPEAPEAV3@12@Z 00000001804e73e0 f jvmtiEnv.obj V [jvm.dll+004a3131] = 004a2040+ 241 jvmti_GetThreadGroupChildren 00000001804a3040 f jvmtiEnter.obj C [jdwp.dll+0x8630]
11-10-2018

requested submitter to verify against 12 ea, below is the response == I have been able to reproduce the problem in both the early access build of OpenJDK12, and also in the latest release of JDK 10. The crash is in the same thread (JDWP Transport Listener: dt_socket), the disassembly looks slightly different than in the JDK 8, though it's still a NULL pointer dereference. The crash dumps are available for download in this location - one from JDK 12, one from JDK 10: http://server.gamagio.com/oracle-jdk/1/jvm_crash_12_ea_14.zip http://server.gamagio.com/oracle-jdk/1/jvm_crash_10_0_2_13.zip
09-10-2018

Additional information from submitter == Hello Fairoz, I can get it to crash even with 8u181. The crash dump is available at http://server.gamagio.com/oracle-jdk/1/jvm_crash_8_181.zip It is a NULL pointer derefrence - from the disassembly it looks like a function is called with NULL "this". I hope you'll be able to find the source of the crash from the dump. I do not have a small reproducible case. The Java Runtime crashes when debugging a large proprietary application with many threads. ==
09-10-2018

Issue reported against 8u121, kindly verify against the latest JDK version [1] If issue is reproducible on latest version, kindly share the stand alone test case to reproduce this issue. [1] https://www.oracle.com/technetwork/java/javase/downloads/jdk8-downloads-2133151.html
09-10-2018