For the past several days my solaris-i586 build machine has been falling into a tight loop while running pack200. /opt/sfw/bin/top shows:
load averages: 3.57, 2.88, 5.18 12:28:48
72 processes: 64 sleeping, 6 running, 1 zombie, 1 on cpu
CPU states: 0.0% idle, 49.8% user, 50.2% kernel, 0.0% iowait, 0.0% swap
Memory: 3327M real, 2384M free, 160M swap in use, 6134M swap free
PID USERNAME LWP PRI NICE SIZE RES STATE TIME CPU COMMAND
20993 otto 11 53 2 84M 84M run 457:15 49.71% pack200
This is using a server VM built from the hotspot master repository.
% pargs 20993
20993: /u/auto/jdk/7/build/130/bin/pack200 -J-esa -J-ea -J-Xmx512m --no-gzip --config-
argv[0]: /u/auto/jdk/7/build/130/bin/pack200
argv[1]: -J-esa
argv[2]: -J-ea
argv[3]: -J-Xmx512m
argv[4]: --no-gzip
argv[5]: --config-file=pack.all.properties
argv[6]: --code-attribute=StackMapTable=strip
argv[7]: /u/auto/jdk/7/build/130/pack/pack-jdk-jars/jre/lib/jsse.pack
argv[8]: /u/auto/jdk/7/build/130/j2sdk-image/jre/lib/jsse.jar
attaching to the process with dbx shows this:
(dbx) threads
> t@1 a l@1 ?() running in __lwp_wait()
t@2 a l@2 JavaMain() running in ___lwp_cond_wait()
t@3 b l@3 java_start() running in ___lwp_cond_wait()
t@4 b l@4 java_start() running in ___lwp_cond_wait()
t@5 b l@5 java_start() running in TryFast()
t@6 b l@6 java_start() running in ___lwp_cond_wait()
t@7 b l@7 java_start() running in ___lwp_cond_wait()
t@9 b l@9 java_start() running in ___lwp_cond_wait()
t@10 b l@10 java_start() running in dominates()
t@11 b l@11 java_start() running in ___lwp_cond_wait()
t@13 b l@13 umem_update_thread() sleep on (unknown) in __lwp_park()
(dbx) thread t@10
t@10 (l@10) stopped in Node::dominates at 0xfeaddc89
0xfeaddc89: dominates+0x00ad: je dominates+0x1d5 [ 0xfeadddb1, .+0x128 ]
(dbx) where
current thread: t@10
=>[1] Node::dominates(0x959b1bc, 0x8e26ba4, 0xd6b9bbc0), at 0xfeaddc89
[2] MemNode::all_controls_dominate(0x959b1fc, 0x8d9cd34), at 0xfeac595f
[3] InitializeNode::detect_init_independence(0x8d9cd34, 0x959b1fc, 0x1, 0xd6b9bddc), at
0xfeaca5c2
[4] InitializeNode::detect_init_independence(0x8d9cd34, 0x959d7e0, 0x1, 0xd6b9bddc), at
0xfeaca633
[5] InitializeNode::detect_init_independence(0x8d9cd34, 0x8e1ab40, 0x1, 0xd6b9bddc), at
0xfeaca633
[6] InitializeNode::detect_init_independence(0x8d9cd34, 0x8e1bbec, 0x1, 0xd6b9bddc), at
0xfeaca633
[7] InitializeNode::detect_init_independence(0x8d9cd34, 0x8e1cfbc, 0x1, 0xd6b9bddc), at
0xfeaca633
[8] InitializeNode::detect_init_independence(0x8d9cd34, 0x8d9c724, 0x1, 0xd6b9bddc), at
0xfeaca633
[9] InitializeNode::can_capture_store(0x8d9cd34, 0x8d9dd6c, 0xd6b9bee0), at 0xfeaca71f
[10] StoreNode::Ideal(0x8d9dd6c, 0xd6b9bee0, 0x1), at 0xfeac909f
[11] PhaseIterGVN::transform_old(0xd6b9bee0, 0x8d9dd6c), at 0xfe69d99b
[12] PhaseIterGVN::optimize(0xd6b9bee0), at 0xfe716bed
[13] Compile::Optimize(0xd6b9d558), at 0xfe7571f9
[14] Compile::Compile(0xd6b9d558, 0xd6b9da50, 0x808eae8, 0x9556010, 0xffffffff, 0x1, 0x0),
at 0xfe90e009
[15] C2Compiler::compile_method(0x808eae8, 0xd6b9da50, 0x9556010, 0xffffffff), at 0xfe754036
[16] CompileBroker::invoke_compiler_on_method(0x8364e88), at 0xfe7545bb
[17] CompileBroker::compiler_thread_loop(0xfecd8000, 0x98, 0xd6b9dc48, 0xfe77d3e8,
0x81b9800, 0x81b9800), at 0xfe7b3dfc
[18] compiler_thread_entry(0x81b9800, 0x81b9800), at 0xfe7b5cdc
[19] JavaThread::thread_main_inner(0x81b9800), at 0xfe77d3e8
[20] JavaThread::run(0x81b9800), at 0xfe77d392
[21] java_start(0x81b9800), at 0xfeae3921
[22] _thr_setup(0xfb081c00), at 0xfeedfc32
[23] _lwp_start(), at 0xfeedff20
If I stepi this thread it goes into find_exact_control, comes back out, and repeats.
Unfortunately I have not been able to reproduce this at will from the command line - it is only happening during my nightly build.
The lab system is baozi.sfbay. Interactive response is poor when the process is running, but you will eventually get a prompt.
There are several core files, pargs output, and notes under /var/tmp/tbell/<pid>