United StatesChange Country, Oracle Worldwide Web Sites Communities I am a... I want to...
JDK-8023720 : (hotspot) setjmp/longjmp changes the process signal mask on OS X

Details
Type:
Bug
Submit Date:
2013-08-26
Status:
Closed
Updated Date:
2014-01-14
Project Name:
JDK
Resolved Date:
2013-08-29
Component:
hotspot
OS:
os_x
Sub-Component:
svc
CPU:
Priority:
P2
Resolution:
Fixed
Affected Versions:
hs24,hs25
Fixed Versions:
hs25 (b49)

Related Reports
Backport:
Backport:
Backport:
Cloners:
Relates:
Relates:

Sub Tasks

Description
This bug originated from the following report: http://mail.openjdk.java.net/pipermail/serviceability-dev/2013-August/011318.html

That report has a reproducer that causes attach to stop working on OS X. The reason is that the process ends up with a signal mask that blocks SIGQUIT:

- The VM sets the signal mask for all threads (except the internal VMThread) to mask out SIGQUIT using pthread_sigmask(). So the process will still respond to SIGQUIT, but only one thread.
- The verifier code calls setjmp() to save the calling context. On OS X this also saves the signal mask. 
- The example code causes a verification error somewhere which causes the verifier to call longjmp().
- longjmp will restore the signal mask using sigprocmask() which sets the signal mask for the _process_.
- Now the process has a signal mask that masks out SIGQUIT and attach stops working.

This only happens on OS X because setjmp/longjmp does not save and restore the signal mask on other platforms. There are functions _setjmp/_longjmp on OS X to skip the signal mask save/restore and I think this is what we need to use in the verifier (also need to check other uses of setjmp/longjmp in the JDK).

                                    

Comments
URL:   http://hg.openjdk.java.net/hsx/hotspot-rt/hotspot/rev/cc56f122f3f7
User:  sla
Date:  2013-08-29 13:40:40 +0000

                                     
2013-08-29
URL:   http://hg.openjdk.java.net/hsx/hsx25/hotspot/rev/cc56f122f3f7
User:  jcoomes
Date:  2013-09-06 21:27:38 +0000

                                     
2013-09-06



Hardware and Software, Engineered to Work Together