United StatesChange Country, Oracle Worldwide Web Sites Communities I am a... I want to...
JDK-8023786 : (jdk) setjmp/longjmp changes the process signal mask on OS X

Details
Type:
Bug
Submit Date:
2013-08-27
Status:
Closed
Updated Date:
2014-01-15
Project Name:
JDK
Resolved Date:
2013-08-29
Component:
core-svc
OS:
os_x
Sub-Component:
tools
CPU:
Priority:
P2
Resolution:
Fixed
Affected Versions:
7u40,8
Fixed Versions:

Related Reports
Backport:
Cloners:
Relates:
Relates:

Sub Tasks

Description
This bug originated from the following report: http://mail.openjdk.java.net/pipermail/serviceability-dev/2013-August/011318.html

That report has a reproducer that causes attach to stop working on OS X. The reason is that the process ends up with a signal mask that blocks SIGQUIT:

- The VM sets the signal mask for all threads (except the internal VMThread) to mask out SIGQUIT using pthread_sigmask(). So the process will still respond to SIGQUIT, but only one thread.
- The verifier code calls setjmp() to save the calling context. On OS X this also saves the signal mask. 
- The example code causes a verification error somewhere which causes the verifier to call longjmp().
- longjmp will restore the signal mask using sigprocmask() which sets the signal mask for the _process_.
- Now the process has a signal mask that masks out SIGQUIT and attach stops working.

This only happens on OS X because setjmp/longjmp does not save and restore the signal mask on other platforms. There are functions _setjmp/_longjmp on OS X to skip the signal mask save/restore and I think this is what we need to use in the verifier (also need to check other uses of setjmp/longjmp in the JDK).

                                    

Comments
URL:   http://hg.openjdk.java.net/jdk8/tl/jdk/rev/779ff9f3b2e3
User:  sla
Date:  2013-08-29 09:31:10 +0000

                                     
2013-08-29
URL:   http://hg.openjdk.java.net/jdk8/jdk8/jdk/rev/779ff9f3b2e3
User:  lana
Date:  2013-08-31 01:29:26 +0000

                                     
2013-08-31



Hardware and Software, Engineered to Work Together