JDK-8234474 : [macos 10.15] Crash in file dialog in sandbox mode
  • Type: Bug
  • Component: javafx
  • Sub-Component: window-toolkit
  • Affected Version: 8u231,openjfx11,openjfx13
  • Priority: P3
  • Status: Resolved
  • Resolution: Fixed
  • OS: os_x
  • CPU: x86
  • Submitted: 2019-11-15
  • Updated: 2020-06-09
  • Resolved: 2020-01-06
The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.
JDK 8 Other
8u261Fixed openjfx11.0.8Fixed
Related Reports
Relates :  
Relates :  
Description
ADDITIONAL SYSTEM INFORMATION :
MacOS 10.15.1

Java 8 update 231

A DESCRIPTION OF THE PROBLEM :
We have had several customers report that on macOS Catalina, as soon as they open a save or open dialog, the app crashes.
I can't reproduce this locally.
The MacOS stack trace shows that it has crashed in libglass.dylib Java_com_sun_glass_ui_mac_MacCommonDialogs__1showFileSaveChooser

I have tried disabling System Integrity Protection on my Mac with macOS Catalina and can't reproduce this but we have had at least 4 customer reports of this.

We have fixed this locally by patching GlassApplication.m and ensure we never call GlassOpenPanel or GlassSavePanel, but use NSOpenPanel or NSSavePanel.

Here is the stack trace:

System Integrity Protection: disabled

Crashed Thread:        0  Dispatch queue: com.apple.main-thread

Exception Type:        EXC_BAD_ACCESS (SIGABRT)
Exception Codes:       KERN_PROTECTION_FAILURE at 0x00000001045dd000
Exception Note:        EXC_CORPSE_NOTIFY

VM Regions Near 0x1045dd000:
    __LINKEDIT             00000001045ce000-00000001045dd000 [   60K] r--/rwx SM=COW  /Applications/Screaming Frog SEO Spider.app/Contents/jre/Home/lib/libjava.dylib
--> VM_ALLOCATE            00000001045dd000-00000001045de000 [    4K] r--/rwx SM=PRV  
    shared memory          00000001045de000-00000001045df000 [    4K] r--/r-- SM=SHM  

Application Specific Information:
*** Terminating app due to uncaught exception 'NSObjectNotAvailableException', reason: 'GlassSavePanel is not a supported subclass for sandboxing'
abort() called
terminating with uncaught exception of type NSException

Application Specific Backtrace 1:
0   CoreFoundation                      0x00007fff3330ff53 __exceptionPreprocess + 250
1   libobjc.A.dylib                     0x00007fff693d5835 objc_exception_throw + 48
2   CoreFoundation                      0x00007fff3330fda9 +[NSException raise:format:] + 189
3   AppKit                              0x00007fff30b65b54 +[NSSavePanel(Instantiation) _crunchyRawUnbonedPanel] + 368
4   libglass.dylib                      0x0000000121111d2b Java_com_sun_glass_ui_mac_MacCommonDialogs__1showFileSaveChooser + 347
5   ???                                 0x000000010ee0d6e7 0x0 + 4544583399
6   ???                                 0x000000010edfd040 0x0 + 4544516160

Thread 0 Crashed:: Dispatch queue: com.apple.main-thread
0   libsystem_kernel.dylib        	0x00007fff6a88749a __pthread_kill + 10
1   libsystem_pthread.dylib       	0x00007fff6a9446cb pthread_kill + 384
2   libsystem_c.dylib             	0x00007fff6a80fa1c abort + 120
3   libc++abi.dylib               	0x00007fff678b8bc8 abort_message + 231
4   libc++abi.dylib               	0x00007fff678b8d7c demangling_terminate_handler() + 262
5   libobjc.A.dylib               	0x00007fff693d77ba _objc_terminate() + 96
6   libc++abi.dylib               	0x00007fff678c5da7 std::__terminate(void (*)()) + 8
7   libc++abi.dylib               	0x00007fff678c5b55 __cxxabiv1::failed_throw(__cxxabiv1::__cxa_exception*) + 27
8   libc++abi.dylib               	0x00007fff678b734f __cxa_throw + 113
9   libobjc.A.dylib               	0x00007fff693d5963 objc_exception_throw + 350
10  com.apple.CoreFoundation      	0x00007fff3330fda9 +[NSException raise:format:] + 189
11  com.apple.AppKit              	0x00007fff30b65b54 +[NSSavePanel(Instantiation) _crunchyRawUnbonedPanel] + 368
12  libglass.dylib                	0x0000000121111d2b Java_com_sun_glass_ui_mac_MacCommonDialogs__1showFileSaveChooser + 347
13  ???                           	0x000000010ee0d6e7 0 + 4544583399
14  ???                           	0x000000010edfd040 0 + 4544516160
15  ???                           	0x000000010edfd040 0 + 4544516160
16  ???                           	0x000000010edfd040 0 + 4544516160
17  ???                           	0x000000010edfd040 0 + 4544516160
18  ???                           	0x000000010edfd040 0 + 4544516160
19  ???                           	0x000000010edfd040 0 + 4544516160
20  ???                           	0x000000010edfd040 0 + 4544516160
21  ???                           	0x000000010edfd040 0 + 4544516160
22  ???                           	0x000000010edfd114 0 + 4544516372
23  ???                           	0x000000010edfd040 0 + 4544516160
24  ???                           	0x000000010edfd114 0 + 4544516372
25  ???                           	0x000000010edfca90 0 + 4544514704
26  ???                           	0x00000001102311cc 0 + 4565701068
27  libjvm.dylib                  	0x00000001058ef9aa JavaCalls::call_helper(JavaValue*, methodHandle*, JavaCallArguments*, Thread*) + 1710
28  libjvm.dylib                  	0x000000010595019c JVM_DoPrivileged + 1208
29  ???                           	0x000000010f172415 0 + 4548142101
30  ???                           	0x000000011022fd2c 0 + 4565695788
31  libjvm.dylib                  	0x00000001058ef9aa JavaCalls::call_helper(JavaValue*, methodHandle*, JavaCallArguments*, Thread*) + 1710
32  libjvm.dylib                  	0x0000000105926bd8 jni_invoke_nonstatic(JNIEnv_*, JavaValue*, _jobject*, JNICallType, _jmethodID*, JNI_ArgumentPusher*, Thread*) + 748
33  libjvm.dylib                  	0x00000001059193bb jni_CallVoidMethod + 363
34  libglass.dylib                	0x0000000121112f0c -[GlassRunnable run] + 108
35  com.apple.Foundation          	0x00007fff3599428a __NSThreadPerformPerform + 254
36  com.apple.CoreFoundation      	0x00007fff3327bb81 __CFRUNLOOP_IS_CALLING_OUT_TO_A_SOURCE0_PERFORM_FUNCTION__ + 17
37  com.apple.CoreFoundation      	0x00007fff3327bb20 __CFRunLoopDoSource0 + 103
38  com.apple.CoreFoundation      	0x00007fff3325f154 __CFRunLoopDoSources0 + 209
39  com.apple.CoreFoundation      	0x00007fff3325e760 __CFRunLoopRun + 1272
40  com.apple.CoreFoundation      	0x00007fff3325dfe3 CFRunLoopRunSpecific + 499
41  com.apple.AppKit              	0x00007fff304740e5 _NSHandleCarbonMenuEvent + 311
42  com.apple.AppKit              	0x00007fff30473ef2 _DPSEventHandledByCarbon + 54
43  com.apple.AppKit              	0x00007fff30469c1e -[NSApplication(NSEvent) _nextEventMatchingEventMask:untilDate:inMode:dequeue:] + 2962
44  com.apple.AppKit              	0x00007fff30463d76 -[NSApplication run] + 658
45  libglass.dylib                	0x0000000121114ed9 -[GlassApplication runLoop:] + 1897
46  com.apple.Foundation          	0x00007fff3599428a __NSThreadPerformPerform + 254
47  com.apple.CoreFoundation      	0x00007fff3327bb81 __CFRUNLOOP_IS_CALLING_OUT_TO_A_SOURCE0_PERFORM_FUNCTION__ + 17
48  com.apple.CoreFoundation      	0x00007fff3327bb20 __CFRunLoopDoSource0 + 103
49  com.apple.CoreFoundation      	0x00007fff3325f154 __CFRunLoopDoSources0 + 209
50  com.apple.CoreFoundation      	0x00007fff3325e760 __CFRunLoopRun + 1272
51  com.apple.CoreFoundation      	0x00007fff3325dfe3 CFRunLoopRunSpecific + 499
52  java                          	0x000000010457047a CreateExecutionEnvironment + 871
53  java                          	0x000000010456c12c JLI_Launch + 1952
54  java                          	0x00000001045724d7 main + 101
55  java                          	0x000000010456b984 start + 52

REGRESSION : Last worked in version 8

STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
Can't reproduce it myself but 4 customers have reported crashes when displaying a open or save dialog

EXPECTED VERSUS ACTUAL BEHAVIOR :
EXPECTED -
The app shouldn't crash when displaying a save or open dialog
ACTUAL -
App crashes when displaying a save or open dialog
Note. All users experiencing crash had SIP disabled on their Mac.

Stack Trace:

System Integrity Protection: disabled

Crashed Thread:        0  Dispatch queue: com.apple.main-thread

Exception Type:        EXC_BAD_ACCESS (SIGABRT)
Exception Codes:       KERN_PROTECTION_FAILURE at 0x00000001045dd000
Exception Note:        EXC_CORPSE_NOTIFY

VM Regions Near 0x1045dd000:
    __LINKEDIT             00000001045ce000-00000001045dd000 [   60K] r--/rwx SM=COW  /Applications/Screaming Frog SEO Spider.app/Contents/jre/Home/lib/libjava.dylib
--> VM_ALLOCATE            00000001045dd000-00000001045de000 [    4K] r--/rwx SM=PRV  
    shared memory          00000001045de000-00000001045df000 [    4K] r--/r-- SM=SHM  

Application Specific Information:
*** Terminating app due to uncaught exception 'NSObjectNotAvailableException', reason: 'GlassSavePanel is not a supported subclass for sandboxing'
abort() called
terminating with uncaught exception of type NSException

Application Specific Backtrace 1:
0   CoreFoundation                      0x00007fff3330ff53 __exceptionPreprocess + 250
1   libobjc.A.dylib                     0x00007fff693d5835 objc_exception_throw + 48
2   CoreFoundation                      0x00007fff3330fda9 +[NSException raise:format:] + 189
3   AppKit                              0x00007fff30b65b54 +[NSSavePanel(Instantiation) _crunchyRawUnbonedPanel] + 368
4   libglass.dylib                      0x0000000121111d2b Java_com_sun_glass_ui_mac_MacCommonDialogs__1showFileSaveChooser + 347
5   ???                                 0x000000010ee0d6e7 0x0 + 4544583399
6   ???                                 0x000000010edfd040 0x0 + 4544516160

Thread 0 Crashed:: Dispatch queue: com.apple.main-thread
0   libsystem_kernel.dylib        	0x00007fff6a88749a __pthread_kill + 10
1   libsystem_pthread.dylib       	0x00007fff6a9446cb pthread_kill + 384
2   libsystem_c.dylib             	0x00007fff6a80fa1c abort + 120
3   libc++abi.dylib               	0x00007fff678b8bc8 abort_message + 231
4   libc++abi.dylib               	0x00007fff678b8d7c demangling_terminate_handler() + 262
5   libobjc.A.dylib               	0x00007fff693d77ba _objc_terminate() + 96
6   libc++abi.dylib               	0x00007fff678c5da7 std::__terminate(void (*)()) + 8
7   libc++abi.dylib               	0x00007fff678c5b55 __cxxabiv1::failed_throw(__cxxabiv1::__cxa_exception*) + 27
8   libc++abi.dylib               	0x00007fff678b734f __cxa_throw + 113
9   libobjc.A.dylib               	0x00007fff693d5963 objc_exception_throw + 350
10  com.apple.CoreFoundation      	0x00007fff3330fda9 +[NSException raise:format:] + 189
11  com.apple.AppKit              	0x00007fff30b65b54 +[NSSavePanel(Instantiation) _crunchyRawUnbonedPanel] + 368
12  libglass.dylib                	0x0000000121111d2b Java_com_sun_glass_ui_mac_MacCommonDialogs__1showFileSaveChooser + 347
13  ???                           	0x000000010ee0d6e7 0 + 4544583399
14  ???                           	0x000000010edfd040 0 + 4544516160
15  ???                           	0x000000010edfd040 0 + 4544516160
16  ???                           	0x000000010edfd040 0 + 4544516160
17  ???                           	0x000000010edfd040 0 + 4544516160
18  ???                           	0x000000010edfd040 0 + 4544516160
19  ???                           	0x000000010edfd040 0 + 4544516160
20  ???                           	0x000000010edfd040 0 + 4544516160
21  ???                           	0x000000010edfd040 0 + 4544516160
22  ???                           	0x000000010edfd114 0 + 4544516372
23  ???                           	0x000000010edfd040 0 + 4544516160
24  ???                           	0x000000010edfd114 0 + 4544516372
25  ???                           	0x000000010edfca90 0 + 4544514704
26  ???                           	0x00000001102311cc 0 + 4565701068
27  libjvm.dylib                  	0x00000001058ef9aa JavaCalls::call_helper(JavaValue*, methodHandle*, JavaCallArguments*, Thread*) + 1710
28  libjvm.dylib                  	0x000000010595019c JVM_DoPrivileged + 1208
29  ???                           	0x000000010f172415 0 + 4548142101
30  ???                           	0x000000011022fd2c 0 + 4565695788
31  libjvm.dylib                  	0x00000001058ef9aa JavaCalls::call_helper(JavaValue*, methodHandle*, JavaCallArguments*, Thread*) + 1710
32  libjvm.dylib                  	0x0000000105926bd8 jni_invoke_nonstatic(JNIEnv_*, JavaValue*, _jobject*, JNICallType, _jmethodID*, JNI_ArgumentPusher*, Thread*) + 748
33  libjvm.dylib                  	0x00000001059193bb jni_CallVoidMethod + 363
34  libglass.dylib                	0x0000000121112f0c -[GlassRunnable run] + 108
35  com.apple.Foundation          	0x00007fff3599428a __NSThreadPerformPerform + 254
36  com.apple.CoreFoundation      	0x00007fff3327bb81 __CFRUNLOOP_IS_CALLING_OUT_TO_A_SOURCE0_PERFORM_FUNCTION__ + 17
37  com.apple.CoreFoundation      	0x00007fff3327bb20 __CFRunLoopDoSource0 + 103
38  com.apple.CoreFoundation      	0x00007fff3325f154 __CFRunLoopDoSources0 + 209
39  com.apple.CoreFoundation      	0x00007fff3325e760 __CFRunLoopRun + 1272
40  com.apple.CoreFoundation      	0x00007fff3325dfe3 CFRunLoopRunSpecific + 499
41  com.apple.AppKit              	0x00007fff304740e5 _NSHandleCarbonMenuEvent + 311
42  com.apple.AppKit              	0x00007fff30473ef2 _DPSEventHandledByCarbon + 54
43  com.apple.AppKit              	0x00007fff30469c1e -[NSApplication(NSEvent) _nextEventMatchingEventMask:untilDate:inMode:dequeue:] + 2962
44  com.apple.AppKit              	0x00007fff30463d76 -[NSApplication run] + 658
45  libglass.dylib                	0x0000000121114ed9 -[GlassApplication runLoop:] + 1897
46  com.apple.Foundation          	0x00007fff3599428a __NSThreadPerformPerform + 254
47  com.apple.CoreFoundation      	0x00007fff3327bb81 __CFRUNLOOP_IS_CALLING_OUT_TO_A_SOURCE0_PERFORM_FUNCTION__ + 17
48  com.apple.CoreFoundation      	0x00007fff3327bb20 __CFRunLoopDoSource0 + 103
49  com.apple.CoreFoundation      	0x00007fff3325f154 __CFRunLoopDoSources0 + 209
50  com.apple.CoreFoundation      	0x00007fff3325e760 __CFRunLoopRun + 1272
51  com.apple.CoreFoundation      	0x00007fff3325dfe3 CFRunLoopRunSpecific + 499
52  java                          	0x000000010457047a CreateExecutionEnvironment + 871
53  java                          	0x000000010456c12c JLI_Launch + 1952
54  java                          	0x00000001045724d7 main + 101
55  java                          	0x000000010456b984 start + 52

FREQUENCY : occasionally



Comments
Changeset: 587f195c Author: Kevin Rushforth <kcr@openjdk.org> Date: 2020-01-06 21:11:20 +0000 URL: https://git.openjdk.java.net/jfx/commit/587f195c
06-01-2020

Currently we use a Glass-specific subclass of the NSSavePanel or NSOpenPanel unless the app is running in sandboxed mode. File dialogs are always out-of-process on Catalina, so are effectively always sandboxed. This means that there is no benefit at all to trying to use the Glass sub-classes on macOS 10.15 or later. They are now ineffective, so the fix for this bug is to change the logic so that we use the NSSavePanel or NSOpenPanel directly in either of the following conditions: 1) the app is running in sandbox mode OR 2) The platform is macOS 10.15 or later
19-12-2019

The code in question hasn't changed since JDK 8u40, so it would also affect openjfx11 and later. I note that this isn't a regression.
21-11-2019

I am also not able to reproduce this. The stack trace suggests that the application is running in sandboxed mode, such that the GlassOpenPanel and GlassSavePanel subclasses of NSOpenPanel and NSSavePanel cannot be used, but the detection code in GlassApplication doesn't correctly detect this. The logic for switching to using NSOpenPanel and NSSavePanel directly in sandboxed mode was added as a fix for JDK-8092977.
21-11-2019