United StatesChange Country, Oracle Worldwide Web Sites Communities I am a... I want to...
Bug ID: JDK-8014326 [OSX] All libjvm symbols are exported
JDK-8014326 : [OSX] All libjvm symbols are exported

Details
Type:
Bug
Submit Date:
2013-05-10
Status:
Closed
Updated Date:
2014-02-23
Project Name:
JDK
Resolved Date:
2013-06-20
Component:
hotspot
OS:
os_x
Sub-Component:
build
CPU:
Priority:
P2
Resolution:
Fixed
Affected Versions:
hs24,hs25
Fixed Versions:
hs25 (b39)

Related Reports
Backport:
Backport:
Backport:
Backport:
Backport:
Duplicate:
Duplicate:
Duplicate:
Relates:
Relates:
Relates:

Sub Tasks

Description
The OSX/BSD port was a modified copy of the linux port. On linux we use mapfiles and linker-scripts to control which symbols get exported from libjvm (the JNI_* and JVM_* primary interfaces plus some jio_ functions.) On OSX the linker does not support linker-scripts so the in the makefile we simple set LDNOMAP=true and this disables the use of the mapfiles. Combined with the fact that the default visibility for symbols is visible the consequence of this is that every symbol in the VM is exported!

The implications of this can be quite significant as the linker will bind symbols to functions based on the order it finds them in the various shared libraries. We discovered during some changes in the VM that some of the JDK native code is actually calling the VM's global operator new! It is even possible that a library might use mismatched functions from different shared objects - eg malloc from liba and free from libb.

There are a number of strange OSX crashes that are currently being investigated, and it is possible that the could be caused due to this linkage problem
                                    

Comments
A test build using an OSX compatible form of the mapfile for 7u was given to an internal developer who had been experiencing these crashes running netbeans, on a regular basis. After using the test build for a few days no crashes were seen. So the indications are that this is indeed the cause of the strange OSX crashes, hence the priority has been raised to P2.

Preliminary 7u webrev:

http://cr.openjdk.java.net/~dholmes/8104326/webrev.hs24.v1/
                                     
2013-05-30
Wasn't expecting this, but neither my MBP (10.6.8) or my MacMini (10.7.4) can
use "nm" on libjvm.dylib from Alejandro's HSX-24-B50 set-ver job:

$ nm hs24-b50-product/jre/lib/server/libjvm.dylib 
nm: for architecture x86_64 object: libjvm.dylib malformed object (unknown load command 14)

I'm going to guess that those bits were built with a newer toolset than I have
installed on my local Macs. To make things even more "special", I can't ssh
into the JPRT-hotspotwest Macs to see if those work.

                                     
2013-06-17
Here's the symbol count info for Alejandro's HSX-24-B50 set-ver job:

$ for lib in hs24-b50-*/*/*/*/libjvm.dylib; do
  echo $lib;
  echo "    global cnt:" `nm -g $lib | wc -l`;
  echo "    all cnt:   " `nm -a $lib | wc -l`;
done
hs24-b50-debug/jre/lib/server/libjvm.dylib
    global cnt: 61399
    all cnt:    301736
hs24-b50-fastdebug/jre/lib/server/libjvm.dylib
    global cnt: 45570
    all cnt:    50187
hs24-b50-product/jre/lib/server/libjvm.dylib
    global cnt: 37033
    all cnt:    38085


                                     
2013-06-17
Here's the symbol count info when David's patch is applied:

$ for lib in hs24-fix-*/*/*/*/libjvm.dylib; do
  echo $lib;
  echo "    global cnt:" `nm -g $lib | wc -l`;
  echo "    all cnt:   " `nm -a $lib | wc -l`;
done
hs24-fix-debug/jre/lib/server/libjvm.dylib
    global cnt: 441
    all cnt:    301792
hs24-fix-fastdebug/jre/lib/server/libjvm.dylib
    global cnt: 442
    all cnt:    50187
hs24-fix-product/jre/lib/server/libjvm.dylib
    global cnt: 435
    all cnt:    38085
                                     
2013-06-18
Here's a comparison of the updated BSD and Linux versions of mapfile-vers-debug
in HSX-24:

$ diff {bsd,linux}-funcs-debug
214c214
<                 JVM_handle_bsd_signal
---
>                 JVM_handle_linux_signal
226a227,229
>                 fork1
>                 numa_warn
>                 numa_error
227a231,233
>                 # Needed because there is no JVM interface for this.
>                 sysThreadAvailableStackWithSlack
> 

Line 214 is an obvious rename.
Lines 227-229 are functions not compiled into BSD/MacOS X.
Line 232 is an error on Linux; sysThreadAvailableStackWithSlack is only on Solaris,
but we won't fix that with this bug since we want the MacOS X fix in HSX24 and
in HSX25.
                                     
2013-06-18
Here's a comparison of the updated BSD and Linux versions of mapfile-vers-product
in HSX-24:

$ diff {bsd,linux}-funcs-product
214c214
<                 JVM_handle_bsd_signal
---
>                 JVM_handle_linux_signal
221a222,224
>                 fork1
>                 numa_warn
>                 numa_error
222a226,228
>                 # Needed because there is no JVM interface for this.
>                 sysThreadAvailableStackWithSlack
>

Line 214 is an obvious rename.
Lines 222-224 are functions not compiled into BSD/MacOS X.
Line 227 is an error on Linux; sysThreadAvailableStackWithSlack is only on Solaris,
but we won't fix that with this bug since we want the MacOS X fix in HSX24 and
in HSX25. 
                                     
2013-06-18
Here's a comparison of the functions in current BSD mapfile-vers-debug and the
updated BSD mapfile-vers-debug files in HSX-24:

$ diff bsd-funcs-debug{.orig,}
191a192
>                 JVM_SetNativeThreadName
215,232d215
<                 # Old reflection routines
<                 # These do not need to be present in the product build in JDK 1.4
<                 # but their code has not been removed yet because there will not
<                 # be a substantial code savings until JVM_InvokeMethod and
<                 # JVM_NewInstanceFromConstructor can also be removed; see
<                 # reflectionCompat.hpp.
<                 JVM_GetClassConstructor
<                 JVM_GetClassConstructors
<                 JVM_GetClassField
<                 JVM_GetClassFields
<                 JVM_GetClassMethod
<                 JVM_GetClassMethods
<                 JVM_GetField
<                 JVM_GetPrimitiveField
<                 JVM_NewInstance
<                 JVM_SetField
<                 JVM_SetPrimitiveField
< 
239,241d221
<                 fork1
<                 numa_warn
<                 numa_error
243,245d222
<                 # Needed because there is no JVM interface for this.
<                 sysThreadAvailableStackWithSlack
< 

Line 192: add missing JVM_SetNativeThreadName entry
Delete the old reflection routines!
Delete other functions not compiled into BSD/MacOS X.

                                     
2013-06-18
Here's a comparison of the functions in current BSD mapfile-vers-product and the
updated BSD mapfile-vers-product files in HSX-24:

$ diff bsd-funcs-product.orig bsd-funcs-product
191a192
>                 JVM_SetNativeThreadName
215,232d215
<                 # Old reflection routines
<                 # These do not need to be present in the product build in JDK 1.4
<                 # but their code has not been removed yet because there will not
<                 # be a substantial code savings until JVM_InvokeMethod and
<                 # JVM_NewInstanceFromConstructor can also be removed; see
<                 # reflectionCompat.hpp.
<                 JVM_GetClassConstructor
<                 JVM_GetClassConstructors
<                 JVM_GetClassField
<                 JVM_GetClassFields
<                 JVM_GetClassMethod
<                 JVM_GetClassMethods
<                 JVM_GetField
<                 JVM_GetPrimitiveField
<                 JVM_NewInstance
<                 JVM_SetField
<                 JVM_SetPrimitiveField
< 
239,241d221
<                 fork1
<                 numa_warn
<                 numa_error
243,245d222
<                 # Needed because there is no JVM interface for this.
<                 sysThreadAvailableStackWithSlack
< 

Line 192: add missing JVM_SetNativeThreadName entry
Delete the old reflection routines!
Delete other functions not compiled into BSD/MacOS X. 
                                     
2013-06-18
Here's the symbol count info for Alejandro's HSX-25-B38 set-ver job: 

$ for lib in hs25-b38-*/*/*/*/libjvm.dylib; do
  echo $lib;
  echo " global cnt:" `nm -g $lib | wc -l`;
  echo " all cnt: " `nm -a $lib | wc -l`;
done 
hs25-b38-fastdebug/jre/lib/server/libjvm.dylib
    global cnt: 47110
    all cnt:    51812
hs25-b38-product/jre/lib/server/libjvm.dylib
    global cnt: 38241
    all cnt:    39316
                                     
2013-06-18
Here's the symbol count info when David's patch is forward ported to HSX-25:

$ for lib in hs25-fix-*/*/*/*/libjvm.dylib; do
  echo $lib;
  echo " global cnt:" `nm -g $lib | wc -l`;
  echo " all cnt: " `nm -a $lib | wc -l`;
done 
hs25-fix-fastdebug/jre/lib/server/libjvm.dylib
    global cnt: 446
    all cnt:    51816
hs25-fix-product/jre/lib/server/libjvm.dylib
    global cnt: 440
    all cnt:    39320
                                     
2013-06-18
Here's a comparison of the updated BSD and Linux versions of mapfile-vers-debug
in HSX-25:

$ diff {bsd,linux}-funcs-debug
218c218
<                 JVM_handle_bsd_signal
---
>                 JVM_handle_linux_signal
230a231,233
>                 fork1
>                 numa_warn
>                 numa_error
231a235,237
>                 # Needed because there is no JVM interface for this.
>                 sysThreadAvailableStackWithSlack
> 

Line 218 is an obvious rename.
Lines 231-233 are functions not compiled into BSD/MacOS X.
Line 236 is an error on Linux; sysThreadAvailableStackWithSlack is only on Solaris,
but we won't fix that with this bug since we want the MacOS X fix in HSX24 and
in HSX25. 
                                     
2013-06-18
Here's a comparison of the updated BSD and Linux versions of mapfile-vers-product
in HSX-25:

$ diff {bsd,linux}-funcs-product
218c218
<                 JVM_handle_bsd_signal
---
>                 JVM_handle_linux_signal
225a226,228
>                 fork1
>                 numa_warn
>                 numa_error
226a230,232
>                 # Needed because there is no JVM interface for this.
>                 sysThreadAvailableStackWithSlack
>

Line 218 is an obvious rename.
Lines 226-228 are functions not compiled into BSD/MacOS X.
Line 231 is an error on Linux; sysThreadAvailableStackWithSlack is only on Solaris,
but we won't fix that with this bug since we want the MacOS X fix in HSX24 and
in HSX25. 
                                     
2013-06-18
Here's a comparison of the functions in current BSD mapfile-vers-debug and the
updated BSD mapfile-vers-debug files in HSX-25:

$ diff bsd-funcs-debug{.orig,}
195a196
>                 JVM_SetNativeThreadName
219,236d219
<                 # Old reflection routines
<                 # These do not need to be present in the product build in JDK 1.4
<                 # but their code has not been removed yet because there will not
<                 # be a substantial code savings until JVM_InvokeMethod and
<                 # JVM_NewInstanceFromConstructor can also be removed; see
<                 # reflectionCompat.hpp.
<                 JVM_GetClassConstructor
<                 JVM_GetClassConstructors
<                 JVM_GetClassField
<                 JVM_GetClassFields
<                 JVM_GetClassMethod
<                 JVM_GetClassMethods
<                 JVM_GetField
<                 JVM_GetPrimitiveField
<                 JVM_NewInstance
<                 JVM_SetField
<                 JVM_SetPrimitiveField
< 
248,250d230
<                 fork1
<                 numa_warn
<                 numa_error
252,254d231
<                 # Needed because there is no JVM interface for this.
<                 sysThreadAvailableStackWithSlack
< 

Line 196: add missing JVM_SetNativeThreadName entry
Delete the old reflection routines!
Delete other functions not compiled into BSD/MacOS X. 
                                     
2013-06-18
Here's a comparison of the functions in current BSD mapfile-vers-product and the
updated BSD mapfile-vers-product files in HSX-25:

$ diff bsd-funcs-product{.orig,}
195a196
>                 JVM_SetNativeThreadName
219,236d219
<                 # Old reflection routines
<                 # These do not need to be present in the product build in JDK 1.4
<                 # but their code has not been removed yet because there will not
<                 # be a substantial code savings until JVM_InvokeMethod and
<                 # JVM_NewInstanceFromConstructor can also be removed; see
<                 # reflectionCompat.hpp.
<                 JVM_GetClassConstructor
<                 JVM_GetClassConstructors
<                 JVM_GetClassField
<                 JVM_GetClassFields
<                 JVM_GetClassMethod
<                 JVM_GetClassMethods
<                 JVM_GetField
<                 JVM_GetPrimitiveField
<                 JVM_NewInstance
<                 JVM_SetField
<                 JVM_SetPrimitiveField
< 
243,245d225
<                 fork1
<                 numa_warn
<                 numa_error
247,249d226
<                 # Needed because there is no JVM interface for this.
<                 sysThreadAvailableStackWithSlack
<

Line 196: add missing JVM_SetNativeThreadName entry
Delete the old reflection routines!
Delete other functions not compiled into BSD/MacOS X. 
                                     
2013-06-18
My most recent HSX-25 vm.quick test run on MacOS X was for 8013057.
Here's the summary (testbase v8r09): new=0, known=69, passed=3288

For this fix, here's the HSX-25 vm.quick MacOS X summary (testbase v8r12):
new=0, known=70, pass=3305
                                     
2013-06-19
My most recent HSX-24 vm.quick test run on MacOS X was for 8013057.
Here's the summary (testbase v7r15): new=1, known=75, passed=3092

For this fix, here's the HSX-25 vm.quick MacOS X summary (testbase v7r15):
new=1, known=75, pass=3092
                                     
2013-06-19
Ron pointed out that I should see some space savings and we do:

34099260 - 30067308
4031952
18025464 - 15547636
2477828
12448152 - 10545824
1902328

That's debug, fastdebug and product... sizes in bytes...

                                     
2013-06-19
URL:   http://hg.openjdk.java.net/hsx/hotspot-rt/hotspot/rev/91acb82a8b7a
User:  dcubed
Date:  2013-06-19 23:56:51 +0000

                                     
2013-06-20
SQE OK, this fix really should be backported to hs24
                                     
2013-06-20
7u40-critical-request justification:

This bug fix is needed to prevent random crashes due to mismatched
functions from different libraries on MacOS X. The mismatches can
happen when the linker binds functions from libjvm.dylib with function
calls from other libraries. These incorrect bindings can happen in the
current system because libjvm.dylib exports all symbols.

This bug fix must be integrated in 7u40/HSX-24 in order to improve the
stability of the MacOS X version of Java. 7u40/HSX-24 and JDK8/HSX-25
versions of the fix have been created and run through the usual
pre-integration and JPRT test cycles.

All of the changes are limited to BSD/MacOS X. They include very small
changes to a build script and a Makefile. The two mapfiles have small
semantic changes, but those are obscured by the syntactic changes. To
make review easier, we provided "normalized" diffs to show the functions
that are added and removed without all the syntactic noise.

Since we are changing from exporting all symbols to only exporting
specific symbols, there is a risk that something unexpected may fail
due to a linkage error. In order to mitigate that risk, we've sanity
checked the new MacOS X mapfiles against their Linux and Solaris
equivalents. We've also integrated the HSX-25 version of the fix
first in order to have a broad range of testing done before trying
to push the HSX-24 version of the fix.

The following four (4) engineers have reviewed both the 7u40/HSX-24
and JDK8/HSX-25 versions of the fix:

    dcubed  - Daniel Daugherty
    dholmes - David Holmes
    rdurbin - Ron Durbin
    coleenp - Coleen Phillimore


Background: The MacOS X port was based on a copy of the Linux port. The
Linux mechanism for limiting which symbols are exported by a library is
different than the MacOS X mechanism. During the port, the limiting of
exported symbols was simply disabled, but no tracking bug was filed for
fixing that issue later. During the testing of a recent HSX-25 change,
it was discovered that a JDK native library was calling the VM's global
"new" operator instead of the intended "new" operator for that native
library. The reason for that strange "new" call was tracked down and
this bug was created. Since then, a number of other open issues have
been tracked down and closed as duplicates of this bug.

David Holmes created the HSX-24 version of the fix at the end of May
and had it tested by an Oracle internal developer that was experiencing
daily crashes with NetBeans. The developer was able to use the fixed
version for a few days without any crashes. Daniel Daugherty forwarded
ported the fix to HSX-25 in mid-June and is in the process of pushing
that fix to RT_Baseline for more extensive testing.

                                     
2013-06-20
URL:   http://hg.openjdk.java.net/hsx/hsx25/hotspot/rev/91acb82a8b7a
User:  amurillo
Date:  2013-06-28 15:53:59 +0000

                                     
2013-06-28



Hardware and Software, Engineered to Work Together