JDK-8196516 : libfontmanager must be built with LDFLAGS allowing unresolved symbols
  • Type: Bug
  • Component: client-libs
  • Sub-Component: java.awt
  • Affected Version: 8u161,10,11
  • Priority: P4
  • Status: Resolved
  • Resolution: Fixed
  • OS: linux
  • Submitted: 2018-01-31
  • Updated: 2019-05-16
  • Resolved: 2018-04-11
The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.
JDK 11 JDK 8
11 b10Fixed 8u192Fixed
Related Reports
Relates :  
JDK-8201524 - [AIX] Don't link libfontmanager against libawt_headless
Relates :  
JDK-8196218 - [linux] libfontmanager should be linked against headless awt library
Relates :  
JDK-8196509 - Linux UI applications broken by the build change for JDK-8196218
Description
See JDK-8196218 for the original issue which got backed out again via JDK-8196509. It seems to be reproducible on Ubuntu 16.04 LTS, but not on F27.
Comments
Latest webrev: http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8196516/webrev.02/
10-04-2018
I've managed to reproduce the problem in JDK-8196509 on a RHEL 7 system. Interestingly copying the built JDK image to a different place in the filesystem makes the problem go away. Anyway, when I set a breakpoint in GetSDOps (the place where it fails) and look at /proc/<pid>/maps then I see libawt_headless.so and libawt_xawt.so being mapped at that point. (gdb) bt #0 0x00007fffd6a14530 in GetSDOps (env=env@entry=0x7ffff000fa00, sData=0x7ffff7fd9230, callSetup=callSetup@entry=1 '\001') at /media/disk/openjdk-hs/src/java.desktop/share/native/libawt/java2d/SurfaceData.c:127 #1 0x00007fffd6a1455f in SurfaceData_GetOps (env=env@entry=0x7ffff000fa00, sData=<optimized out>) at /media/disk/openjdk-hs/src/java.desktop/share/native/libawt/java2d/SurfaceData.c:139 #2 0x00007fffd6763d01 in X11SurfaceData_GetOps (env=0x7ffff000fa00, sData=<optimized out>) at /media/disk/openjdk-hs/src/java.desktop/unix/native/common/java2d/x11/X11SurfaceData.c:296 #3 0x00007fffd676865d in Java_sun_java2d_xr_XRSurfaceData_freeXSDOPicture (env=<optimized out>, xsd=<optimized out>, pXSData=<optimized out>) at /media/disk/openjdk-hs/src/java.desktop/unix/native/libawt_xawt/java2d/x11/XRSurfaceData.c:135 #4 0x00007fffd7b2955b in () #5 0x00007ffff7fd9200 in () #6 0x00007fffd7b292bb in () #7 0x00007fffd7b29276 in () #8 0x00007ffff7fd91b8 in () #9 0x00007fffd55cca28 in () #10 0x00007ffff7fd9230 in () #11 0x00007fffd55d5d58 in () #12 0x0000000000000000 in () $ ps ax | grep java | grep SwingSet 3284 pts/3 S+ 0:05 gdb --args /media/disk/openjdk-hs/build/linux-x86_64-normal-server-release/images/jdk/bin/java -jar /home/sgehwolf/SwingSet2.jar 3286 pts/3 tl 0:00 /media/disk/openjdk-hs/build/linux-x86_64-normal-server-release/images/jdk/bin/java -jar /home/sgehwolf/SwingSet2.jar $ cat /proc/3286/maps | grep -E 'libawt_xawt|libawt_headless' 7fffd4c6a000-7fffd4c72000 r-xp 00000000 fc:11 6027 /media/disk/openjdk-hs/build/linux-x86_64-normal-server-release/images/jdk/lib/libawt_headless.so 7fffd4c72000-7fffd4e71000 ---p 00008000 fc:11 6027 /media/disk/openjdk-hs/build/linux-x86_64-normal-server-release/images/jdk/lib/libawt_headless.so 7fffd4e71000-7fffd4e72000 r--p 00007000 fc:11 6027 /media/disk/openjdk-hs/build/linux-x86_64-normal-server-release/images/jdk/lib/libawt_headless.so 7fffd4e72000-7fffd4e73000 rw-p 00008000 fc:11 6027 /media/disk/openjdk-hs/build/linux-x86_64-normal-server-release/images/jdk/lib/libawt_headless.so 7fffd673e000-7fffd679b000 r-xp 00000000 fc:11 6018 /media/disk/openjdk-hs/build/linux-x86_64-normal-server-release/images/jdk/lib/libawt_xawt.so 7fffd679b000-7fffd699a000 ---p 0005d000 fc:11 6018 /media/disk/openjdk-hs/build/linux-x86_64-normal-server-release/images/jdk/lib/libawt_xawt.so 7fffd699a000-7fffd699b000 r--p 0005c000 fc:11 6018 /media/disk/openjdk-hs/build/linux-x86_64-normal-server-release/images/jdk/lib/libawt_xawt.so 7fffd699b000-7fffd699e000 rw-p 0005d000 fc:11 6018 /media/disk/openjdk-hs/build/linux-x86_64-normal-server-release/images/jdk/lib/libawt_xawt.so
10-04-2018
Webrev: http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8196516/webrev.01/ Review-thread: http://mail.openjdk.java.net/pipermail/build-dev/2018-April/021575.html
09-04-2018
All we can do to fix the build issue is to filter "-Xlinker -z -Xlinker defs" and "-Wl,-z,defs" from LDFLAGS when libfontmanager is being built.
09-04-2018
$ nm -D ./build/linux-x86_64-normal-server-release/images/jdk/lib/libawt_xawt.so | grep T | grep AWTDrawGlyphList 0000000000028a50 T AWTDrawGlyphList $ nm -D ./build/linux-x86_64-normal-server-release/images/jdk/lib/libawt_headless.so | grep T | grep AWTDrawGlyphList 0000000000003f70 T AWTDrawGlyphList
09-04-2018
In summary we have: 1) An OpenJDK build might fail with configure option --with-extra-ldflags="-Xlinker -z -Xlinker defs" on JDK 11. Same is true for JDK 8 with configure option --with-extra-ldflags="-Wl,-z,defs" 2) On Linux, when linking libfontmanager.so, there might be unresolved symbols from AWT libraries. E.g. AWTDrawGlyphList. Those symbols are provided by *both* libawt_headless *and* libawt_xawt. The former is the headless lib useful for server deployments where libawt_xawt is not available. The latter is the headful library. On build-time it cannot be decided which one to link against. It must happen at runtime (via dlopen). If build-time-linked with libawt_headless, then on headful deployments both libraries get loaded and we run into the vagaries of runtime symbol resolutions as described by Phil above.
09-04-2018
Thanks, Erik. So it looks like we have to keep the filtering of linker flags for libfontmanager. Yet, there is two options to pass problematic linker flags: -Wl,-z,defs or -Xlinker -z -Xlinker defs and only one of them is being filtered out. It depends on the JDK version as different versions filter different syntax. So in my book, we should filter both variants from the linker flags so as to not run into this problem again if somebody happens to build with the non-filtered LD flags syntax.
06-04-2018
[~sgehwolf] The problem on Solaris is that the linker will not let us override the -z defs setting with a later command line option. On Solaris this is weirdly considered an error. The only way to disable -z defs seems to be to filter it out of the flags. We are also unable to link to either libawt_xawt or libawt_headless on Solaris, for the same reason as on Linux. It seems to be non deterministic which library is actually used at runtime (the one we linked with or the one we explicitly loaded at runtime).
05-04-2018
@Magnus: How does it break the build on solaris? Could you post the error message?
05-04-2018
Unfortuntately, the suggested fix will break the build on solaris. :-(
28-03-2018
Ah ok. He's on leave now until April, but with the first attempt reverted, I don't see any rush for this change.
12-02-2018
No, I just left it here as a suggestion for Severin.
08-02-2018
Worked for me with a slightly modified version on OpenJDK 8 (different filtering on LDFLAGS there): Linking libfontmanager.so /usr/x86_64-pc-linux-gnu/gcc-bin/6.3.0/g++ -Wl,--as-needed -Wl,-O1 -Wl,--hash-style=gnu -Wl,--sort-common -fuse-ld=gold -Wl,-z,\ defs -Xlinker -z -Xlinker defs -Xlinker -O1 -shared -L/home/andrew/builder/icedtea8/jdk/lib/amd64 -L/home/andrew/builder/icedte\ a8/jdk/lib/amd64/server -Xlinker -z -Xlinker origin -Xlinker -rpath -Xlinker \$ORIGIN -Xlinker --unresolved-symbols=ignore-all\ -Xlinker -version-script=/home/andrew/projects/openjdk/upstream/icedtea8/jdk/make/mapfiles/libfontmanager/mapfile-vers.openjdk\ -Xlinker -soname=libfontmanager.so -o /home/andrew/builder/icedtea8/jdk/lib/amd64/libfontmanager.so <object files> -lfreetype -lfontconfig -lfreetype -lawt -lm -lstdc++ -ljava -ljvm -lc It looks to be the right solution, as a similar option (-Xlinker --allow-shlib-undefined) is used in common/autoconf/toolchain.m4 so executables can link against these libraries with unresolved symbols. Have you posted this to a mailing list for review?
08-02-2018
I have now browsed through the ld man page on Linux, hunting for anything useful. What I found is --unresolved-symbols=ignore-all which seems to negate the effect of -z defs, so adding this flag instead of trying to string subst out the -z defs combination seems like a more robust solution. For me at least, the below patch is working (and it also works if you add your extra ldflags on the configure command line): diff -r 917868f73209 make/lib/Awt2dLibraries.gmk --- a/make/lib/Awt2dLibraries.gmk +++ b/make/lib/Awt2dLibraries.gmk @@ -683,9 +683,10 @@ hidevf w_novirtualdescr arrowrtn2, \ DISABLED_WARNINGS_microsoft := 4267 4244 4018 4090 4996 4146 4334 4819 4101, \ MAPFILE := $(BUILD_LIBFONTMANAGER_MAPFILE), \ - LDFLAGS := $(subst -Wl$(COMMA)-z$(COMMA)defs,,$(LDFLAGS_JDKLIB)) $(LDFLAGS_CXX_JDK) \ + LDFLAGS := $(LDFLAGS_JDKLIB) $(LDFLAGS_CXX_JDK) \ $(call SET_SHARED_LIBRARY_ORIGIN), \ LDFLAGS_unix := -L$(INSTALL_LIBRARIES_HERE), \ + LDFLAGS_linux := -Xlinker --unresolved-symbols=ignore-all, \ LDFLAGS_macosx := -undefined dynamic_lookup, \ LIBS := $(BUILD_LIBFONTMANAGER_FONTLIB), \ LIBS_unix := -lawt -ljava -ljvm $(LIBM) $(LIBCXX), \
01-02-2018
Linking against libawt_headless is going to be a problem here. A footnote in the description of the original bug https://bugs.openjdk.java.net/browse/JDK-8196218 says > As it turns out libawt_headless.so exports the missing symbols, but libfontmanager.so's link command does not include -lawt_headless. .. > libawt.so (the non-headless version) on the other hand doesn't have it: Well there's a mistake in there. The non-headless (ie headful) equivalent is actually libawt_xawt.so It does have the symbols - as well ! At runtime JDK determines dynamically if it wants to load libawt_headless.so or libawt_xawt.so It then uses System.load() - and dlopen - to load the one it wants. Having an 'ldd' dependency on headless, but trying to run headful, we end up with both libraries loaded. We are then at the mercy of the vagaries of the runtime symbol resolution which gets used. On Fedora 27 - where it can't be reproduced - the dlopened library seems to be taking precedence. This is probably what Solaris has been doing all along too. But on Ubuntu 16.04 and (someone else here confirmed) OEL 7 (similar to RHEL7) we seem to be getting the one that was linked at build time. So we need to leave the -Wl,z,defs line there .. and also not link against headless .. and pass whatever additional flags will cause the Fedora linker to allow unresolved symbols. An alternative would be a flag that requires that the symbols be resolved somehow at build time, but does not write the name of the library that satisfied the dependencies into the list of dependent libraries. Of course that would only prove that the symbols are defined in one of the two libraries ..
01-02-2018