JDK-6538311 : OpenJDK build fails with 'cannot restore segment prot after reloc'
  • Type: Bug
  • Component: hotspot
  • Sub-Component: runtime
  • Affected Version: 7
  • Priority: P3
  • Status: Closed
  • Resolution: Fixed
  • OS: linux_2.6
  • CPU: x86
  • Submitted: 2007-03-23
  • Updated: 2012-10-08
  • Resolved: 2007-05-24
The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.
JDK 6 JDK 7 Other
6u4Fixed 7Fixed hs10Fixed
When building OpenJDK sources on Fedora 6, build fails with error:

./gamma: error while loading shared libraries: ./libjvm.so: cannot restore segment prot after reloc: Permission denied 

make[7]: Leaving directory `/home/openjdk/Desktop/opensource-testing/openjdk-build10-output/hotspot/outputdir/linux_i486_compiler2/product'
make[6]: Leaving directory `/home/openjdk/Desktop/opensource-testing/openjdk-build10-output/hotspot/outputdir/linux_i486_compiler2/product'
All done.
make[5]: Leaving directory `/home/openjdk/Desktop/opensource-testing/openjdk-build10-output/hotspot/outputdir/linux_i486_compiler2/product'
cd linux_i486_compiler2/product && ./test_gamma
./gamma: error while loading shared libraries: ./libjvm.so: cannot restore segment prot after reloc: Permission denied
make[4]: *** [product] Error 127
make[4]: Leaving directory `/home/openjdk/Desktop/opensource-testing/openjdk-build10-output/hotspot/outputdir'
make[3]: *** [generic_build2] Error 2
make[3]: Leaving directory `/home/openjdk/Desktop/opensource-testing/hotspot/make'
make[2]: *** [product] Error 2
make[2]: Leaving directory `/home/openjdk/Desktop/opensource-testing/hotspot/make'
make[1]: *** [hotspot-build] Error 2
make[1]: Leaving directory `/home/openjdk/Desktop/opensource-testing/control/make'
make: *** [dev-build] Error 2

EVALUATION The root cause of seeing this error (cannot restore segment prot after reloc) can be explained as following. When a DSO (shared object libraries) is loaded, at some point, the runtime linker determines that is needs to perform some text relocations to the DSO, so it uses mprotect call to set the mapped region to read/write. After text relocation, the code needs to be mapped back to read/exec which triggers the access check. Normally, a DSO does not need to do text relocation. However, due to the significant benefit we saw from the non-PIC version of libjvm.so, we are now compiling libjvm.so as non-PIC. To avoid this, we basically do chcon -t textrel_shlib_t libjvm.so after it is built. Certain checks to make sure SELinux is enabled are required, otherwise, chcon will fail. It turns out checking the return result of /usr/sbin/selinuxenabled command will be sufficient to see whether SELinux is enabled or not.

WORK AROUND Instead of allowing all unconfined executables to use libraries requiring text relocation, you could use an even smaller hammer, the unconfined_domain java_t type for java, and customize its policy. From Dan Walsh's blog, http://danwalsh.livejournal.com/ "There are a few unconfined domains in a targeted policy, the main reason for this is different transition rules. As was stated in previous blogs, we have started added some confinement to the unconfined_domains, in the form of executable memory checking. This has led to a few new domains. For example we may want to turn off execstack for most of the unconfined domain, but java applications by design need this access. So an unconfined_domain java_t was added to java apps with no policy defined."

SUGGESTED FIX Red Hat SELinux tools suggest the following: If you trust /export/ws/openjdk/control/build/rhel5-i586/hotspot/outputdir/linux_i486_compiler2/product/libjvm.so to run correctly, you can change the file context to textrel_shlib_t. "chcon -t textrel_shlib_t /export/ws/openjdk/control/build/rhel5-i586/hotspot/outputdir/linux_i486_compiler2/product/libjvm.so" The following command will allow this access: chcon -t textrel_shlib_t /export/ws/openjdk/control/build/rhel5-i586/hotspot/outputdir/linux_i486_compiler2/product/libjvm.so Perhaps the Makefiles can be modified with something similar to the below: if [ -x /usr/bin/chcon ] ; then /usr/bin/chcon -t textrel_shlib_t \ $OUTPUTDIR/hotspot/outputdir/linux_i486_compiler2/product/libjvm.so fi This is untested. Perhaps the test should be more elaborate.

WORK AROUND Disablining SELinux is applying a rather large hammer to work around this issue. One only needs to disable this one check. 1) Select System->Administration->SELinux Management 2) In the SELinux Management Tool which appears, Select "Boolean" from the menu on the left, creatively labeled "Select:" 3) Expand the "Memory Protection" group 4) Check the first item, labeled "Allow all unconfined executables to use libraries requiring text relocation ..."

EVALUATION See the comments section for more details. In summary, the problem is caused by building non-PIC library with SELinux enabled. We don't have an easy way to get around this for now. So the temporary solution is to ask people to disable SELinux temporarily if they want to build hotspot. The FAQ page will soon contain the problem and solution.

WORK AROUND To disable SELinux: 1)$ su root 2)# system-config-securitylevel 3)In the window that appears, select the SELinux tab 4) Disable SELinux.