JDK-8023956 : Provide a work-around to broken Linux 32 bit "Exec Shield" using CS for NX emulation (crashing with SI_KERNEL)
  • Type: Enhancement
  • Component: hotspot
  • Sub-Component: runtime
  • Priority: P3
  • Status: Closed
  • Resolution: Fixed
  • OS: linux
  • CPU: x86
  • Submitted: 2013-08-29
  • Updated: 2016-01-19
  • Resolved: 2013-09-25
The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.
JDK 7 JDK 8 Other
7u60Fixed 8Fixed hs25Fixed
Related Reports
Duplicate :  
Relates :  
Relates :  
Relates :  
Description
JDK-8015837 contains several crashes on 32 bit Linux due to a bug a patch from RedHat, part of "Exec Shield".

RHEL 5 & 6 are affected, and earlier Ubuntu releases 10.04 (LTS), 11.04, and 12.04 have also taken in the patch. Whilst earlier Exec Shield patches had a switch, current incantations have no work-around, no way to disable.

Details of the problem see RH bug filed: https://bugzilla.redhat.com/show_bug.cgi?id=996149

To summarize: to emulate NX feature on X86_32 code segment is used to limit execution to the highest executable VA. There is a tiny race on SMP MM invalidation code which can cause the lazy CS update code in trap handling to think a general protection fault wasn't cause by itself. This results in sending the JVM a useless SIGSEGV with si_code:SI_KERNEL, results in JVM signal handling forcing a dump.

The suggested work around (limited to 32 bit Linux): is to enable execution (PROT_EXEC) on a high address and execute some code. 

The memory layout on these distros is similar: TASK_LIMIT set to 3G (0xC0000000), and the main stack typically grows down from around here, so:
1) using a page in main stack at JVM start up, test for said layout (don't bother if its not what we expect, i.e. exotic Linux config/distro)
2) enable execution (mprotect)
3) Place a single "ret" instruction and call it.

This will cause CS to be updated to a high enough address to basically disable any further updates (due to code generation), effectively disabling the feature (which is already of questionable value, NX is the way to go).