JDK-6772697 : Unexpected execution attempt in memory (gamma/test_gamma)
  • Type: Bug
  • Component: hotspot
  • Sub-Component: runtime
  • Affected Version: 7
  • Priority: P4
  • Status: Closed
  • Resolution: Not an Issue
  • OS: linux
  • CPU: x86
  • Submitted: 2008-11-18
  • Updated: 2014-06-23
  • Resolved: 2014-06-23
Description
FULL PRODUCT VERSION :
java version "1.6.0_0"
IcedTea6 1.3.1 (Gentoo) Runtime Environment (build 1.6.0_0-b12)
OpenJDK 64-Bit Server VM (build 10.0-b19, mixed mode)

ADDITIONAL OS VERSION INFORMATION :
Linux localhost 2.6.26-hardened-r4 #2 SMP PREEMPT Wed Oct 15 20:24:20 CEST 2008 x86_64 Intel(R) Core(TM)2 Quad CPU Q6600 @ 2.40GHz GenuineIntel GNU/Linux

EXTRA RELEVANT SYSTEM CONFIGURATION :
Kernel hardened with PaX/Grsecurity + hardened toolchain (-> http://www.gentoo.org/proj/en/hardened/)

A DESCRIPTION OF THE PROBLEM :
During compilation test_gamma executes a task called gamma, which tries to execute something in a memory reagion, where this is not expected (and not allowed by PaX).

  See also https://bugs.gentoo.org/show_bug.cgi?id=244901

STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
1. Add PaX support to kernel
2. Compile OpenJDK

EXPECTED VERSUS ACTUAL BEHAVIOR :
EXPECTED -
No action triggering the termination by PaX, JDK compiling
ACTUAL -
Last lines of build before failure:

eam.o yieldingWorkgroup.o vectset.o  linux_amd64.o -Wl,-Bstatic -lstdc++
-Wl,-Bdynamic -lm -ldl -lpthread;                                                                   rm -f libjvm.so.1; ln -s libjvm.so libjvm.so.1;
                         if [ -x /usr/sbin/selinuxenabled ] ; then                                 /usr/sbin/selinuxenabled;                                               if [ $? = 0 ] ; then                                                      /usr/bin/chcon -t textrel_shlib_t libjvm.so;
                if [ $? != 0 ]; then                                                     echo "ERROR: Cannot chcon libjvm.so"; exit 1;
                fi                                                                    fi                                                                    fi                                                                  }
Linking vm...
{             echo Linking launcher...;                          gcc -m64 -Xlinker -O1 -m64 -export-dynamic  -L `pwd` -o gamma
launcher.o -ljvm -lm -ldl -lpthread;                      }
Linking launcher...
make[6]: Leaving directory
`/var/tmp/portage/dev-java/icedtea6-1.3.1/work/icedtea6-1.3.1/openjdk/control/build/linux-amd64/hotspot/outputdir/linux_amd64_compiler2/product'
All done.
make[5]: Leaving directory
`/var/tmp/portage/dev-java/icedtea6-1.3.1/work/icedtea6-1.3.1/openjdk/control/build/linux-amd64/hotspot/outputdir/linux_amd64_compiler2/product'
cd linux_amd64_compiler2/product && ./test_gamma
./test_gamma: line 10: 15951 Get��tet                ./${gamma:-gamma} -Xbatch
-showversion Queens < /dev/null
make[4]: *** [product] Fehler 137
make[4]: Leaving directory
`/var/tmp/portage/dev-java/icedtea6-1.3.1/work/icedtea6-1.3.1/openjdk/control/build/linux-amd64/hotspot/outputdir'
make[3]: *** [generic_build2] Fehler 2
make[3]: Leaving directory
`/var/tmp/portage/dev-java/icedtea6-1.3.1/work/icedtea6-1.3.1/openjdk/hotspot/make'
make[2]: *** [product] Fehler 2
make[2]: Leaving directory
`/var/tmp/portage/dev-java/icedtea6-1.3.1/work/icedtea6-1.3.1/openjdk/hotspot/make'
make[1]: *** [hotspot-build] Fehler 2
make[1]: Leaving directory
`/var/tmp/portage/dev-java/icedtea6-1.3.1/work/icedtea6-1.3.1/openjdk/control/make'
make: *** [stamps/icedtea.stamp] Fehler 2


output of Pax/Grsec:

log-2008-10-28-17:10:07:Oct 28 18:13:57 [kernel] PAX: execution attempt in:
<anonymous mapping>, 2f95ff054000-2f95ff2c4000 2f95ff054000
log-2008-10-28-17:10:07:Oct 28 18:13:57 [kernel] PAX: terminating task:
/var/tmp/portage/dev-java/icedtea6-1.3.1/work/icedtea6-1.3.1/openjdk/control/build/linux-amd64/hotspot/outputdir/linux_amd64_compiler2/product/gamma(gamma):15951,
uid/euid: 250/250, PC: 00002f95ff054060, SP: 000077dc7bf16c28
log-2008-10-28-17:10:07:Oct 28 18:13:57 [kernel] PAX: bytes at PC: 85 f6 0f 84
11 00 00 00 0f ae f0 0f ae 3f 48 83 c7 20 ff ce
log-2008-10-28-17:10:07:Oct 28 18:13:57 [kernel] PAX: bytes at SP-8:
log-2008-10-28-17:10:07:Oct 28 18:13:57 [kernel] grsec: denied resource
overstep by requesting 4096 for RLIMIT_CORE against limit 0 for
/var/tmp/portage/dev-java/icedtea6-1.3.1/work/icedtea6-1.3.1/openjdk/control/build/linux-amd64/hotspot/outputdir/linux_amd64_compiler2/product/gamma[gamma:15951]
uid/euid:250/250 gid/egid:250/250, parent
/var/tmp/portage/dev-java/icedtea6-1.3.1/work/icedtea6-1.3.1/openjdk/control/build/linux-amd64/hotspot/outputdir/linux_amd64_compiler2/product/test_gamma[test_gamma:15949]
uid/euid:250/250 gid/egid:250/250

REPRODUCIBILITY :
This bug can be reproduced always.

CUSTOMER SUBMITTED WORKAROUND :
Boot a kernel without PaX support and compile it there.

Comments
The gamma launcher is not around anymore, closing this as NAI
23-06-2014