JDK-8155659 : Bootstrapping issues with lambdas in custom SecurityManager
  • Type: Bug
  • Component: core-libs
  • Sub-Component: java.lang.invoke
  • Affected Version: 8,9
  • Priority: P4
  • Status: Open
  • Resolution: Unresolved
  • Submitted: 2016-04-28
  • Updated: 2018-09-27
The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.
Other
tbdUnresolved
Related Reports
Relates :  
JDK-8156930 - Bootstrapping circularity when profiling lambda-enabled code with Solaris Perf Analyzer
Relates :  
JDK-8158851 - MH.publicLookup() init circularity, triggered by custom SecurityManager with String concat and -limitmods java.base
Relates :  
JDK-8155090 - String concatenation fails with a custom SecurityManager that uses concatenation
Description
This issue is similar to JDK-8155090, but with lambdas:

public class WithLambdas {
    public static void main(String[] args) throws Throwable {
        SecurityManager sm = new SecurityManager() {
            @Override
            public void checkPermission(Permission perm) {
                  Runnable r = () -> System.out.println("Boo");
            }
        };
        System.setSecurityManager(sm);
        ClassLoader cl = new ClassLoader() {};
    }
}

...fails with bootstrapping errors, because lambda linkage requires java.lang.invoke, which does privileged calls, which doubles back on custom SecurityManager. This eventually fails with the stack overflow:

java.lang.BootstrapMethodError: call site initialization exception
	at java.lang.invoke.CallSite.makeSite(java.base@9-internal/CallSite.java:347)
	at java.lang.invoke.MethodHandleNatives.linkCallSiteImpl(java.base@9-internal/MethodHandleNatives.java:250)
	at java.lang.invoke.MethodHandleNatives.linkCallSite(java.base@9-internal/MethodHandleNatives.java:240)
	at WithLambdas$1.checkPermission(WithLambdas.java:40)
	at java.lang.Class.checkMemberAccess(java.base@9-internal/Class.java:2585)
	at java.lang.Class.getDeclaredConstructors(java.base@9-internal/Class.java:2197)
	at java.lang.invoke.InnerClassLambdaMetafactory$1.run(java.base@9-internal/InnerClassLambdaMetafactory.java:198)
	at java.lang.invoke.InnerClassLambdaMetafactory$1.run(java.base@9-internal/InnerClassLambdaMetafactory.java:195)
	at java.security.AccessController.doPrivileged(java.base@9-internal/Native Method)
	at java.lang.invoke.InnerClassLambdaMetafactory.buildCallSite(java.base@9-internal/InnerClassLambdaMetafactory.java:194)
	at java.lang.invoke.LambdaMetafactory.metafactory(java.base@9-internal/LambdaMetafactory.java:304)
	at java.lang.invoke.CallSite.makeSite(java.base@9-internal/CallSite.java:308)
	at java.lang.invoke.MethodHandleNatives.linkCallSiteImpl(java.base@9-internal/MethodHandleNatives.java:250)
	at java.lang.invoke.MethodHandleNatives.linkCallSite(java.base@9-internal/MethodHandleNatives.java:240)
	at WithLambdas$1.checkPermission(WithLambdas.java:40)
	at java.lang.Class.checkMemberAccess(java.base@9-internal/Class.java:2585)
	at java.lang.Class.getDeclaredConstructors(java.base@9-internal/Class.java:2197)
	at java.lang.invoke.InnerClassLambdaMetafactory$1.run(java.base@9-internal/InnerClassLambdaMetafactory.java:198)
	at java.lang.invoke.InnerClassLambdaMetafactory$1.run(java.base@9-internal/InnerClassLambdaMetafactory.java:195)
	at java.security.AccessController.doPrivileged(java.base@9-internal/Native Method)
	at java.lang.invoke.InnerClassLambdaMetafactory.buildCallSite(java.base@9-internal/InnerClassLambdaMetafactory.java:194)
	at java.lang.invoke.LambdaMetafactory.metafactory(java.base@9-internal/LambdaMetafactory.java:304)
	at java.lang.invoke.CallSite.makeSite(java.base@9-internal/CallSite.java:308)
	at java.lang.invoke.MethodHandleNatives.linkCallSiteImpl(java.base@9-internal/MethodHandleNatives.java:250)
	at java.lang.invoke.MethodHandleNatives.linkCallSite(java.base@9-internal/MethodHandleNatives.java:240)
	at WithLambdas$1.checkPermission(WithLambdas.java:40)
Comments
Here's how we can break the cycle: @Override CallSite buildCallSite() throws LambdaConversionException { final Class<?> innerClass = spinInnerClass(); if (invokedType.parameterCount() == 0) { Object inst; try { MethodHandle mh = MethodHandles.Lookup.IMPL_LOOKUP.findConstructor(innerClass, MethodType.methodType(void.class)); inst = mh.invoke(); // Could also use invokeBasic, which although unsafe is more efficient } catch (Throwable t) { throw new LambdaConversionException(t); } return new ConstantCallSite(MethodHandles.constant(samBase, inst)); } else { try { UNSAFE.ensureClassInitialized(innerClass); return new ConstantCallSite( MethodHandles.Lookup.IMPL_LOOKUP .findStatic(innerClass, NAME_FACTORY, invokedType)); } catch (ReflectiveOperationException e) { throw new LambdaConversionException("Exception finding constructor", e); } } }
10-02-2017
Indeed, on latest jdk9/dev it fails with stack overflow. I have updated the bug, and here is the test I tried: http://cr.openjdk.java.net/~shade/8155659/WithLambdas.java I don't think there is anything easy to do here: LambdaMetafactory needs SecurityManager.checkPermission to work.
17-05-2016
The failure mode with JDK 8 is BootstrapMethodError caused by a stack overflow. To date we've had to tell people to workaround the issue by avoiding lambda expressions and method references in custom security managers. For JDK 9 then the issue will likely move around until a solution is found. As regards this specific sighting then assume that publicLookup() will change in JDK 9. We had to move to an unnamed module for the initial integration but we know how to move it back to java.lang.Object. If this is causing problems then we can force it to be initialized early by calling publicLookup() in System::initPhase3 before the security manager is set. We can also replace the lambda usage in jdk.internal.loader.BuiltinClassLoader but we need to keep the changes to a minimum as there are changes coming for a future refresh.
14-05-2016