JDK-8154236 : Deserialization of lambda causes ClassCastException
  • Type: Bug
  • Component: tools
  • Sub-Component: javac
  • Affected Version: 8,9,18
  • Priority: P4
  • Status: Open
  • Resolution: Unresolved
  • OS: generic
  • CPU: generic
  • Submitted: 2016-04-13
  • Updated: 2022-03-08
The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.
Other
tbd_majorUnresolved
Related Reports
Duplicate :  
Relates :  
Relates :  
Relates :  
Relates :  
Relates :  
Description
FULL PRODUCT VERSION :
java version "1.8.0_73"
Java(TM) SE Runtime Environment (build 1.8.0_73-b02)
Java HotSpot(TM) 64-Bit Server VM (build 25.73-b02, mixed mode)

ADDITIONAL OS VERSION INFORMATION :
Microsoft Windows [Version 6.3.9600]

A DESCRIPTION OF THE PROBLEM :
- Two (distinct) classes, B and C, both of which extend the same base class, A, which has a method, String f().
- Create a Supplier reference to method f() for an object of type B; call this bf [new B()::f].
- Create a Supplier reference to method f() for an object of type C; cal this cf [new C()::f].
- Serialize cf (ObjectOutputStream#writeObject)
- When the serialized cf is deserialized (ObjectInputStream#readObject), a ClassCastException is thrown saying that class C cannot be cast to class B

The problem seems to be with the generate byte code (using javap to decompile):
-  javac generates a call to invokevirtual that references the shared based class (invokevirtual SerializationTest$A.f:()Ljava/lang/String;) 
- the Eclipse compiler generates a call invokevirtual that references the actual class the lambda was created from 

-- javac generated -- 

  0: #109 invokestatic java/lang/invoke/LambdaMetafactory.altMetafactory:(Ljava/lang/invoke/MethodHandles$Lookup;Ljava/lang/String;Ljava/lang/invoke/MethodType;[Ljava/lang/Object;)Ljava/lang/invoke/CallSite;
    Method arguments:
      #110 ()Ljava/lang/Object;
      #111 invokevirtual SerializationTest$A.f:()Ljava/lang/String;
      #112 ()Ljava/lang/String;
      #113 5
      #114 0
  1: #109 invokestatic java/lang/invoke/LambdaMetafactory.altMetafactory:(Ljava/lang/invoke/MethodHandles$Lookup;Ljava/lang/String;Ljava/lang/invoke/MethodType;[Ljava/lang/Object;)Ljava/lang/invoke/CallSite;
    Method arguments:
      #110 ()Ljava/lang/Object;
      #111 invokevirtual SerializationTest$A.f:()Ljava/lang/String;
      #112 ()Ljava/lang/String;
      #113 5
      #114 0

-- Eclipse compiler generated --

BootstrapMethods:
  0: #172 invokestatic java/lang/invoke/LambdaMetafactory.altMetafactory:(Ljava/lang/invoke/MethodHandles$Lookup;Ljava/lang/String;Ljava/lang/invoke/MethodType;[Ljava/lang/Object;)Ljava/lang/invoke/CallSite;
    Method arguments:
      #173 ()Ljava/lang/Object;
      #176 invokestatic SerializationTest.lambda$0:(LSerializationTest$B;)Ljava/lang/String;
      #177 ()Ljava/lang/String;
      #178 1
  1: #172 invokestatic java/lang/invoke/LambdaMetafactory.altMetafactory:(Ljava/lang/invoke/MethodHandles$Lookup;Ljava/lang/String;Ljava/lang/invoke/MethodType;[Ljava/lang/Object;)Ljava/lang/invoke/CallSite;
    Method arguments:
      #179 ()Ljava/lang/Object;
      #182 invokestatic SerializationTest.lambda$1:(LSerializationTest$C;)Ljava/lang/String;
      #183 ()Ljava/lang/String;
      #178 1

STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
Run the program below. 

EXPECTED VERSUS ACTUAL BEHAVIOR :
EXPECTED -
No ClassCastException when the lambda is deserialized.
ACTUAL -
See exception below.

ERROR MESSAGES/STACK TRACES THAT OCCUR :
Exception in thread "main" java.io.IOException: unexpected exception type
        at java.io.ObjectStreamClass.throwMiscException(Unknown Source)
        at java.io.ObjectStreamClass.invokeReadResolve(Unknown Source)
        at java.io.ObjectInputStream.readOrdinaryObject(Unknown Source)
        at java.io.ObjectInputStream.readObject0(Unknown Source)
        at java.io.ObjectInputStream.readObject(Unknown Source)
        at scratch.SerializationTest.main(SerializationTest.java:31)
Caused by: java.lang.reflect.InvocationTargetException
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
        at java.lang.reflect.Method.invoke(Unknown Source)
        at java.lang.invoke.SerializedLambda.readResolve(Unknown Source)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
        at java.lang.reflect.Method.invoke(Unknown Source)
        ... 5 more
Caused by: java.lang.ClassCastException: SerializationTest$C cannot be cast to SerializationTest$B
        at SerializationTest.$deserializeLambda$(SerializationTest.java:10)
        ... 14 more

REPRODUCIBILITY :
This bug can be reproduced always.

---------- BEGIN SOURCE ----------
import java.io.ByteArrayInputStream;
import java.io.ByteArrayOutputStream;
import java.io.ObjectInputStream;
import java.io.ObjectOutputStream;
import java.io.Serializable;
import java.util.function.Supplier;

public class SerializationTest {

    static class A implements Serializable {
        public String f() { return toString(); }
    }

    static class B extends A { }

    static class C extends A { }

    public static void main(String[] args) throws Exception {
        Supplier<String> bs = (Supplier<String> & Serializable) new B()::f;
        Supplier<String> cs = (Supplier<String> & Serializable) new C()::f;

        ByteArrayOutputStream caos = new ByteArrayOutputStream();

        try (ObjectOutputStream coos = new ObjectOutputStream(caos);) {
            coos.writeObject(cs);
        }

        try (ObjectInputStream cis = new ObjectInputStream(new ByteArrayInputStream(caos.toByteArray()));) {
            Supplier<String> ccs = (Supplier<String>) cis.readObject();
        }

    }

}
---------- END SOURCE ----------

CUSTOMER SUBMITTED WORKAROUND :
Don't use multiple lambdas that reference the same method in a base class and that get serialized and deserialized in the same class.


Comments
Important observation for this particular test case: the "qualifying type" (i.e., the class named by the bytecode) of a method reference should be the same as the qualifying type of an invocation: the type of the receiver. javac is wrong to be using the type of the declaring class. See JDK-8059632. Fix that bug, and I think the issue with different captured types goes away.
17-03-2017

See also JDK-8174865, which observes the same kinds of problems for method references that differ in their superinterfaces (possibly also bridges, but maybe that's impossible). The ultimate solution is for SerializedLambda to store all parameters to LambdaMetafactory—see JDK-8174864—and for $deserialize$ to use them all as a key. Open question how we get there and what we do about compatibility.
17-03-2017

The situation is murkier than it seemed at first. The core of the issue is that the logic for deserializing a functional expression tries hard to reconstruct the information that was available at the callsite. Just looking at the BSM is often not enough - consider this case: interface F1 { void m(); } interface F2 { void m(); } class Test { void test() { Object o1 = (F1 & Serializable)Test::g; Object o2 = (F2 & Serializable)Test::g; } static void g() { } } IN this case we get only ONE BSM entry - the only difference between the two invokedynamic is in their dynamic type: 0: invokedynamic #2, 0 // InvokeDynamic #0:m:()LF1; 12: invokedynamic #5, 0 // InvokeDynamic #0:m:()LF2; So, in this case, the dynamic type is crucial in order to be able to reconstruct the serialized form and giving it the right functional target. This means that, in the general case, the lambda deserialization scheme is designed to work on a per-callsite basis rather than on a per-BSM basis. If two callsites differ in any way (including differences in the dynamic type - as for the example in this bug report), the deserialization logic will attempt to reconstruct different captures. While this approach is fine in principle (and even needed in cases like the one shown above where the dynamic type is the only source of information about what the target type actually is) - some information about the callsite is lost - such as the types of the captured arguments. Such information is not reified in the SerializedLambda class. Therefore, we need a way to validate the receiver type associated with a serialized lambda. This can be done by extracting the first captured argument, get its Class object and validating it - this could become part of the set of conditions checked in a deserialization case. I believe this logic is only needed for the special 'receiver' argument because: * method references can only capture 'this' * each lambda generates a different BSM, therefore no sharing can occur Another solution would be to simply do what Eclipse seems to do - which is to desugar bounded method references into lambdas - since lambdas generate unique BSM, no sharing occurs and deserialization is again unambiguous. Both solutions have their pros and cons. In terms of compatibility, the first solution is superior, because it merely adds to the set of checks that were already there, by making them more precise. So, it would still be possible to deserialize a method reference that was serialized using the old serialization scheme, as the differences will be limited in the $deserialize$ method, and will basically amount at an extra check in a deserializaton case. The second solution has more problem compatibility-wise - if javac started to desugar all bound method references as lambdas, it would be impossible (unless some ad-hoc migration aid is added) to deserialize a legacy lambda-less bound method reference. On the other hand, the first solution potentially affects the size of the $deserialize$ method (which is already a constraint - given that it takes ~600 serialized lambdas to hit the method code size limit). The second solution would retain the same $deserialize$ shape as now, so there would be no problems there.
22-11-2016

I've tried another, related example: //pkg/A.java package pkg; class A implements java.io.Serializable { public String f() { return toString(); } } //pkg/B.java package pkg; public class B extends A { } //pkg/C.java package pkg; public class C extends A { } //SerializationTest.java import java.io.ByteArrayInputStream; import java.io.ByteArrayOutputStream; import java.io.ObjectInputStream; import java.io.ObjectOutputStream; import java.io.Serializable; import java.util.function.Supplier; import pkg.*; class SerializationTest { public static void main(String[] args) throws Exception { Supplier<String> bs = (Supplier<String> & Serializable) new B()::f; Supplier<String> cs = (Supplier<String> & Serializable) new C()::f; ByteArrayOutputStream caos = new ByteArrayOutputStream(); try (ObjectOutputStream coos = new ObjectOutputStream(caos);) { coos.writeObject(cs); } try (ObjectInputStream cis = new ObjectInputStream(new ByteArrayInputStream(caos.toByteArray()));) { Supplier<String> ccs = (Supplier<String>) cis.readObject(); } } } And this example works fine. Since the compiler detects that A is inaccessible, it won't try to generate a shared method reference, so the translation scheme would be more similar to the one used by Eclipse. This means that the rule outlined above (one deserialization case per bootstrap method) should work well - and that accessibility is not a problem, since it has been handled already.
11-11-2016

Looking at the body of the deserialization method reveals the issue: /*synthetic*/ private static Object $deserializeLambda$(final java.lang.invoke.SerializedLambda lambda) { switch (lambda.getImplMethodName()) { case "f": if (lambda.getImplMethodKind() == 5 && lambda.getFunctionalInterfaceClass().equals("java/util/function/Supplier") && lambda.getFunctionalInterfaceMethodName().equals("get") && lambda.getFunctionalInterfaceMethodSignature().equals("()Ljava/lang/Object;") && lambda.getImplClass().equals("SerializationTest$A") && lambda.getImplMethodSignature().equals("()Ljava/lang/String;")) return java.lang.invoke.LambdaMetafactory.altMetafactory((SerializationTest.B)lambda.getCapturedArg(0)); if (lambda.getImplMethodKind() == 5 && lambda.getFunctionalInterfaceClass().equals("java/util/function/Supplier") && lambda.getFunctionalInterfaceMethodName().equals("get") && lambda.getFunctionalInterfaceMethodSignature().equals("()Ljava/lang/Object;") && lambda.getImplClass().equals("SerializationTest$A") && lambda.getImplMethodSignature().equals("()Ljava/lang/String;")) return java.lang.invoke.LambdaMetafactory.altMetafactory((SerializationTest.C)lambda.getCapturedArg(0)); break; } throw new IllegalArgumentException("Invalid lambda deserialization"); } In other words, both method references have an identical metafactory recipe: in fact they point at the same method - the only difference being the type of the captured argument. I believe the bug here is that we have only one metafactory recipe, and still we get two deserialization cases - more specifically, we should only get one deserialization case for a given BSM entry. The cast to Test$B/Test$C is unnecessary - given that there's only one underlying implementation for both references and that such implementation will be accepting a Test$A. However, using only Test$A in place of Test$B and Test$C could well result in accessibility issues (if both B and C are public subtypes of a package-private superclass A residing in a different package).
11-11-2016

Test Result: ######### OS: Windows 7 64-bit JDK: 8uXX: Fail (Scenario II) 9ea+111 : Fail (Scenario II) 8uXX: Pass (Scenario I) 9ea+111 : Pass (Scenario I) ================================================================== Scenario I: ######### import java.io.ByteArrayOutputStream; import java.io.ObjectInputStream; import java.io.ObjectOutputStream; import java.io.Serializable; import java.util.function.Supplier; public class SerializationTest { static class A implements Serializable { public String f() { return toString(); } } static class C extends A { } static class B extends A { } public static void main(String[] args) throws Exception { Supplier<String> cs = (Supplier<String> & Serializable) new C()::f; Supplier<String> bs = (Supplier<String> & Serializable) new B()::f; ByteArrayOutputStream caos = new ByteArrayOutputStream(); try (ObjectOutputStream coos = new ObjectOutputStream(caos);) { coos.writeObject(cs); System.out.println("Write cs"); coos.writeObject(bs); System.out.println("Write bs"); } try (ObjectInputStream cis = new ObjectInputStream(new ByteArrayInputStream(caos.toByteArray()));) { Supplier<String> ccs = (Supplier<String>) cis.readObject(); System.out.println("Read "); } } } -------------------------------- O/P: Write cs Write bs Read -------------------------------- ============================================================================= Scenario II: ########## package com.abhijit.oracle.tools; import java.io.ByteArrayInputStream; import java.io.ByteArrayOutputStream; import java.io.ObjectInputStream; import java.io.ObjectOutputStream; import java.io.Serializable; import java.util.function.Supplier; public class SerializationTest { static class A implements Serializable { public String f() { return toString(); } } static class C extends A { } static class B extends A { } public static void main(String[] args) throws Exception { Supplier<String> bs = (Supplier<String> & Serializable) new B()::f; Supplier<String> cs = (Supplier<String> & Serializable) new C()::f; ByteArrayOutputStream caos = new ByteArrayOutputStream(); try (ObjectOutputStream coos = new ObjectOutputStream(caos);) { coos.writeObject(cs); System.out.println("Write cs"); coos.writeObject(bs); System.out.println("Write bs"); } try (ObjectInputStream cis = new ObjectInputStream(new ByteArrayInputStream(caos.toByteArray()));) { Supplier<String> ccs = (Supplier<String>) cis.readObject(); System.out.println("Read "); } } } ---------------------------- O/P: Write cs Write bs Exception in thread "main" java.io.IOException: unexpected exception type at java.io.ObjectStreamClass.throwMiscException(ObjectStreamClass.java:1582) at java.io.ObjectStreamClass.invokeReadResolve(ObjectStreamClass.java:1154) at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1810) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351) at java.io.ObjectInputStream.readObject(ObjectInputStream.java:371) at com.abhijit.oracle.tools.SerializationTest.main(SerializationTest.java:34) Caused by: java.lang.reflect.InvocationTargetException at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at java.lang.invoke.SerializedLambda.readResolve(SerializedLambda.java:230) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at java.io.ObjectStreamClass.invokeReadResolve(ObjectStreamClass.java:1148) ... 4 more Caused by: java.lang.ClassCastException: com.abhijit.oracle.tools.SerializationTest$C cannot be cast to com.abhijit.oracle.tools.SerializationTest$B at com.abhijit.oracle.tools.SerializationTest.$deserializeLambda$(SerializationTest.java:10) ... 14 more
14-04-2016