JDK-8087168 : Sporadic RMI communication errors: "java.io.InvalidClassException: Not a proxy"
  • Type: Bug
  • Component: core-libs
  • Sub-Component: java.rmi
  • Affected Version: 7u76
  • Priority: P4
  • Status: Open
  • Resolution: Unresolved
  • OS: windows_2012
  • CPU: x86
  • Submitted: 2015-06-08
  • Updated: 2017-06-28
Related Reports
Relates :  
Relates :  
Description
FULL PRODUCT VERSION :
java version "1.7.0_76"
Java(TM) SE Runtime Environment (build 1.7.0_76-b13)
Java HotSpot(TM) 64-Bit Server VM (build 24.76-b04, mixed mode)

ADDITIONAL OS VERSION INFORMATION :
Windows 2012 server

A DESCRIPTION OF THE PROBLEM :
We have a fairly complex server application using RMI for remote communication. From time to time we experience communication failures of the following form:

java.util.concurrent.ExecutionException: java.rmi.ServerException: RemoteException occurred in server thread; nested exception is: 
	java.rmi.UnmarshalException: error unmarshalling arguments; nested exception is: 
	java.io.InvalidClassException: Not a proxy

We believe this might be a regression which was introduced during a major overhaul of the internal cache management in class java.lang.reflect.Proxy between JDK 1.7.0_17 and JDK_1.7.0_72.

We have traced the root cause down to java.io.ObjectInputStream.readProxyDesc(boolean) :

...
            if ((cl = resolveProxyClass(ifaces)) == null) {
                resolveEx = new ClassNotFoundException("null class");
            } else if (!Proxy.isProxyClass(cl)) {
                throw new InvalidClassException("Not a proxy");
...

java.lang.reflect.Proxy.isProxyClass:

        return Proxy.class.isAssignableFrom(cl) && proxyClassCache.containsValue(cl);

Since 'cl' cannot be null it basically means that the assertion whether the returned object is a proxy is failing because between the call to 'resolveProxyClass' and 'Proxy.isProxyClass' the internally stored cache entry in the java.lang.reflect.WeakCache 'Proxy.proxyClassCache' has been removed.

Please note that older version (for example JDK1.7.0_17) did not have this additional check after the call to java.io.ObjectInputStream.resolveProxyClass(String[]).


STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
Unfortunately we have not yet been able to reproduce this deterministically.


REPRODUCIBILITY :
This bug can be reproduced occasionally.


Comments
Received following note from the submitter: ----------------------------------------------------------------------------------------------------- "On 7/30/2015 2:49 PM, .............. wrote: > Hi .........., > > I have noticed a comment in the filed ticket to keep strong references to exported objects in our code. Frankly I am not a big fan of this idea but I will nevertheless give it a shot. Note that we are already keeping weak references to our exported objects and not only strong references on our call stacks. > > I would like to point out (again) that the new cache management likely has introduced a possible race condition and I kindly request that the responsible developer spends time on an investigation. I have seen that the ticket has been flagged as a minor (P4) bug. Do you have an expected ETA when this will be looked at? I am happy to talk to the responsible developer if that helps. > > I am repeating the problematic section in the code for your convenience here. See ObjectInputStream.readProxyDesc : > > ... > try { > if ((cl = resolveProxyClass(ifaces)) == null) { > resolveEx = new ClassNotFoundException("null class"); > } else if (!Proxy.isProxyClass(cl)) { > throw new InvalidClassException("Not a proxy"); > ... > > The call to resolveProxyClass must have returned a non-null value which in turn means that some value has been added to the internal WeakCache. For this particular (and in fact the only!) use case of this class the WeakCache internally maps variable sized arrays of Strings which contain the class name(s) to their associated class. In addition a reverse map is maintained for the mapping of classes to their class name(s) array. Since the call to Proxy.isProxyClass in turn makes a lookup in the reverse map of the very same cache it means that in some rare cases the cache entry has already been removed. > > Best regards > ................ ------------------------------------------------------------------------------------------
30-07-2015

Note, the suggestion in JDK-8030943 is for the application to keep a strong reference to the *exported object* (not its stub) in a field instead of in a local variable. If the application doesn't keep a reference to the exported object, it's liable to be GC'ed unexpectedly, resulting in odd errors. This is definitely a change from earlier versions of the JDK. The reason is that Hotspot can detect that an object has become unreachable (and is therefore subject to collection) even if a reference is held in a local variable that is still in scope. I don't know whether that is the cause of this problem, however it's something that might be worth checking for.
24-06-2015

Received following response from the submitter: ------------------------------------------------------------------------------ -----Original Message----- From: ......... Sent: Montag, 8. Juni 2015 09:12 To: '..........' Subject: RE: Incident Report 9059113 : Sporadic RMI communication errors: Hi ........., As already mentioned in the ticket we are not able to reproduce this deterministically. It happens a few times per week in our test lab on different servers at different places in the code. I have found a similar issue here: https://bugs.openjdk.java.net/browse/JDK-8030943 without any progress since December 2013. We were hoping this would be fixed with Java 8 but the cache management in java.lang.reflect.Proxy has not been changed. Frankly we are not very happy with the suggested workaround to keep strong references to all of our exported RMI stubs. Regards, .... ---------------------------------------------------------------------------------
11-06-2015

This issue is difficult to reproduce in the absence of detailed information or a test case. I have written to the submitter for more information: -------------------------------------------------------------------------- On 6/8/2015 12:18 PM, ........ wrote: > Hi Marc, > > Can you share a simple test case that could help us evaluate this issue quickly? > > Thank You, > .......... ----------------------------------------------------------------------------------
08-06-2015