JDK-8168940 : sun/security/ssl/ServerHandshaker/AnonCipherWithWantClientAuth.java failed with "Invalid TLS padding data"
  • Type: Bug
  • Component: security-libs
  • Sub-Component: javax.net.ssl
  • Affected Version: 9
  • Priority: P3
  • Status: Closed
  • Resolution: Duplicate
  • Submitted: 2016-10-31
  • Updated: 2016-12-26
  • Resolved: 2016-11-10
Related Reports
Cloners :  
JDK-8172005 - sun/security/ssl/ServerHandshaker/AnonCipherWithWantClientAuth.java failed with "Invalid TLS padding data"
Duplicate :  
JDK-8168064 - sun/security/ssl/ServerHandshaker/AnonCipherWithWantClientAuth.java failed with "Received fatal alert: handshake_failure"
Relates :  
JDK-8168064 - sun/security/ssl/ServerHandshaker/AnonCipherWithWantClientAuth.java failed with "Received fatal alert: handshake_failure"
Description
Test: sun/security/ssl/ServerHandshaker/AnonCipherWithWantClientAuth.java

Platform: SunOS 5.12 (sparcv9)

Log:
javax.net.ssl.SSLHandshakeException: Invalid TLS padding data
	at sun.security.ssl.Alerts.getSSLException(java.base@9-ea/Alerts.java:198)
	at sun.security.ssl.SSLSocketImpl.fatal(java.base@9-ea/SSLSocketImpl.java:1962)
	at sun.security.ssl.SSLSocketImpl.readRecord(java.base@9-ea/SSLSocketImpl.java:1031)
	at sun.security.ssl.SSLSocketImpl.readRecord(java.base@9-ea/SSLSocketImpl.java:968)
	at sun.security.ssl.SSLSocketImpl.performInitialHandshake(java.base@9-ea/SSLSocketImpl.java:1395)
	at sun.security.ssl.SSLSocketImpl.bytesInCompletePacket(java.base@9-ea/SSLSocketImpl.java:902)
	at sun.security.ssl.AppInputStream.read(java.base@9-ea/AppInputStream.java:144)
	at sun.security.ssl.AppInputStream.read(java.base@9-ea/AppInputStream.java:84)
	at AnonCipherWithWantClientAuth.doServerSide(AnonCipherWithWantClientAuth.java:109)
	at AnonCipherWithWantClientAuth.startServer(AnonCipherWithWantClientAuth.java:245)
	at AnonCipherWithWantClientAuth.<init>(AnonCipherWithWantClientAuth.java:202)
	at AnonCipherWithWantClientAuth.main(AnonCipherWithWantClientAuth.java:185)
	at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(java.base@9-ea/Native Method)
	at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(java.base@9-ea/NativeMethodAccessorImpl.java:62)
	at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(java.base@9-ea/DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(java.base@9-ea/Method.java:537)
	at com.sun.javatest.regtest.agent.MainWrapper$MainThread.run(MainWrapper.java:110)
	at java.lang.Thread.run(java.base@9-ea/Thread.java:844)
Caused by: javax.crypto.BadPaddingException: Invalid TLS padding data
	at sun.security.ssl.CipherBox.removePadding(java.base@9-ea/CipherBox.java:802)
	at sun.security.ssl.CipherBox.decrypt(java.base@9-ea/CipherBox.java:574)
	at sun.security.ssl.InputRecord.decrypt(java.base@9-ea/InputRecord.java:403)
	at sun.security.ssl.InputRecord.decrypt(java.base@9-ea/InputRecord.java:377)
	at sun.security.ssl.SSLSocketInputRecord.decodeInputRecord(java.base@9-ea/SSLSocketInputRecord.java:244)
	at sun.security.ssl.SSLSocketInputRecord.decode(java.base@9-ea/SSLSocketInputRecord.java:178)
	at sun.security.ssl.SSLSocketImpl.readRecord(java.base@9-ea/SSLSocketImpl.java:1026)
	... 15 more
Comments
I was wondering this is a intermittent failure. There is an update on the test case, JDK-8168064, which will try to avoid the anti-free-port impact. This test run before JDK-8168064 was merged. I will close this bug as the log file is not accessible any more and JDK-8168064 may have fixed the anti-free-port impact. Please file a new one and attach the log file (jtr) if it fails again in the future.
10-11-2016
Siba, Could you reproduce it? Is it reproducible only on Solaris? If so, on which versions? If so, this may be an issue in Solaris crypto libs.
31-10-2016