JDK-7016078 : javax.net.ssl.SSLException: Received fatal alert: internal_error starting JDK 7 b126
  • Type: Bug
  • Component: security-libs
  • Sub-Component: java.security
  • Affected Version: OpenJDK6,7
  • Priority: P2
  • Status: Closed
  • Resolution: Fixed
  • OS: generic,solaris
  • CPU: generic,x86,sparc
  • Submitted: 2011-01-31
  • Updated: 2017-08-21
  • Resolved: 2011-06-22
The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.
JDK 7 Other
7 b138Fixed OpenJDK6Resolved
Related Reports
Duplicate :  
JDK-7017521 - jdk7 b128 many reg tests fail ex. jdk/test/com/sun/net/httpserver Test6a.java fails on solaris-amd64
Duplicate :  
JDK-7016003 - OpenDS fails to start up on Solaris starting JDK 7 b126
Duplicate :  
JDK-7011238 - pit jdk7 b126 sqe test src/jhttp/http/JhttpClient4001 fails on solaris platform.
Duplicate :  
JDK-7032257 - Testcase failures: sun/security/pkcs11/ec/ReadCertificates.java
Description
Tests
	sun/management/jmxremote/bootstrap/RmiRegistrySslTest.sh	
	sun/management/jmxremote/bootstrap/RmiSslBootstrapTest.sh	

fail on specific Solaris machine starting JDK 7 b126 with
...
-------------------------------------------------------------
RmiRegistrySslTest : SSL RMIRegistry - SSL Lookup
-------------------------------------------------------------
RmiRegistry lookup...
KO: Got unexpected exception!
java.rmi.ConnectIOException: error during JRMP connection establishment; nested exception is: 
        javax.net.ssl.SSLException: Received fatal alert: internal_error
        at sun.rmi.transport.tcp.TCPChannel.createConnection(TCPChannel.java:304)
        at sun.rmi.transport.tcp.TCPChannel.newConnection(TCPChannel.java:202)
        at sun.rmi.server.UnicastRef.newCall(UnicastRef.java:340)
        at sun.rmi.registry.RegistryImpl_Stub.list(Unknown Source)
        at RmiRegistrySslTest.main(RmiRegistrySslTest.java:75)
Caused by: javax.net.ssl.SSLException: Received fatal alert: internal_error
        at sun.security.ssl.Alerts.getSSLException(Alerts.java:208)
        at sun.security.ssl.Alerts.getSSLException(Alerts.java:154)
        at sun.security.ssl.SSLSocketImpl.recvAlert(SSLSocketImpl.java:1854)
        at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:1024)
        at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1259)
        at sun.security.ssl.SSLSocketImpl.writeRecord(SSLSocketImpl.java:677)
        at sun.security.ssl.AppOutputStream.write(AppOutputStream.java:85)
        at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
        at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
        at java.io.DataOutputStream.flush(DataOutputStream.java:123)
        at sun.rmi.transport.tcp.TCPChannel.createConnection(TCPChannel.java:229)
        ... 4 more
----------System.err:(20/1366)----------
Exception in thread "main" java.rmi.ConnectIOException: error during JRMP connection establishment; nested exception is: 
        javax.net.ssl.SSLException: Received fatal alert: internal_error
        at sun.rmi.transport.tcp.TCPChannel.createConnection(TCPChannel.java:304)
        at sun.rmi.transport.tcp.TCPChannel.newConnection(TCPChannel.java:202)
        at sun.rmi.server.UnicastRef.newCall(UnicastRef.java:340)
        at sun.rmi.registry.RegistryImpl_Stub.list(Unknown Source)
        at RmiRegistrySslTest.main(RmiRegistrySslTest.java:75)
Caused by: javax.net.ssl.SSLException: Received fatal alert: internal_error
        at sun.security.ssl.Alerts.getSSLException(Alerts.java:208)
        at sun.security.ssl.Alerts.getSSLException(Alerts.java:154)
        at sun.security.ssl.SSLSocketImpl.recvAlert(SSLSocketImpl.java:1854)
        at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:1024)
        at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1259)
        at sun.security.ssl.SSLSocketImpl.writeRecord(SSLSocketImpl.java:677)
        at sun.security.ssl.AppOutputStream.write(AppOutputStream.java:85)
        at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
        at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
        at java.io.DataOutputStream.flush(DataOutputStream.java:123)
        at sun.rmi.transport.tcp.TCPChannel.createConnection(TCPChannel.java:229)
        ... 4 more
result: Failed. Execution failed: exit code 1
These tests are part of JDK workspace (look under jdk/test/sun/management/jmxremote/bootstrap/). The test verifies that RMI registry over SSL works. There is no separate peer - everything is done in one Java VM, the code for server part is in in JDK. jdk/src/share/classes/javax/rmi/ssl/SslRMIClientSocketFactory.java creates SSLSocketFactory, RMI writes few bytes to socket output stream and flushes it (sun/rmi/transport/tcp/TCPChannel.java:229).

Adding -Dcom.sun.net.ssl.enableECC=false makes the test pass.
Comments
EVALUATION Some PKCS11 implementations do not wrap EC keys in OCTET STRING. Make the changes required to accept keys in wrapped and unwrapped format.
10-03-2011
EVALUATION From the attached debug log, we can see that when disable ECC (-Dcom.sun.net.ssl.enableECC=false), the issue disappears. From the first debug log, which enables ECC algorithms, we can find the following exception in TLS server side: RMI TCP Connection(1)-10.133.160.187, handling exception: java.lang.RuntimeException: Could not parse key values The above exception may imply that the sun.security.pkcs11.P11KEY$P11ECPublicKey.fetchValues() run into problems. If removing SunPKCS11 provider from java.security, the test passed. Redirect the bug to PKCS11 for further evaluation. Personally, I guess it is caused by the failure to parse the elliptic curves, secp256r1.
01-02-2011
EVALUATION Please enable the debuggers and attach the debug logs. It would be nice to tell us where to find the test source code and how to parse the result.
01-02-2011
EVALUATION This is just telling us that the peer had an internal error, and sent us that message. What is the local/peer configurations? It may help to get the debug output. Please run again with: -Djavax.net.debug=all and -Djavax.net.debug=all -Dcom.sun.net.ssl.enableECC=false and attach the logs. At least we'll know what is being sent to the peer. Also, if you could describe the architecture of how this is put together, that will help. For example, is the machine where the hang occurs receiving requests from clients, and then sending those requests on to another server? *** (#1 of 1): [ UNSAVED ] ###@###.###
01-02-2011