JDK-8259516 : Alerts sent by peer may not be received correctly during TLS handshake
  • Type: Bug
  • Component: security-libs
  • Sub-Component: javax.net.ssl
  • Affected Version: openjdk8u272,11,15
  • Priority: P3
  • Status: Open
  • Resolution: Unresolved
  • Submitted: 2021-01-09
  • Updated: 2024-04-10
Related Reports
Blocks :  
Relates :  
Description
During a handshake process, The client or server occasionally enter a loop of producers like in T12ClientHelloConsumer::consume 

https://github.com/openjdk/jdk/blob/7ddc2b5606cd5353a3c25dde476ff0e6ccc2ed23/src/java.base/share/classes/sun/security/ssl/ClientHello.java#L1102-L1121

The producers in the loop create and write to the socket. The client receives and starts processing the message immediately. In certain cases, where the network is faster than the computation, the peer can receive the message and respond before the loop of producers are completed. If the peer encounters a fatal condition, it would sends a fatal and closes the socket on its end. The currently running producer is unaware of this case and attempts to write to the socket nonetheless. This will cause the write to fail with a SocketException (Broken Pipe). 

Since the host has not attempted to read from the socket, it's unaware that the peer has sent a fatal and closed the socket. It ends up throwing SocketException instead of a SSLHandshakeException. 

Example of the Case
```
SERVER                                                                  CLIENT
*                                <------------                       CLIENT_HELLO
CLIENT_HELLO_CONSUMER
SERVER_HELLO_PRODUCER            ------------->                     SERVER_HELLO_CONSUMER
CERTIFICATE_PRODUCER             ------------->                     CERTIFICATE_CONSUMER
CERTIFICATE_STATUS               ------------->                     Still in CERTIFICATE_CONSUMER
START SERVER_KEY_EXCHANGE_PRODUCER
*                                <-------------                    CERTIFICATE_CONSUMER sends bad_certificate alert
*                                <-------------                    CLIENT_CLOSES_SOCKET
SERVER_KEY_EXCHANGE_PRODUCER
attempts to write to socket      --------||||
(broken_pipe exception)

Server throws a SocketException(broken_pipe) exception instead of SSLHandshakeException(bad_certificate)
```

Up until JDK-8237578, the issue was masked because SocketExceptions were being wrapped into SSLExceptions.
Comments
As in the case described above, The socket was closed by the client because the server sent a bad_certificate. However, the server will throw SocketException instead of an SSLHandshakeException. This is also seen in Test https://github.com/openjdk/jdk/blob/8452535d4937b26aa5318a8ca822acf5824be0c7/test/jdk/javax/net/ssl/SSLSession/TestEnabledProtocols.java#L89 The test expects an SSLException when the client may have closed the connection. The SocketException thrown is wrapped into an SSLException. Should the server throw a SocketException in this case?
28-01-2021

As the socket has been closed, are we really care about the peer alerts? is there a known compatibility impact?
28-01-2021