JDK-8242294 : JSSE Client does not throw SSLException when an alert occurs during handshaking
  • Type: Bug
  • Component: security-libs
  • Sub-Component: javax.net.ssl
  • Affected Version: 11,14,15
  • Priority: P3
  • Status: Closed
  • Resolution: Fixed
  • Submitted: 2020-04-07
  • Updated: 2022-02-03
  • Resolved: 2020-04-08
The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.
JDK 11 JDK 13 JDK 15 JDK 8
11.0.8-oracleFixed 13.0.4Fixed 15 b18Fixed 8u261Fixed
Description
In 8u the JSSE implementation would have the client-side throw SSLExceptions if an alert condition occurs during handshaking and a read or write operation on the input/output streams occur.  In JDK 11 and onward, the new handshaker does not cause an exception to be delivered to the client when a read operation happens after a failed/alerted handshake.

On JDK 8u, the exception delivered to the client will look like this:

javax.net.ssl.SSLException: Connection has been shutdown: javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path validation failed: java.security.cert.CertPathValidatorException: Algorithm constraints check failed on keysize limits. RSA 952bit key used with certificate: <DN OMITTED> Usage was tls server
        at sun.security.ssl.SSLSocketImpl.checkEOF(SSLSocketImpl.java:1533)
        at sun.security.ssl.AppInputStream.read(AppInputStream.java:95)
        at sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:284)
        at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:326)
        at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:178)
        at java.io.InputStreamReader.read(InputStreamReader.java:184)
        at java.io.BufferedReader.fill(BufferedReader.java:161)
        at java.io.BufferedReader.readLine(BufferedReader.java:324)
        at java.io.BufferedReader.readLine(BufferedReader.java:389)
        at ServerClientSSLSocket.doClientSide(ServerClientSSLSocket.java:45)
        at ServerClientSSLSocket.main(ServerClientSSLSocket.java:62)
Caused by: javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path validation failed: java.security.cert.CertPathValidatorException: Algorithm constraints check failed on keysize limits. RSA 952bit key used with certificate: <DN OMITTED>. Usage was tls server
        at sun.security.ssl.Alerts.getSSLException(Alerts.java:192)
        at sun.security.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1946)
        at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:316)
        at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:310)
        at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1640)
        at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:223)
        at sun.security.ssl.Handshaker.processLoop(Handshaker.java:1037)
        at sun.security.ssl.Handshaker.process_record(Handshaker.java:965)
        at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:1064)
        at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1367)
        at sun.security.ssl.SSLSocketImpl.writeRecord(SSLSocketImpl.java:750)
        at sun.security.ssl.AppOutputStream.write(AppOutputStream.java:123)
        at sun.nio.cs.StreamEncoder.writeBytes(StreamEncoder.java:221)
        at sun.nio.cs.StreamEncoder.implFlushBuffer(StreamEncoder.java:291)
        at sun.nio.cs.StreamEncoder.implFlush(StreamEncoder.java:295)
        at sun.nio.cs.StreamEncoder.flush(StreamEncoder.java:141)
        at java.io.OutputStreamWriter.flush(OutputStreamWriter.java:229)
        at java.io.BufferedWriter.flush(BufferedWriter.java:254)
        at java.io.PrintWriter.flush(PrintWriter.java:320)
        at ServerClientSSLSocket.doClientSide(ServerClientSSLSocket.java:44)
        ... 1 more
Caused by: sun.security.validator.ValidatorException: PKIX path validation failed: java.security.cert.CertPathValidatorException: Algorithm constraints check failed on keysize limits. RSA 952bit key used with certificate: <DN OMITTED> Usage was tls server
        at sun.security.validator.PKIXValidator.doValidate(PKIXValidator.java:362)
        at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:259)
        at sun.security.validator.Validator.validate(Validator.java:262)
        at sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:330)
        at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:237)
        at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:132)
        at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1622)
        ... 16 more
Caused by: java.security.cert.CertPathValidatorException: Algorithm constraints check failed on keysize limits. RSA 952bit key used with certificate: <DN OMITTED>. Usage was tls server
        at sun.security.provider.certpath.PKIXMasterCertPathValidator.validate(PKIXMasterCertPathValidator.java:135)
        at sun.security.provider.certpath.PKIXCertPathValidator.validate(PKIXCertPathValidator.java:238)
        at sun.security.provider.certpath.PKIXCertPathValidator.validate(PKIXCertPathValidator.java:146)
        at sun.security.provider.certpath.PKIXCertPathValidator.engineValidate(PKIXCertPathValidator.java:85)
        at java.security.cert.CertPathValidator.validate(CertPathValidator.java:292)
        at sun.security.validator.PKIXValidator.doValidate(PKIXValidator.java:357)
        ... 22 more
Caused by: java.security.cert.CertPathValidatorException: Algorithm constraints check failed on keysize limits. RSA 952bit key used with certificate:<DN OMITTED> Usage was tls server
        at sun.security.util.DisabledAlgorithmConstraints$KeySizeConstraint.permits(DisabledAlgorithmConstraints.java:817)
        at sun.security.util.DisabledAlgorithmConstraints$Constraints.permits(DisabledAlgorithmConstraints.java:419)
        at sun.security.util.DisabledAlgorithmConstraints.permits(DisabledAlgorithmConstraints.java:167)
        at sun.security.provider.certpath.AlgorithmChecker.check(AlgorithmChecker.java:326)
        at sun.security.provider.certpath.PKIXMasterCertPathValidator.validate(PKIXMasterCertPathValidator.java:125)
        ... 27 more

While on 11 and later, you will see behavior similar to this:

$ java OPTIONS_OMITTED ServerClientSSLSocket
doServerSide start
doServerSide ready
doClientSide start
Server sslSocket: /10.100.192.143
response is null
Comments
Fix request (13u) Requesting backport to 13u for parity with 11u, applies cleanly.
02-06-2020
Fix request (11u) -- will label after testing completed. I would like to downport this for parity with 11.0.8-oracle. Applies clean except for copyright. Where jdk11u-dev has 2018, the patch wants to change 2019 to 2020. --- a/src/java.base/share/classes/sun/security/ssl/SSLSocketImpl.java +++ b/src/java.base/share/classes/sun/security/ssl/SSLSocketImpl.java @@ -1,5 +1,5 @@ /* - * Copyright (c) 1996, 2018, Oracle and/or its affiliates. All rights reserved. + * Copyright (c) 1996, 2020, Oracle and/or its affiliates. All rights reserved. * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. * * This code is free software; you can redistribute it and/or modify it
16-04-2020
URL: https://hg.openjdk.java.net/jdk/jdk/rev/e39e9ca550fa User: jnimeh Date: 2020-04-08 07:04:37 +0000
08-04-2020
A read operation on the input stream dives down into SSLSocketImpl's checkEOF method, which behaves differently in JDK 11+ vs. JDK 8. The latter throws an exception, the former returns true from checkEOF, which results in a -1 cascading upward through StreamDecoder's read methods, ultimately resulting in no exception and no returned data from the server (because it has already shut down the socket due to the reception of the alert from the client).
07-04-2020