JDK-8252716 : Enabling FFDHE causes TLS handshake failures
  • Type: Bug
  • Component: security-libs
  • Sub-Component: javax.net.ssl
  • Affected Version: 8u261
  • Priority: P3
  • Status: Closed
  • Resolution: Not an Issue
  • Submitted: 2020-09-02
  • Updated: 2020-10-05
  • Resolved: 2020-10-05
Description
javax.net.ssl|SEVERE|C8|pool-16-thread-3|2020-08-13 10:14:42.823
PDT|TransportContext.java:319|Fatal (HANDSHAKE_FAILURE): Received fatal
alert: handshake_failure (
"throwable" : {
  javax.net.ssl.SSLHandshakeException: Received fatal alert:
handshake_failure
        at sun.security.ssl.Alert.createSSLException(Alert.java:131)
        at sun.security.ssl.Alert.createSSLException(Alert.java:117)
        at sun.security.ssl.TransportContext.fatal(TransportContext.java:314)
        at sun.security.ssl.Alert$AlertConsumer.consume(Alert.java:293)
        at sun.security.ssl.TransportContext.dispatch(TransportContext.java:187)
        at sun.security.ssl.SSLTransport.decode(SSLTransport.java:154)
        at sun.security.ssl.SSLSocketImpl.decode(SSLSocketImpl.java:1198)
        at sun.security.ssl.SSLSocketImpl.readHandshakeRecord(SSLSocketImpl.java:1107)
        at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:400)
        at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:372)
        at org.apache.http.conn.ssl.SSLConnectionSocketFactory.createLayeredSocket(SSLConnectionSocketFactory.java:436)
        at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:384)
Comments
Issue didn't turn out to be JDK related in the end. Looks like server intolerance for new FFDHE arguments.
05-10-2020

Asked submitter to test with "-Djsse.enableFFDHE=false" system property. FFDHE is new for 8u261. Seems like that provided relief and their TLS server was happy to proceed with handshake. See the 8u261 release note on FFDHE: https://www.oracle.com/java/technologies/javase/8u261-relnotes.html#JDK-8140436 Asked submitter to check if their server is TLS compliant ? Here's the relevant part from RFC AFAIK: ===== https://tools.ietf.org/html/rfc7919 4. Server Behavior If a compatible TLS server receives a Supported Groups extension from a client that includes any FFDHE group (i.e., any codepoint between 256 and 511, inclusive, even if unknown to the server), and if none of the client-proposed FFDHE groups are known and acceptable to the server, then the server MUST NOT select an FFDHE cipher suite. In this case, the server SHOULD select an acceptable non-FFDHE cipher suite from the client's offered list. If the extension is present with FFDHE groups, none of the client's offered groups are acceptable by the server, and none of the client's proposed non-FFDHE cipher suites are acceptable to the server, the server MUST end the connection with a fatal TLS alert of type insufficient_security(71). === Their server should have chosen one of the non-ffdhe values AFAIK: Supported Group: secp256r1 (0x0017) Supported Group: secp384r1 (0x0018) Supported Group: secp521r1 (0x0019)
18-09-2020

Will need more data - esp. from server logs on why it appears to send a fatal alert record.
03-09-2020