JDK-8199440 : JDK selects wrong certificate during two-way SSL handshake
  • Type: Bug
  • Component: security-libs
  • Sub-Component: javax.net.ssl
  • Affected Version: 8,9,10
  • Priority: P3
  • Status: Closed
  • Resolution: Not an Issue
  • OS: linux
  • CPU: x86_64
  • Submitted: 2018-02-25
  • Updated: 2021-08-24
  • Resolved: 2021-08-24
The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.
JDK 18
18Resolved
Related Reports
Relates :  
JDK-8272875 - Change the default key manager to PKIX
Description
FULL PRODUCT VERSION :
java version "1.8.0_112"
Java(TM) SE Runtime Environment (build 1.8.0_112-b15)
Java HotSpot(TM) 64-Bit Server VM (build 25.112-b15, mixed mode)

ADDITIONAL OS VERSION INFORMATION :
Linux hades.workgroup 4.11.6-201.fc25.x86_64 #1 SMP Tue Jun 20 20:21:11 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux

A DESCRIPTION OF THE PROBLEM :
When the default Key Manager is used with the default of just a single keystore (set by system property javax.net.ssl.keyStore), and the JVM acts as a server for incoming HTTPS requests, and a client for outgoing HTTPS requests, both of which require two-way SSL, the JVM is incapable of completing the handshake for both incoming and outgoing connections.

The problem is in two methods of the class SunX509KeyManagerImpl:

- chooseServerAlias
- chooseClientAlias

Both end up selecting the first alias, rather than using some more complex selection criteria such as "extended key usage" extensions of the certificates.

Full details are available here:

https://github.com/maxant/jvm-two-way-ssl

STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
Run a downstream / backend server which requires two-way SSL for incoming connections, for example: https://github.com/maxant/jvm-two-way-ssl/blob/master/src/main/java/Back.java

Build a server/client application which will accept incoming connections and then make outgoing connections to the above backend server, for example https://github.com/maxant/jvm-two-way-ssl/blob/master/src/main/java/Middle.java

Connect to the socket on the middle application above, with a test client, for example https://github.com/maxant/jvm-two-way-ssl/blob/master/src/main/java/Front.java

The end to end connection from Front over Middle to Back will not be possible, either failing during SSL Handshake of the front to middle or middle to back, depending upon the hashcode of the alias/certificate combination.

EXPECTED VERSUS ACTUAL BEHAVIOR :
EXPECTED -
The end to end connection from front over middle to back work, and no exceptions occur.
ACTUAL -
SSL Exceptions occur and SSL handshakes fail. 

ERROR MESSAGES/STACK TRACES THAT OCCUR :
See 

https://github.com/maxant/jvm-two-way-ssl/blob/master/unsuccessful_client.md 

and 

https://github.com/maxant/jvm-two-way-ssl/blob/master/unsuccessful_server.md

REPRODUCIBILITY :
This bug can be reproduced always.

---------- BEGIN SOURCE ----------
https://github.com/maxant/jvm-two-way-ssl/blob/master/src/test/java/SunX509KeyManagerImplTest.java
---------- END SOURCE ----------

CUSTOMER SUBMITTED WORKAROUND :
Either make the application use one keystore per certificate, or patch the SunX509KeyManagerImpl, for example as shown here:

https://github.com/maxant/jvm-two-way-ssl/blob/master/src/main/java/PatchedSunX509KeyManagerImpl.java

A possible fix is here:

https://github.com/maxant/jvm-two-way-ssl/blob/master/SunX509KeyManagerImpl.patch
Comments
The SunX509 key manager is not expected to check the local certificate, and thus could have problem if there are multiple certificates in the key store. In such circumstances, the PKIX key manger should be used instead. I filed an enhancement to have PKIX as the default key manager in JDK-8272875.
24-08-2021
To reproduce the issue, follow the steps as outlined in the bug report. JDK 8u161 - Fail JDK 10-ea +44 - Fail Front log: Exception in thread "main" java.net.SocketException: Unexpected end of file from server at java.base/sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:862) at java.base/sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:689) at java.base/sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:859) at java.base/sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:689) at java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1604) at java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1509) at java.base/java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:527) at java.base/sun.net.www.protocol.https.HttpsURLConnectionImpl.getResponseCode(HttpsURLConnectionImpl.java:329) at Front.main(Front.java:24) Middle log: HTTP-Dispatcher, handling exception: java.net.SocketException: Software caused connection abort: socket write error %% Invalidated: [Session-5, TLS_DHE_DSS_WITH_AES_256_GCM_SHA384] HTTP-Dispatcher, SEND TLSv1.2 ALERT: fatal, description = unexpected_message HTTP-Dispatcher, WRITE: TLSv1.2 Alert, length = 2 HTTP-Dispatcher, Exception sending alert: java.net.SocketException: Software caused connection abort: socket write error HTTP-Dispatcher, called closeSocket() HTTP-Dispatcher, called closeInbound() HTTP-Dispatcher, fatal error: 80: Inbound closed before receiving peer's close_notify: possible truncation attack? javax.net.ssl.SSLException: Inbound closed before receiving peer's close_notify: possible truncation attack? %% Invalidated: [Session-4, TLS_DHE_DSS_WITH_AES_256_GCM_SHA384] Back log: HTTP-Dispatcher, fatal error: 46: General SSLEngine problem sun.security.validator.ValidatorException: Extended key usage does not permit use for TLS client authentication %% Invalidated: [Session-3, TLS_DHE_DSS_WITH_AES_256_GCM_SHA384] HTTP-Dispatcher, fatal: engine already closed. Rethrowing javax.net.ssl.SSLHandshakeException: General SSLEngine problem HTTP-Dispatcher, called closeInbound() HTTP-Dispatcher, fatal: engine already closed. Rethrowing javax.net.ssl.SSLException: Inbound closed before receiving peer's close_notify: possible truncation attack?
08-03-2018