JDK-8333317 : Test sun/security/pkcs11/sslecc/ClientJSSEServerJSSE.java failed with: Invalid ECDH ServerKeyExchange signature
  • Type: Bug
  • Component: security-libs
  • Sub-Component: javax.crypto:pkcs11
  • Affected Version: 23,24
  • Priority: P3
  • Status: Resolved
  • Resolution: Fixed
  • Submitted: 2024-05-31
  • Updated: 2024-11-11
  • Resolved: 2024-08-01
The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.
JDK 11 JDK 17 JDK 21 JDK 23 JDK 24
11.0.27-oracleFixed 17.0.14-oracleFixed 21.0.6-oracleFixed 23.0.2Fixed 24 b10Fixed
Related Reports
Duplicate :  
JDK-8335448 - sun/security/pkcs11/ec/TestECDSA2.java failed with Error: Signature verification failed
Relates :  
JDK-8321543 - Update NSS to version 3.96
Relates :  
JDK-8316183 - sun/security/pkcs11/sslecc/ClientJSSEServerJSSE.java fails intermittently on Linux ppc64le
Sub Tasks
JDK-8333829 :  
ProblemList sun/security/pkcs11/sslecc/ClientJSSEServerJSSE.java due to JDK-8333317 - Resolved
Description
Test: sun/security/pkcs11/sslecc/ClientJSSEServerJSSE.java

javax.net.ssl.SSLHandshakeException: (handshake_failure) Received fatal alert: handshake_failure
	at java.base/sun.security.ssl.Alert.createSSLException(Alert.java:130)
	at java.base/sun.security.ssl.Alert.createSSLException(Alert.java:117)
	at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:365)
	at java.base/sun.security.ssl.Alert$AlertConsumer.consume(Alert.java:287)
	at java.base/sun.security.ssl.TransportContext.dispatch(TransportContext.java:204)
	at java.base/sun.security.ssl.SSLTransport.decode(SSLTransport.java:172)
	at java.base/sun.security.ssl.SSLSocketImpl.decode(SSLSocketImpl.java:1507)
	at java.base/sun.security.ssl.SSLSocketImpl.readHandshakeRecord(SSLSocketImpl.java:1422)
	at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:455)
	at java.base/sun.security.ssl.SSLSocketImpl.ensureNegotiated(SSLSocketImpl.java:924)
	at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(SSLSocketImpl.java:1015)
	at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(SSLSocketImpl.java:975)
	at CipherTest$Server.handleRequest(CipherTest.java:80)
	at JSSEServer$1.run(JSSEServer.java:87)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
	at java.base/java.lang.Thread.run(Thread.java:1575)
javax.net.ssl.SSLHandshakeException: (handshake_failure) Invalid ECDH ServerKeyExchange signature
	at java.base/sun.security.ssl.Alert.createSSLException(Alert.java:130)
	at java.base/sun.security.ssl.Alert.createSSLException(Alert.java:117)
	at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:365)
	at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:321)
	at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:312)
	at java.base/sun.security.ssl.ECDHServerKeyExchange$ECDHServerKeyExchangeMessage.<init>(ECDHServerKeyExchange.java:312)
	at java.base/sun.security.ssl.ECDHServerKeyExchange$ECDHServerKeyExchangeConsumer.consume(ECDHServerKeyExchange.java:523)
	at java.base/sun.security.ssl.ServerKeyExchange$ServerKeyExchangeConsumer.consume(ServerKeyExchange.java:122)
	at java.base/sun.security.ssl.SSLHandshake.consume(SSLHandshake.java:393)
	at java.base/sun.security.ssl.HandshakeContext.dispatch(HandshakeContext.java:476)
	at java.base/sun.security.ssl.HandshakeContext.dispatch(HandshakeContext.java:447)
	at java.base/sun.security.ssl.TransportContext.dispatch(TransportContext.java:201)
	at java.base/sun.security.ssl.SSLTransport.decode(SSLTransport.java:172)
	at java.base/sun.security.ssl.SSLSocketImpl.decode(SSLSocketImpl.java:1507)
	at java.base/sun.security.ssl.SSLSocketImpl.readHandshakeRecord(SSLSocketImpl.java:1422)
	at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:455)
	at java.base/sun.security.ssl.SSLSocketImpl.ensureNegotiated(SSLSocketImpl.java:924)
	at java.base/sun.security.ssl.SSLSocketImpl$AppOutputStream.write(SSLSocketImpl.java:1293)
	at java.base/java.io.OutputStream.write(OutputStream.java:124)
	at CipherTest$Client.sendRequest(CipherTest.java:269)
	at JSSEClient.runTest(JSSEClient.java:74)
	at CipherTest$Client.run(CipherTest.java:252)
	at java.base/java.lang.Thread.run(Thread.java:1575)
java.lang.Exception: *** Test 'Client JSSE - Server JSSE' failed ***
	at CipherTest.run(CipherTest.java:209)
	at CipherTest.main(CipherTest.java:337)
	at ClientJSSEServerJSSE.main(ClientJSSEServerJSSE.java:67)
	at PKCS11Test.premain(PKCS11Test.java:820)
	at PKCS11Test.testNSS(PKCS11Test.java:485)
	at PKCS11Test.main(PKCS11Test.java:192)
	at ClientJSSEServerJSSE.main(ClientJSSEServerJSSE.java:57)
	at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
	at java.base/java.lang.reflect.Method.invoke(Method.java:580)
	at com.sun.javatest.regtest.agent.MainWrapper$MainTask.run(MainWrapper.java:138)
	at java.base/java.lang.Thread.run(Thread.java:1575)
Comments
Fix request [17u] I backport this for parity with 17.0.14-oracle. No risk, only a test change. Clean backport from 21. Pkcs11 tests pass. Sap nightly testing passed.
09-11-2024
A pull request was submitted for review. Branch: master URL: https://git.openjdk.org/jdk17u-dev/pull/3037 Date: 2024-11-08 09:00:56 +0000
08-11-2024
Fix request [21u] I backport this for parity with 21.0.6-oracle. No risk, only a test change Resolved ProblemList Test passes. SAP nightly testing passed.
22-10-2024
A pull request was submitted for review. Branch: master URL: https://git.openjdk.org/jdk21u-dev/pull/1071 Date: 2024-10-21 14:51:19 +0000
21-10-2024
[jdk23u-fix-request] Approval Request from Ramesh Gangadhar Backporting to JDK23u release
10-10-2024
A pull request was submitted for review. Branch: master URL: https://git.openjdk.org/jdk23u/pull/148 Date: 2024-10-10 03:18:28 +0000
10-10-2024
This patch should be ported to all update releases. The getNSSVersion() test logic in older releases is flawed and will cause TestDSAKeyLength.java to fail once NSS libraries > 3.99 are in use Ideally, the bug synopsis should have been updated to something like "Fix up getNSSVersion() test logic" before being committed to main line.
04-10-2024
Changeset: c6f0a35e Branch: master Author: Matthew Donovan <mdonovan@openjdk.org> Date: 2024-08-01 11:40:44 +0000 URL: https://git.openjdk.org/jdk/commit/c6f0a35e9e3eeaab1e238e8712051a626b337e0b
01-08-2024
It is likely that the NSS issue is also causing JDK-8335448. This patch should also fix that intermittent test failure.
12-07-2024
A pull request was submitted for review. Branch: master URL: https://git.openjdk.org/jdk/pull/19855 Date: 2024-06-24 12:07:49 +0000
09-07-2024
I added this test to the ProblemList until we can build and test the new version of NSS.
10-06-2024
Likely related to NSS upgrade in JDK-8321543 NSS 3.101 may have a fix (https://bugzilla.mozilla.org/show_bug.cgi?id=1893404)
10-06-2024
Initial investigation as per LOG, the failure Test Cases are grouped into 2: 1) Invalid ECDH ServerKeyExchange signature with following combination, Config: TLSv1.2, TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA, ECDSA keypair, ECHDE KeyExchange support with Mutual Authentication enabled Server Config: ECDSA keypair with Mutual Authentication enabled. Client Config: ECDSA keypair with Mutual Authentication enabled. [Failure count: 1 till date] 2) Invalid CertificateVerify message: invalid signature with following combination, Config: TLSv1.1, SSL_RSA_WITH_NULL_MD5, RSA keypair, RSA KeyExchange support with Mutual Authentication enabled Server Config: RSA keypair with Mutual Authentication enabled. Client Config: ECDSA keypair with Mutual Authentication enabled. [Failure count: 1 till date] -------------- Config: TLSv1, TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA, RSA keypair, ECDHE KeyExchange support with Mutual Authentication enabled Server Config: RSA keypair with Mutual Authentication enabled. Client Config: ECDSA keypair with Mutual Authentication enabled. [Failure count: 2 till date] Note: There are also cases when client is using ECDSA keypair with Mutual Authentication enabled and the Test cases passed. Here are few statements from different logs, Passed TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA in TLSv1.2 mode with ECDSA client authentication Passed SSL_RSA_WITH_NULL_MD5 in TLSv1.1 mode with ECDSA client authentication Passed TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA in TLSv1 mode with ECDSA client authentication Case#1) Invalid ECDH ServerKeyExchange signature When Server side KeyExchange is ECDHE, the Server send it's DH(named group compatible with EC curve) public key through ServerKeyExchange Handshake record to client. This record is signed with Servers Private key which is from a server configured ECDSA key pair as per cipher selected from the Testcase#1. The client is also using a ECDSA keypair for mutual authentication. The possible cause could be a Test issue which use a mismatched keys in client side during signature verification of ServerKeyExchange handshake record when received. Case#2) Invalid CertificateVerify message: invalid signature When CertificateVerify Handshake record sent from client to server during mutual authentication, this record is signed with clients ECDSA configured private key as per Testcase#2. The possible cause could be a Test issue which use a mismatched keys in server side during signature verification of CertificateVerify handshake record when received. Each time when the Test run in different machine, it failed for different test cases while other time the same Test case passed. So it looks like there is some kind of issue related to synchronization or keys or cache.
05-06-2024
Also failing with: javax.net.ssl.SSLHandshakeException: (handshake_failure) Invalid CertificateVerify message: invalid signature at java.base/sun.security.ssl.Alert.createSSLException(Alert.java:130) at java.base/sun.security.ssl.Alert.createSSLException(Alert.java:117) at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:365) at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:321) at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:312) at java.base/sun.security.ssl.CertificateVerify$T10CertificateVerifyMessage.<init>(CertificateVerify.java:398) at java.base/sun.security.ssl.CertificateVerify$T10CertificateVerifyConsumer.consume(CertificateVerify.java:556) at java.base/sun.security.ssl.SSLHandshake.consume(SSLHandshake.java:393) at java.base/sun.security.ssl.HandshakeContext.dispatch(HandshakeContext.java:476) at java.base/sun.security.ssl.HandshakeContext.dispatch(HandshakeContext.java:447) at java.base/sun.security.ssl.TransportContext.dispatch(TransportContext.java:201) at java.base/sun.security.ssl.SSLTransport.decode(SSLTransport.java:172) at java.base/sun.security.ssl.SSLSocketImpl.decode(SSLSocketImpl.java:1507) at java.base/sun.security.ssl.SSLSocketImpl.readHandshakeRecord(SSLSocketImpl.java:1422) at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:455) at java.base/sun.security.ssl.SSLSocketImpl.ensureNegotiated(SSLSocketImpl.java:924) at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(SSLSocketImpl.java:1015) at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(SSLSocketImpl.java:975) at CipherTest$Server.handleRequest(CipherTest.java:80) at JSSEServer$1.run(JSSEServer.java:87) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642) at java.base/java.lang.Thread.run(Thread.java:1575) javax.net.ssl.SSLHandshakeException: (handshake_failure) Received fatal alert: handshake_failure at java.base/sun.security.ssl.Alert.createSSLException(Alert.java:130) at java.base/sun.security.ssl.Alert.createSSLException(Alert.java:117) at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:365) at java.base/sun.security.ssl.Alert$AlertConsumer.consume(Alert.java:287) at java.base/sun.security.ssl.TransportContext.dispatch(TransportContext.java:204) at java.base/sun.security.ssl.SSLTransport.decode(SSLTransport.java:172) at java.base/sun.security.ssl.SSLSocketImpl.decode(SSLSocketImpl.java:1507) at java.base/sun.security.ssl.SSLSocketImpl.readHandshakeRecord(SSLSocketImpl.java:1422) at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:455) at java.base/sun.security.ssl.SSLSocketImpl.ensureNegotiated(SSLSocketImpl.java:924) at java.base/sun.security.ssl.SSLSocketImpl$AppOutputStream.write(SSLSocketImpl.java:1293) at java.base/java.io.OutputStream.write(OutputStream.java:124) at CipherTest$Client.sendRequest(CipherTest.java:269) at JSSEClient.runTest(JSSEClient.java:74) at CipherTest$Client.run(CipherTest.java:252) at java.base/java.lang.Thread.run(Thread.java:1575) java.lang.Exception: *** Test 'Client JSSE - Server JSSE' failed *** at CipherTest.run(CipherTest.java:209) at CipherTest.main(CipherTest.java:337) at ClientJSSEServerJSSE.main(ClientJSSEServerJSSE.java:67) at PKCS11Test.premain(PKCS11Test.java:820) at PKCS11Test.testNSS(PKCS11Test.java:485) at PKCS11Test.main(PKCS11Test.java:192) at ClientJSSEServerJSSE.main(ClientJSSEServerJSSE.java:57) at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103) at java.base/java.lang.reflect.Method.invoke(Method.java:580) at com.sun.javatest.regtest.agent.MainWrapper$MainTask.run(MainWrapper.java:138) at java.base/java.lang.Thread.run(Thread.java:1575)
31-05-2024