JDK-8242928 : Weak DHE_DSS cipher suites cannot work with pre-TLSv1.2 protocols
  • Type: Bug
  • Component: security-libs
  • Sub-Component: javax.net.ssl
  • Affected Version: 15
  • Priority: P3
  • Status: Closed
  • Resolution: Won't Fix
  • Submitted: 2020-04-16
  • Updated: 2020-04-24
  • Resolved: 2020-04-23
Related Reports
Relates :  
JDK-8243549 - sun/security/ssl/CipherSuite/NamedGroupsWithCipherSuite.java failed with Unsupported signature algorithm: DSA
Relates :  
JDK-8243558 - JDK Provider Guide should document that DSA signature generation is now subject to a key strength check
Description
Protocol: TLSv1.1
Cipher suite: TLS_DHE_DSS_WITH_AES_256_CBC_SHA
Certificate: SHA256withDSA with key size 2048

The above simple TLS handshaking case failed with the below error,
Caused by: javax.net.ssl.SSLException: Unsupported signature algorithm: DSA
	at java.base/sun.security.ssl.Alert.createSSLException(Alert.java:133)
	at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:327)
	at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:270)
	at java.base/sun.security.ssl.DHServerKeyExchange$DHServerKeyExchangeMessage.<init>(DHServerKeyExchange.java:154)
	at java.base/sun.security.ssl.DHServerKeyExchange$DHServerKeyExchangeProducer.produce(DHServerKeyExchange.java:487)
	at java.base/sun.security.ssl.ClientHello$T12ClientHelloConsumer.consume(ClientHello.java:1101)
	at java.base/sun.security.ssl.ClientHello$ClientHelloConsumer.onClientHello(ClientHello.java:851)
	at java.base/sun.security.ssl.ClientHello$ClientHelloConsumer.consume(ClientHello.java:810)
	at java.base/sun.security.ssl.SSLHandshake.consume(SSLHandshake.java:392)
	at java.base/sun.security.ssl.HandshakeContext.dispatch(HandshakeContext.java:451)
	at java.base/sun.security.ssl.HandshakeContext.dispatch(HandshakeContext.java:428)
	at java.base/sun.security.ssl.TransportContext.dispatch(TransportContext.java:184)
	at java.base/sun.security.ssl.SSLTransport.decode(SSLTransport.java:164)
	at java.base/sun.security.ssl.SSLSocketImpl.decode(SSLSocketImpl.java:1151)
	at java.base/sun.security.ssl.SSLSocketImpl.readHandshakeRecord(SSLSocketImpl.java:1062)
	at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:402)
	at java.base/sun.security.ssl.SSLSocketImpl.ensureNegotiated(SSLSocketImpl.java:716)
	at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(SSLSocketImpl.java:799)
	at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(SSLSocketImpl.java:758)
	at SSLSocketTemplate.runServerApplication(SSLSocketTemplate.java:99)
	at SSLSocketTemplate.doServerSide(SSLSocketTemplate.java:288)
	at SSLSocketTemplate.startServer(SSLSocketTemplate.java:592)
	at SSLSocketTemplate.bootup(SSLSocketTemplate.java:506)
	... 2 more
Caused by: java.security.InvalidKeyException: The security strength of SHA-1 digest algorithm is not sufficient for this key size
	at java.base/sun.security.provider.DSA.checkKey(DSA.java:124)
	at java.base/sun.security.provider.DSA.engineInitSign(DSA.java:156)
	at java.base/java.security.Signature$Delegate.tryOperation(Signature.java:1306)
	at java.base/java.security.Signature$Delegate.chooseProvider(Signature.java:1255)
	at java.base/java.security.Signature$Delegate.engineInitSign(Signature.java:1352)
	at java.base/java.security.Signature.initSign(Signature.java:634)
	at java.base/sun.security.ssl.DHServerKeyExchange$DHServerKeyExchangeMessage.getSignature(DHServerKeyExchange.java:441)
	at java.base/sun.security.ssl.DHServerKeyExchange$DHServerKeyExchangeMessage.<init>(DHServerKeyExchange.java:150)

This case also failed with SSLv3 and TLSv1, but passed with TLSv1.2.
Comments
The following two causes result in the issue: 1. For TLS 1.1 and prior versions, SHA-1 is used for DSA certificate verification. See section 7.4.3 of RFC 4346. 2. In the Sun provider, the DSA signature is not allowed if the hash function providers a lower security strength than the DSA key ((L, N) pair). See section 4.2 of FIPS 186-3, there is a recommendation ("should not" is sued, so not requirement): "A hash function that provides a lower security strength than the (L, N) pair ordinarily should not be used, since this would reduce the security strength of the digital signature process to a level no greater than that provided by the hash function." So for TLS 1.1, when 2048 bits DSA cert get used, SHA-1 will be used as the hash algorithm for signature. However, SHA-1 does not meet #2. And the signature cannot be generated. Looking at the strength per NIST SP800-57, the hash algorithms strength looks like: hash algorithm | strength -----------------+------------- SHA-1 | 80 bit -----------------+------------- SHA-224 | 112 bit -----------------+------------- SHA-256 | 128 bit -----------------+------------- SHA-384 | 192 bit -----------------+------------- SHA-512 | 256 bit -----------------+------------- And DSA: DSA (L, N) param | strength -----------------+------------- (1024, 160) | 80 bit -----------------+------------- (2048, 224) | 112 bit -----------------+------------- (2048, 256) | 128 bit -----------------+------------- (3072, 256) | 128 bit -----------------+------------- Per the current DSA implementation in the Sun provider. SHA-1 cannot work with DSA keys where N > 160, and SHA-224 cannot work with DSA keys where N > 224. As may result in a few problems in practice, for example 2048 bits DSA certificate cannot be used in TLS 1.1/1.0. I did not see a significant impact yet. In the section 4.2 of FIPS 186-3, there is a requirement ("shall" used, so it is requirement): "The security strength associated with the DSA digital signature process is no greater than the minimum of the security strength of the (L, N) pair and the security strength of the hash function that is employed. Both the security strength of the hash function used and the security strength of the (L, N) pair shall meet or exceed the security strength required for the digital signature process." Per this section, I think SHA1 with 2048-bit DSA key could be used for signature if the security strength requirement is no more than the strength of SHA-1. However, there is no public APIs to specify the required security strength in the current JCA framework. We may not want to loose the restriction in the Sun provider, as it is the recommended behavior. I will just close this bug as "will not fix" for now, as it should be rare to use DSA 2048 for TLS 1.1/1.0. If there is a real impact in practice, please re-open the bug. At that time, we may have a better feel about how to fix the issue.
23-04-2020
The attached SSLSocketTemplate.java can reproduce this issue.
16-04-2020