JDK-7146728 : Inconsistent length for the generated secret using DH key agreement impl from SunJCE and PKCS11
  • Type: Bug
  • Component: security-libs
  • Sub-Component: javax.crypto
  • Affected Version: 1.0,7
  • Priority: P4
  • Status: Closed
  • Resolution: Fixed
  • OS: generic
  • CPU: generic
  • Submitted: 2012-02-17
  • Updated: 2016-04-29
  • Resolved: 2012-09-04
The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.
6u115Fixed 7u6Fixed 8 b34Fixed
Related Reports
Duplicate :  
Relates :  
For certain DH parameter values, a shorter secret is generated, i.e. the leading one or two bytes have value 0x00.

Thus, the regression test "com/sun/crypto/provider/KeyAgreement/DHKeyAgreement2.java" failed intermittently when such shorter secret is generated.

A slight modification to the above test program reveals the frequency of the failures as following:

Failure reproduced at 421
Failure reproduced at 241
Failure reproduced at 433
Failure reproduced at 85
Failure reproduced at 118
Failure reproduced at 184
Failure reproduced at 1418
Failure reproduced at 170

Anyway, a ShortBufferException is thrown when the failure is reproduced.

EVALUATION The current impl of SunJCE provider will strip off the first byte when the bit length is multiples of 8. However, the impl of SunPKCS11 provider strips off all leading 0s as it tries to match what SunJCE provider does. According to the comment in SunJCE provider source code, the removal of 1st byte is meant to remove the "sign" byte returned by BigInteger.toByteArray() calls. However, in the same impl class, the checking of output buffer didn't take into account of the "sign" byte, thus leading to ShortBufferException as demonstrated in the com/sun/crypto/provider/KeyAgreement/DHKeyAgreement2.java regression test which allocate an output buffer based on the result returned by the KeyAgreement.generateSecret() call. As for SunPKCS11 provider, it strips off all leading 0x00 bytes as an attempt to "follow JCE convention" when KeyAgreement.generateSecret() is called. However, in its impl of KeyAgreement.generateSecret(byte[] out, int outOfs), it again errors out if the output buffer 'out' starting offset 'outOfs' cannot hold byte arrays as long as the modulus. This inconsistency between KeyAgreement.generateSecret() and KeyAgreement.generateSecret(byte[], int) is very confusing and should be straightened out. The current javadoc of KeyAgreement didn't specify whether the leading 0s should be preserved or not, so it could go either way. Given that some internet standard such as RFC2631 explicitly mentions that the leading zeros should be preserved so the generated secret occupies as many octets as p. I think the correct fix is to change the SunJCE and SunPKCS11 provider to return a fixed size secret which is the same size as the modulus. This makes maintaining compatibility across applications easier since the generated secret length will then stay the same as the modulus size.