JDK-8294906 : Memory leak in PKCS11 NSS TLS server
  • Type: Bug
  • Component: security-libs
  • Sub-Component: javax.crypto:pkcs11
  • Affected Version: 8,11,17,19,20
  • Priority: P3
  • Status: Resolved
  • Resolution: Fixed
  • Submitted: 2022-10-06
  • Updated: 2022-10-17
  • Resolved: 2022-10-13
The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.
JDK 20
20 masterFixed
Related Reports
Relates :  
JDK-7030966 - Support AEAD CipherSuites
Description
P11TlsKeyMaterialGenerator leaks 2 handles (hClientMacSecret and hServerMacSecret) every time a connection using AES-GCM is established. PKCS11 library used was NSS.

MAC keys are not needed when using AEAD, and the PKCS spec [1] suggests that they don't need to be created. However, it doesn't prohibit creating MAC keys, and come libraries like NSS create them. 

https://docs.oasis-open.org/pkcs11/pkcs11-spec/v3.1/cs01/pkcs11-spec-v3.1-cs01.html#_Toc111203671
Comments
Changeset: 94caecbe Author: Daniel Jeliński <djelinski@openjdk.org> Date: 2022-10-13 09:29:17 +0000 URL: https://git.openjdk.org/jdk/commit/94caecbe574227b232e22d9f56361f8ecd507be6
13-10-2022
The issue is present in NSS 3.82, which is 2 months old at the moment; verified with Ubuntu binaries.
12-10-2022
added noreg-hard because the new tests don't really check for presence/absence of the leak; they only confirm that the new code can still generate keys for AES+GCM
11-10-2022
A pull request was submitted for review. URL: https://git.openjdk.org/jdk/pull/10594 Date: 2022-10-06 13:27:23 +0000
06-10-2022