JDK-2170532 : SSL stress test with GF leads to 32 bit max process size in less than 5 minutes,with PCKS11 provider
  • Type: Backport
  • Backport of: JDK-6750401
  • Component: security-libs
  • Sub-Component: javax.crypto:pkcs11
  • Priority: P1
  • Status: Closed
  • Resolution: Duplicate
  • Submitted: 2008-12-09
  • Updated: 2010-05-11
  • Resolved: 2009-10-03
The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.
JDK 6 JDK 7
6u18Resolved 7Fixed
Comments
EVALUATION Perhaps, as a workaround, CKM_TLS_PRF mechanism should be disabled by default in the configuration file of SunPKCS11-Solaris provider.
03-10-2009
EVALUATION Filed 6887619 to keep track of this Solaris issue.
02-10-2009
EVALUATION During the troubleshooting process, it is found that the memory growth problem went away if JSSE 5U19 jar is used. As it turns out that one major JSSE enhancement that went into JDK 6 is 6316539: Support for CKM_SSL3_* and CKM_TLS_* mechanisms which switches JSSE from using its own crypto code to delegating to JCE providers for these functionalities. Among the newly added SSL3 and TLS mechanisms, it seems that CKM_TLS_PRF is the culprit for the memory growth problem. Based on the test results at hand, the memory usage remains flat when either: 1) the CKM_TLS_PRF mechanism is disabled, or 2) pass 0 as the value for the handle of the base key in the C_DeriveKey(...) call when CKM_TLS_PRF mechanism is enabled and used. Since everything else on java side is the same, it looks to be a Solaris crypto implementation issue.
02-10-2009
EVALUATION Changes will be made in both JSSE and SunPKCS11 provider to mitigate the problem of finalizer not able to catch up with the number of objects in its queue.
09-12-2008