Bug ID: JDK-8295343 sun/security/pkcs11 tests fail on Linux RHEL 8.6 and newer

JDK-8295343 : sun/security/pkcs11 tests fail on Linux RHEL 8.6 and newer

Type: Bug
Component: security-libs
Sub-Component: javax.crypto:pkcs11
Affected Version: 7,8,11,17,20,21

Priority: P3
Status: Resolved
Resolution: Fixed
OS: linux
CPU: generic

Submitted: 2022-10-14
Updated: 2025-05-20
Resolved: 2023-11-22

Versions (Unresolved/Resolved/Fixed)

The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.

JDK 11	JDK 17	JDK 21	JDK 22	JDK 8
11.0.24Fixed	17.0.12-oracleFixed	21.0.3-oracleFixed	22 b26Fixed	8-poolUnresolved

Related Reports

Relates :	JDK-8296675 - Exclude linux-aarch64 in NSS tests
Relates :	JDK-8320690 - sun/security/pkcs11/Provider/MultipleLogins.sh fails after 8295343
Relates :	JDK-8296611 - Problemlist several sun/security tests until JDK-8295343 is resolved
Relates :	JDK-8295405 - Add cause in a couple of IllegalArgumentException and InvalidParameterException shown by sun/security/pkcs11 tests
Relates :	JDK-8324841 - PKCS11 tests still skip execution

Sub Tasks

JDK-8301611 :	Problem list sun/security/pkcs11/Signature/ tests until JDK-8295343 is resolved - Resolved
JDK-8317977 :	update problemlist to include MacOS for sun/security/tools/keytool/NssTest.java - Resolved

Description

On our RHEL 8.6  Linux x86_64 test machine, a couple of sun/security/pkcs11 tests fail.
Below are a number of examples. It looks like the native libs of the system do not support any more RSA keys with 512 bit length.
Should the tests be adjusted so that some keys are not tested any more in case the system is too new?

Examples of errors :

sun/security/pkcs11/KeyStore/Basic.java
----------------------------------------------
 ....
test 31 passed
test 32 passed
test 33 passed
got [pk1] signing key: SunPKCS11-nss RSA private key, 512 bits token object, not sensitive, extractable)
test Basic.testBasic(): failure
java.security.InvalidKeyException: RSA key must be at least 1023 bits
	at jdk.crypto.cryptoki/sun.security.pkcs11.P11Signature.checkKeySize(P11Signature.java:430)
	at jdk.crypto.cryptoki/sun.security.pkcs11.P11Signature.engineInitSign(P11Signature.java:496)
	at java.base/java.security.Signature$Delegate.engineInitSign(Signature.java:1357)
	at java.base/java.security.Signature.initSign(Signature.java:636)
	at Basic.signAlias(Basic.java:313)
	at Basic.sign(Basic.java:287)
	at Basic.doTest(Basic.java:234)
	at Basic.main(Basic.java:190)
	at PKCS11Test.premain(PKCS11Test.java:174)
	at PKCS11Test.testNSS(PKCS11Test.java:550)
	at PKCS11Test.main(PKCS11Test.java:210)
	at Basic.testBasic(Basic.java:116)
	at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:104)
	at java.base/java.lang.reflect.Method.invoke(Method.java:578)
	at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:132)
	at org.testng.internal.TestInvoker.invokeMethod(TestInvoker.java:599)
	at org.testng.internal.TestInvoker.invokeTestMethod(TestInvoker.java:174)
	at org.testng.internal.MethodRunner.runInSequence(MethodRunner.java:46)
	at org.testng.internal.TestInvoker$MethodInvocationAgent.invoke(TestInvoker.java:822)
	at org.testng.internal.TestInvoker.invokeTestMethods(TestInvoker.java:147)
	at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:146)
	at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:128)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
	at org.testng.TestRunner.privateRun(TestRunner.java:764)
	at org.testng.TestRunner.run(TestRunner.java:585)
	at org.testng.SuiteRunner.runTest(SuiteRunner.java:384)
	at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:378)
	at org.testng.SuiteRunner.privateRun(SuiteRunner.java:337)
	at org.testng.SuiteRunner.run(SuiteRunner.java:286)
	at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:53)
	at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:96)
	at org.testng.TestNG.runSuitesSequentially(TestNG.java:1218)
	at org.testng.TestNG.runSuitesLocally(TestNG.java:1140)
	at org.testng.TestNG.runSuites(TestNG.java:1069)
	at org.testng.TestNG.run(TestNG.java:1037)
	at com.sun.javatest.regtest.agent.TestNGRunner.main(TestNGRunner.java:93)
	at com.sun.javatest.regtest.agent.TestNGRunner.main(TestNGRunner.java:53)
	at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:104)
	at java.base/java.lang.reflect.Method.invoke(Method.java:578)
	at com.sun.javatest.regtest.agent.MainWrapper$MainThread.run(MainWrapper.java:125)
	at java.base/java.lang.Thread.run(Thread.java:1591)

sun/security/pkcs11/rsa/TestKeyFactory.java
------------------------------------------------

java.security.InvalidKeyException: Could not create RSA public key
	at jdk.crypto.cryptoki/sun.security.pkcs11.P11RSAKeyFactory.implTranslatePublicKey(P11RSAKeyFactory.java:70)
	at jdk.crypto.cryptoki/sun.security.pkcs11.P11KeyFactory.engineTranslateKey(P11KeyFactory.java:153)
	at java.base/java.security.KeyFactory.translateKey(KeyFactory.java:469)
	at TestKeyFactory.testPublic(TestKeyFactory.java:164)
	at TestKeyFactory.test(TestKeyFactory.java:221)
	at TestKeyFactory.main(TestKeyFactory.java:241)
	at PKCS11Test.premain(PKCS11Test.java:174)
	at PKCS11Test.testNSS(PKCS11Test.java:550)
	at PKCS11Test.main(PKCS11Test.java:210)
	at TestKeyFactory.main(TestKeyFactory.java:228)
	at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:104)
	at java.base/java.lang.reflect.Method.invoke(Method.java:578)
	at com.sun.javatest.regtest.agent.MainWrapper$MainThread.run(MainWrapper.java:125)
	at java.base/java.lang.Thread.run(Thread.java:1591)
Caused by: sun.security.pkcs11.wrapper.PKCS11Exception: CKR_ATTRIBUTE_VALUE_INVALID
	at jdk.crypto.cryptoki/sun.security.pkcs11.wrapper.PKCS11.C_CreateObject(Native Method)
	at jdk.crypto.cryptoki/sun.security.pkcs11.P11RSAKeyFactory.generatePublic(P11RSAKeyFactory.java:193)
	at jdk.crypto.cryptoki/sun.security.pkcs11.P11RSAKeyFactory.implTranslatePublicKey(P11RSAKeyFactory.java:59)
	... 13 more


sun/security/pkcs11/rsa/TestKeyPairGenerator.java
-------------------------------------------------------
sunpkcs11: Initializing PKCS#11 library /usr/lib64/libsoftokn3.so
java.security.InvalidParameterException: RSA key must be at least 1023 bits. The specific key size 512 is not supported
	at jdk.crypto.cryptoki/sun.security.pkcs11.P11KeyPairGenerator.initialize(P11KeyPairGenerator.java:154)
	at java.base/java.security.KeyPairGenerator$Delegate.initialize(KeyPairGenerator.java:665)
	at java.base/java.security.KeyPairGenerator.initialize(KeyPairGenerator.java:377)
	at TestKeyPairGenerator.main(TestKeyPairGenerator.java:127)
	at PKCS11Test.premain(PKCS11Test.java:174)
	at PKCS11Test.testNSS(PKCS11Test.java:550)
	at PKCS11Test.main(PKCS11Test.java:210)
	at TestKeyPairGenerator.main(TestKeyPairGenerator.java:108)
	at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:104)
	at java.base/java.lang.reflect.Method.invoke(Method.java:578)
	at com.sun.javatest.regtest.agent.MainWrapper$MainThread.run(MainWrapper.java:125)
	at java.base/java.lang.Thread.run(Thread.java:1591)

Comments

NSS libraries were moved to Oracle Artifact Repo. The libraries used for this test for JDK8U is much different from master bug. It needs much efforts to build the library for each of systems. This backport on jdk8u is on hold.
11-03-2025
Fix request [11u] I backport this for parity with 11.0.24-oracle. No risk, only tests change. Also backport for JDK-8231351 for the testing.html, some small change. Test pass. SAP nightly testing passed.
06-05-2024
A pull request was submitted for review. URL: https://git.openjdk.org/jdk11u-dev/pull/2694 Date: 2024-04-30 09:23:08 +0000
30-04-2024
Fix request [17u] I backport this for parity with 17.0.11-oracle. Small change. Test pass. SAP nightly testing passed.
19-04-2024
A pull request was submitted for review. URL: https://git.openjdk.org/jdk17u-dev/pull/2412 Date: 2024-04-16 01:57:32 +0000
16-04-2024
Fix request [21u] I backport this for parity with 21.0.3-oracle. No risk, only a test change. Clean backport. Will push this after follow-up JDK-8320690 aka JDK-8319128 to avoid test failures. Tests pass. SAP nightly testing passed.
24-01-2024
A pull request was submitted for review. URL: https://git.openjdk.org/jdk21u-dev/pull/186 Date: 2024-01-17 14:17:48 +0000
19-01-2024
After the change, sun/security/pkcs11/Provider/MultipleLogins.sh starts to fail in our test infra (e.g. on Linux. Mac). we get in stderr : Exception in thread "main" jtreg.SkippedException: Warning: unsupported OS: Linux-amd64-64, please initialize NSS library location, skipping test at PKCS11Test.getNSSLibPath(PKCS11Test.java:262) at PKCS11Test.getNSSLibDir(PKCS11Test.java:243) at PKCS11Test.getNSSLibDir(PKCS11Test.java:239) at PKCS11Test.getNssConfig(PKCS11Test.java:484) at MultipleLogins.main(MultipleLogins.java:49) Probably Provider/MultipleLogins.sh still cannot deal correctly with the cases where now a SkippedException is thrown . I created JDK-8320690 .
24-11-2023
Changeset: 6ce0ebb8 Author: Matthew Donovan <mdonovan@openjdk.org> Date: 2023-11-22 13:00:56 +0000 URL: https://git.openjdk.org/jdk/commit/6ce0ebb858d3112f136e12d3ad595f805f6871a0
22-11-2023
A pull request was submitted for review. URL: https://git.openjdk.org/jdk/pull/16294 Date: 2023-10-20 19:18:47 +0000
20-10-2023
For sun/security/pkcs11/KeyStore/Basic.java, it looks like we need to regenerate all the keys and certs using the generally supported RSA key size. 512 is too dated. It'd be nice to update the test and we can resume running this test.
10-10-2023
Hi [~akhuntia], I believe so.
05-10-2023
Hi, [~mbaesken], JDK-8296631 was caused by a change in database formats. It looks like KeyStore/Basic.java fails because of incorrect key sizes. I didn't work on 8296631 so I don't know if they're related.
05-10-2023
Hi [~mdonovan], thanks for the info about JDK-8296631. However I notice that e.g. sun/security/pkcs11/KeyStore/Basic.java is still excluded : https://github.com/openjdk/jdk/blob/master/test/jdk/ProblemList.txt#L624 Is this still valid ?
05-10-2023
[~akhuntia] those errors were fixed in https://bugs.openjdk.org/browse/JDK-8296631 and merged just a day or two ago.
04-10-2023
@Matthias: 1) The problem with the minimum size is not in the test itself but in the key that the test is using (obtained from a PKCS #11 key store but originally pushed to the key store from a serialized byte[] in a file). My impression is that we need to generate keys and certificates again. It would be helpful and self-documented if, as part of fixing this issue, we add an script to generate all the keys, certificates and key stores that this test depends on. 2) You're right, if the patch is downstream only, checking the NSS version would not be helpful. I can confirm that RHEL 9 has exactly the same patch than RHEL 8. We can explore the method to skip the test but I wish we could also generate the keys again and run the test. It's possible that this NSS patch is pushed upstream or applied in other OS in the future so we would have less coverage if we keep these old keys.
09-02-2023
+1 for changing the hard-coded values as a quick fix. Perhaps we can add a pkg private method to the SunPKCS11 provider which returns the min and max value of a particular PKCS11 mechanism. This may then be used by the tests to skip certain values if not within the supported range.
21-11-2022
Following tests also failed - sun/security/tools/keytool/NssTest.java sun/security/pkcs11/Signature/TestRSAKeyLength.java sun/security/pkcs11/rsa/TestSignatures.java
10-11-2022
I do not see really much about min key sizes in the test itself https://github.com/openjdk/jdk/blob/master/test/jdk/sun/security/pkcs11/KeyStore/Basic.java . Is this stored somehow in the data files found at test/jdk/sun/security/pkcs11/KeyStore/BasicData ? PKCS11Test.java has already some coding for finding out specific nss lib versions (getNssInfo) but probably such a check would not help much in this test on RHEL 8 /9 if this is a specific NSS RHEL-downstream RPM patch for NSS 3.79 (so we would not find it in other NSS 3.79 libs on other OS, is my understanding correct) ?
08-11-2022
The min RSA key value supported comes from the NSS Software Token. In particular, OpenJDK obtains the CK_MECHANISM_INFO info for a given mechanism (such as CKM_RSA_PKCS_KEY_PAIR_GEN in the case of TestKeyPairGenerator) with the C_GetMechanismInfo PKCS#11 API. The relevant field in the CK_MECHANISM_INFO is ulMinKeySize and, for RSA mechanisms, specifies the min modulus in bits. In NSS upstream, min modulus for RSA keys have been 128 bits for the last several years [1] [2]. The min RSA key enforcement from 128 to 1023 bits is not RHEL crypto-policies related but comes from an NSS RHEL-downstream RPM patch for NSS 3.79, that is shipped with the latest RHEL 8.6.0 release [3]. Thus, changing the crypto-policies profile to LEGACY will not make any difference. Using an older RHEL release -one that uses NSS < 3.79- will make the test pass. In my view, we can either increase the test's min key size to a fixed value greater than 1023 or make it more context-aware and adjust to the min value provided from the underlying NSS Software Token. The latter can be a bit involved because we probably don't have public API to retrieve the min key size for a given mechanism, and would need to either break encapsulation or parse the error message. I'd go with the former idea of updating the test's hard-coded values for simplicity. -- [1] - https://github.com/nss-dev/nss/blob/NSS_3_67_RTM/lib/freebl/blapit.h#L138 [2] - https://github.com/nss-dev/nss/blob/NSS_3_67_RTM/lib/softoken/pkcs11.c#L316 [3] - https://git.centos.org/rpms/nss/blob/dc0051d01350d07bd453815ac689db13c9f22e59/f/SOURCES/nss-3.79-rhel-8-fips-signature-policy.patch#_369
07-11-2022
I got some feedback from RedHat about the topic. Key size limits are obtained from underlaying pkcs11 implementation . The crypto policies, on RHEL-8 and 9, require RSA keys >=1024 (on RHEL-8) and >=2048 (on RHEL-9) : https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/8/html/security_hardening/using-the-system-wide-cryptographic-policies_security-hardening https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/9/html/security_hardening/using-the-system-wide-cryptographic-policies_security-hardening RedHat confirmed that jdk17 portable builds currently fail these tests (ones mentioned in JDK-8295343) on RHEL-8 even with legacy CP, however same build passes these tests on RHEL-7 (RHEL-7 does not yet have system-wide crypto policies). So we should probably increase the RSA key sizes used in these tests.
07-11-2022
Looks like the RHEL8 system defaults https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/8/html/security_hardening/using-the-system-wide-cryptographic-policies_security-hardening do not allow RSA keys with less than 1024 bits (not sure about those exceptions "RSA key must be at least 1023 bits" , shouldn't this be 1024 the current value looks a bit odd).
19-10-2022