Bug ID: JDK-8212134 Failing tests in sun/security/pkcs11/Secmod with libnss 3.35

Type: Bug
Component: security-libs
Sub-Component: javax.crypto:pkcs11
Affected Version: 8u202,9.0.4

Priority: P3
Status: Closed
Resolution: Won't Fix

Submitted: 2018-10-12
Updated: 2019-02-11
Resolved: 2019-01-31

We started running tests on a new Linux distro (based on debian testing) and noticed that two tests started failing on jdk8u (but jdk head seems OK).  The most likely culprit is the change in libnss version.  We're excluding them

sun/security/pkcs11/Secmod/GetPrivateKey.java generic-all
sun/security/pkcs11/Secmod/JksSetPrivateKey.java generic-all

STDOUT:
SunPKCS11-NSSKeyStore version 1.8
entries: 0
[]
null
Signing...
STDERR:
java.security.InvalidKeyException: Key must not be null
	at sun.security.rsa.RSAKeyFactory.engineTranslateKey(RSAKeyFactory.java:182)
	at sun.security.rsa.RSAKeyFactory.toRSAKey(RSAKeyFactory.java:111)
	at sun.security.rsa.RSASignature.engineInitSign(RSASignature.java:106)
	at sun.security.rsa.RSASignature.engineInitSign(RSASignature.java:99)
	at java.security.Signature$Delegate.init(Signature.java:1155)
	at java.security.Signature$Delegate.chooseProvider(Signature.java:1115)
	at java.security.Signature$Delegate.engineInitSign(Signature.java:1179)
	at java.security.Signature.initSign(Signature.java:530)
	at GetPrivateKey.main(GetPrivateKey.java:65)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.sun.javatest.regtest.agent.MainWrapper$MainThread.run(MainWrapper.java:115)
	at java.lang.Thread.run(Thread.java:748)

JavaTest Message: Test threw exception: java.security.InvalidKeyException: Key must not be null
JavaTest Message: shutting down test

---


STDOUT:
SunPKCS11-NSSKeyStore version 1.8
entries: 0
[]
null
STDERR:
java.security.KeyStoreException: Cannot store non-PrivateKeys
java.lang.IllegalArgumentException: Private key must be accompanied by certificate chain
	at java.security.KeyStore.setKeyEntry(KeyStore.java:1136)
	at JksSetPrivateKey.main(JksSetPrivateKey.java:73)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.sun.javatest.regtest.agent.MainWrapper$MainThread.run(MainWrapper.java:115)
	at java.lang.Thread.run(Thread.java:748)

I suppose this issue is related to the NSS libs on the Linux. Currently, PKCS11 tests use local NSS libs on Linux. Different Linux systems may use different NSS versions.
31-01-2019
Both of the tests pass with NSS 3.35 libs on JDK 12+16 and 11.0.2+1.
26-10-2018
The tests also pass on JDK 9.0.4+11 -- sun/security/pkcs11/Secmod/GetPrivateKey.java SunPKCS11-NSSKeyStore version 9 entries: 1 [mykey] SunPKCS11-NSSKeyStore RSA private key, 1024 bits (id 3392055628, token object, sensitive, extractable) Signing... OK -- sun/security/pkcs11/Secmod/JksSetPrivateKey.java SunPKCS11-NSSKeyStore version 9 entries: 1 [mykey] SunPKCS11-NSSKeyStore RSA private key, 1024 bits (id 3392055628, token object, sensitive, extractable) OK OK
26-10-2018
Reassigning to John as it might be related to JDK-8164639.
12-10-2018
9.0.4 has the same problem, but 10.0.2 seems fine.
12-10-2018
Amusingly, the change in Linux system caused a test in JDK-8180837 to start pasxing while two new tests started failing! Most likely both are related to the libnss version change.
12-10-2018