JDK-8225739 : sun/security/pkcs11/tls/tls12/FipsModeTLS12.java is not reliable
  • Type: Bug
  • Component: security-libs
  • Sub-Component: java.security
  • Priority: P3
  • Status: Open
  • Resolution: Unresolved
  • Submitted: 2019-06-13
  • Updated: 2024-04-24
Related Reports
Relates :  
JDK-8223482 - Unsupported ciphersuites may be offered by a TLS client
Relates :  
JDK-8225678 - sun/security/pkcs11/tls/tls12/FipsModeTLS12.java fails due to SSLProtocolException
Description
This test returns false PASS results more than it actually runs

On Windows, Mac, and solaris-sparcv9, the test fails to initialize NSS and returns a pass.  On Linux it appears to be NSS dependent as it will skip TLS 1.2 testing.

Since test machine choice is random and the limited support, this not a reliable regression test.
Comments
[~mbalao] Have you looked any further into this? I am not sure there is a good resolution to this issue, so I would suggest closing this out as "Won't Fix" if the test is working similarly as other pkcs11 tests, or perhaps file a more general pkcs11 test improvement issue if there are some steps that you think would improve the reliability of the pkcs11 tests.
24-04-2024
@Anthony: thanks for clarification. In regards to working across different environments, this test requires NSS already installed and is not different than other SunPKCS11 tests: SecmodTest::initSecmod is called to locate and initialize NSS. At some point down the call stack, PKCS11Test::getNssLibPaths(String osId) gets called. There are some per-OS and per-arch default mappings there. I'm not sure of the environments you've tried, and won't be able to help much unless I'm able to reproduce there; but please note that the test cannot do much in an environment where NSS is not installed. Just for the record, what I do in my local Windows environment to run with (and debug) my local build of NSS is setting "pkcs11test.nss.libdir and "test.nss.lib.paths" properties with the correct paths.
18-07-2019
My point was not that it should fail in the wrong environment. My issue is that the test is in the wrong environment most of the time and that the test needs to work in a larger set of environments. Either loading a usable NSS or something. This test, when working properly would have picked up a bug in my recent putback, but after many many runs it was never in the proper environment to detect the issue.
17-06-2019
The question is whether or not the test must fail when run in an environment without NSS. Please notice that filtering by operating system is not relevant here. There can be arguments on both sides. My position was to fail, as described in this report. However, the rationale behind skipping is preventing false positive noise when run across different environments as part of a suite. Of course: if you always run it in environments without NSS, you won't exercise and test the FIPS code. But then, running SunPKCS11 tests without a proper PKCS#11 backend -like NSS- is non-sense. This has been previously discussed. See https://mail.openjdk.java.net/pipermail/security-dev/2018-August/017953.html and https://mail.openjdk.java.net/pipermail/security-dev/2018-August/018065.html.
13-06-2019