Bug ID: JDK-8326123 The MSCAPI provider does not fully support the CNG APIs

Type: Enhancement
Component: security-libs
Sub-Component: javax.crypto

Priority: P4
Status: Open
Resolution: Unresolved
OS: windows_10
CPU: x86_64

Submitted: 2024-02-16
Updated: 2024-04-05

ADDITIONAL SYSTEM INFORMATION :
All Windows implementations

A DESCRIPTION OF THE PROBLEM :
Windows moved to CNG from the cryptoAPI in vista (documented in bug https://bugs.openjdk.org/browse/JDK-8026953). Some functionality such as private keys have been moved into the cryptoAPI, but others (such as PRNG) still use the depreciated APIs (see CryptGenRandom vs BCryptGenRandom). 

Move all mscapi.cpp implementations from <wincrypt.h> to <bcrypt.h>

OK, so you are saying that RNGs of CAPI and CNG are using the same underlying mechanism but the CNG function is certified recently and it's closer to the underlying mechanism (or itself is). The only concern is still the setSeed() method. If BCRYPT_RNG_USE_ENTROPY_IN_BUFFER is ignored now, it means setSeed has no effect in CNG. But, does it still have effect in CAPI? If yes, how could CAPI and CNG's RNG be equivalent? Or, does CAPI maintain its own mixed-in RNG that's seeded separately?
05-04-2024
Further information from the submitter: The benefit to switching to the new function is 3 fold: 1. NIST hasn’t certified (or Microsoft hasn’t submitted it for certification) the RSAENH.dll that CryptGenRandom comes from since December 2014 (almost 10 years ago). 2. The function clearly states in the Windows 8 submission (https://csrc.nist.gov/csrc/media/projects/cryptographic-module-validation-program/documents/security-policies/140sp1894.pdf) that CryptGenRandom “merely forwards the call to a FIPS approved RNG from the cryptographic Primitives Library (bcryptprimitives.dll)”. Therefore, moving to BCryptGenRandom will save the code an unnecessary step of forwarding every call to the bcryptprimitives.dll instead of just calling the function directly. 3. Microsoft hasn’t deprecated the CryptoAPI: Next Generation and since the call is functionally equivalent (on windows 8 and later) moving it bears no issue. Since windows 7 went EOL in 2020, all versions of the CryptGenRandom function that would have used the seed value (windows 7 and previous) haven’t been using it for over 4 years.
05-03-2024
Thanks a lot for the additional information. However, the BCryptGenRandom page still claims BCRYPT_RNG_USE_ENTROPY_IN_BUFFER is ignored since Windows 8. Also, if BCryptGenRandom and CryptGenRandom are equivalent, what is the benefit to switch to the new function?
04-03-2024
Additional information from the submitter: Sorry for belaboring a point, but I want to add in one more comment before this comes into review to hopefully add some context to help ease the move to BCryptGenRandom. In Windows 7 Microsoft updated CryptGenRandom to behave almost identically to BCryptGenRandom. https://learn.microsoft.com/en-us/windows/win32/api/wincrypt/nf-wincrypt-cryptgenrandom (Under Remarks). https://learn.microsoft.com/en-us/windows/win32/api/bcrypt/nf-bcrypt-bcryptgenrandom (Under Remarks). Microsoft’s submission for the updated Enhanced Cryptographic Provider (which is the provider that the Java MSCAPI provider uses) contains more information. https://csrc.nist.gov/projects/cryptographic-module-validation-program/certificate/1330 The section on CryptGenRandom states that the generator is initialized by the in-kernel PRNG and that caller supplied data “is mixed with the seed”. The in-kernel PRNG is in CNG.sys which is has the #4515 certificate. https://csrc.nist.gov/projects/cryptographic-module-validation-program/certificate/4515 This in-kernel PRNG is SystemPrng and both CryptGenRandom and BCryptGenRandom use it internally. (page 24-25, and the diagram in page 8). Because of the change between CAPI released with Windows XP and Vista and CAPI released with Windows 7 and later and its dependency on the in-kernel PRNG, there really isn’t a difference between calling CryptGenRandom the way that it’s been called, and moving to BCryptGenRandom. The result is the same that the user’s input data is put into the pool of entropy and then used to generate the seed for reseeding the PRNG from which the random number is generated.
04-03-2024
Additional information from the submitter: If the preference is to leave the current CAPI provider as it is, maybe there could be a new sun.security.mscng.SunMSCNG provider with the appropriate classes. The windows folder would then have a libsunmscng folder with a security.cpp file that implements the same functionality as the current security.cpp but with the cng functions (in the bcrypt.h header). The new Java release would have this new provider available for use (and hopefully backported) and would just have a comment in the javadoc saying that the engineSetSeed on the spi or the setSeed function on the SecureRandom that will say that the seed will be used as extra entropy (or will be discarded based on choice of implementation).
01-03-2024
Additional information from the submitter: This function doesn’t support seeding in a typical sense the way CAPI supports it. However, if you call it with dwFlags is set to BCRYPT_RNG_USE_ENTROPY_IN_BUFFER then you should be able to put the seed value set in the input buffer and use that as additional entropy. However, NIST has approved BCryptGenRandom for its functionality and it’s automatically reseeded by the platform and doesn’t need to be seeded externally. Like IBM (whose z platform also reseeds its approved RNG), it might be prudent to deprecate the function for the ms-capi provider when switching to BCryptGenRandom because there’s no reason to seed it.
27-02-2024
Requested above information from the submitter.
26-02-2024
We've looked into BCryptGenRandom in JDK-8213621 but it does not support the setSeed() method. Does the bug reporter have other particular areas where CNG is necessary?
22-02-2024
Moved to JDK for further investigations.
19-02-2024