Summary
-------
Update the ML-KEM and ML-DSA private key encodings to align with [Section 6 of RFC XXXX](https://datatracker.ietf.org/doc/html/rfcXXXX#autoid-7) and [Section 6 of RFC 9881](https://datatracker.ietf.org/doc/html/rfc9881#name-private-key-format). Specifically, the `privateKey` field inside the PKCS #8 encoding will follow the DER-encoded ASN.1 CHOICE structure as defined in these RFCs. All 3 CHOICE options will be supported at both encoding and decoding.
Problem
-------
When ML-KEM and ML-DSA were introduced in JDK 24, these RFCs had not been published yet. The drafts at the time, draft-ietf-lamps-kyber-certificates and draft-ietf-lamps-dilithium-certificates, described the private key only as βan opaque byte sequence,β without specifying a concrete format.
Based on that, the JDK 24 implementation adopted the encodings defined in NIST FIPS 203 and 204 β specifically, using the private key output from the ML-KEM.KeyGen function (Section 7.1 of FIPS 203), and the ML-DSA.KeyGen function (Section 5.1 of FIPS 204).
Now that RFC XXX (ML-KEM) and RFC 9881 (ML-DSA) have been published, their private key formats are formally defined as DER-encoded ASN.1 CHOICE types. For example, for ML-KEM-512:
ML-KEM-512-PrivateKey ::= CHOICE {
seed [0] OCTET STRING (SIZE (64)),
expandedKey OCTET STRING (SIZE (1632)),
both SEQUENCE {
seed OCTET STRING (SIZE (64)),
expandedKey OCTET STRING (SIZE (1632))
}
}
Similar structures are defined for ML-KEM-768, ML-KEM-1024, ML-DSA-44, ML-DSA-65, and ML-DSA-87.
The JDK 24 implementation currently uses the second option, `expandedKey OCTET STRING`. To comply with the final RFCs, we need to update our implementation to support the other two CHOICEs as well.
Solution
--------
For reading, all three CHOICEs are supported when parsing an existing PKCS #8 encoded private key.
For writing, new security properties are introduced to control which CHOICE is used when creating a new private key. When `KeyPairGenerator::generateKey` or `KeyFactory::translateKey` is called, the resulting private key will be encoded according to the specified security property setting.
Note: If `KeyFactory::translateKey` is used to convert a key that does not contain a seed into a format that requires one, the operation will fail with an `InvalidKeyException`.
Specification
-------------
New security properties, `jdk.mlkem.pkcs8.encoding` and `jdk.mldsa.pkcs8.encoding`, are introduced to control the PKCS #8 encoding of newly created ML-KEM and ML-DSA private keys.
The following is added to the `java.security` file:
#
# PKCS #8 encoding format for newly created ML-KEM and ML-DSA private keys
#
# RFC XXXX and RFC 9881 define three possible formats for a private key:
# a seed (64 bytes for ML-KEM, 32 bytes for ML-DSA), an expanded private key,
# or a sequence containing both.
#
# The following security properties determine the encoding format used when a
# new keypair is generated with a KeyPairGenerator, and the output of the
# translateKey method on an existing key using a ML-KEM or ML-DSA KeyFactory.
#
# Valid values for these properties are "seed", "expandedKey", and "both"
# (case-insensitive). The default is "seed".
#
# If a system property of the same name is also specified, it supersedes the
# security property value defined here.
#
# Note: These properties are currently used by the SunJCE (for ML-KEM) and SUN
# (for ML-DSA) providers in the JDK Reference implementation. They are not
# guaranteed to be supported by other implementations or third-party security
# providers.
#
#jdk.mlkem.pkcs8.encoding = seed
#jdk.mldsa.pkcs8.encoding = seed