JDK-8238448 : RSASSA-PSS signature verification fail when using certain odd key sizes
  • Type: Bug
  • Component: security-libs
  • Sub-Component: java.security
  • Affected Version: 8u251,openjdk8u252,11.0.1
  • Priority: P3
  • Status: Resolved
  • Resolution: Fixed
  • OS: generic
  • CPU: generic
  • Submitted: 2020-02-04
  • Updated: 2020-11-30
  • Resolved: 2020-02-12
The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.
JDK 11 JDK 15 JDK 8 Other
11.0.9Fixed 15 b10Fixed 8u270Fixed openjdk8u282Fixed
Related Reports
Relates :  
For certain odd key sizes (N = 8x+1, i.e. multiples of 8 plus 1), the encoded message for PSS processing is one-byte shorter than the key size. For example, per RFC 8017, PKCS#1 v2.2, if the key size is 1025 bits (rounded up to 129 bytes), the length of encoded message should be 128 byte. However, the current impl of SunRsaSign provider uses the key length which has been rounded up in bytes as the length of the encoded message instead of calculating from the key length in bits. This causes the signature verification to fail unexpectedly as the verification is being performed on wrong range of bytes.
Fix Request (8u) I would like to backport this patch to OpenJDK8u for parity with Oracle 8u270. The original patch does not apply cleanly, but conflicts are minor and has been reviewed by phh.

8u code review: https://mail.openjdk.java.net/pipermail/jdk8u-dev/2020-October/012812.html

Fix request (11u) -- will label after testing completed. I would like to downport this for parity with 11.0.9-oracle. Applies clean.

URL: https://hg.openjdk.java.net/jdk/jdk/rev/0469ead7113c User: valeriep Date: 2020-02-12 02:45:50 +0000

src/java.base/share/classes/sun/security/rsa/RSAPSSSignature.java, insides its decodeSignature() method: Instead of setting emLen to em.length, it should be calculated from the emBits value, i.e. (emBits + 7) >> 3;

Note that the generated signature bytes are correct. Just the verification code needs to be fixed.