JDK-8320192 : SHAKE256 does not work correctly if n >= 137
  • Type: Bug
  • Component: security-libs
  • Sub-Component: java.security
  • Affected Version: 17,21,22
  • Priority: P3
  • Status: Closed
  • Resolution: Fixed
  • Submitted: 2023-11-15
  • Updated: 2024-09-17
  • Resolved: 2023-11-20
The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.
JDK 21 JDK 22
21.0.6Fixed 22 b25Fixed
Related Reports
Relates :  
Description
The internal SHAKE256 class fails the NIST CAVP test when n reaches 137. The following program succeeds in the 1st check but fails the 2nd one.

import sun.security.provider.SHAKE256;

import java.util.Arrays;
import java.util.HexFormat;

public class A2 {
    public static void main(String[] args) throws Exception {
        //https://csrc.nist.gov/CSRC/media/Projects/Cryptographic-Algorithm-Validation-Program/documents/sha3/shakebytetestvectors.zip
        //SHAKE256VariableOut.rsp:
        //3380: COUNT = 674
        //3381: Outputlen = 1088
        //3382: Msg = 6ae23f058f0f2264a18cd609acc26dd4dbc00f5c3ee9e13ecaea2bb5a2f0bb6b
        //3383: Output = b9b92544fb25cfe4ec6fe437d8da2bbe00f7bdaface3de97b8775a44d753c3adca3f7c6f183cc8647e229070439aa9539ae1f8f13470c9d3527fffdeef6c94f9f0520ff0c1ba8b16e16014e1af43ac6d94cb7929188cce9d7b02f81a2746f52ba16988e5f6d93298d778dfe05ea0ef256ae3728643ce3e29c794a0370e9ca6a8bf3e7a41e8677067
        //3384:
        //3385: COUNT = 675
        //3386: Outputlen = 1096
        //3387: Msg = 8d8001e2c096f1b88e7c9224a086efd4797fbf74a8033a2d422a2b6b8f6747e4
        //3388: Output = 2e975f6a8a14f0704d51b13667d8195c219f71e6345696c49fa4b9d08e9225d3d39393425152c97e71dd24601c11abcfa0f12f53c680bd3ae757b8134a9c10d429615869217fdd5885c4db174985703a6d6de94a667eac3023443a8337ae1bc601b76d7d38ec3c34463105f0d3949d78e562a039e4469548b609395de5a4fd43c46ca9fd6ee29ada5e
        var s = new SHAKE256(1088/8);
        var msg = HexFormat.of().parseHex("6ae23f058f0f2264a18cd609acc26dd4dbc00f5c3ee9e13ecaea2bb5a2f0bb6b");
        var output = HexFormat.of().parseHex("b9b92544fb25cfe4ec6fe437d8da2bbe00f7bdaface3de97b8775a44d753c3adca3f7c6f183cc8647e229070439aa9539ae1f8f13470c9d3527fffdeef6c94f9f0520ff0c1ba8b16e16014e1af43ac6d94cb7929188cce9d7b02f81a2746f52ba16988e5f6d93298d778dfe05ea0ef256ae3728643ce3e29c794a0370e9ca6a8bf3e7a41e8677067");
        s.update(msg, 0, msg.length);
        if (!Arrays.equals(s.digest(), output)) {
            throw new RuntimeException();
        }
        s = new SHAKE256(1096/8);
        msg = HexFormat.of().parseHex("8d8001e2c096f1b88e7c9224a086efd4797fbf74a8033a2d422a2b6b8f6747e4");
        output = HexFormat.of().parseHex("2e975f6a8a14f0704d51b13667d8195c219f71e6345696c49fa4b9d08e9225d3d39393425152c97e71dd24601c11abcfa0f12f53c680bd3ae757b8134a9c10d429615869217fdd5885c4db174985703a6d6de94a667eac3023443a8337ae1bc601b76d7d38ec3c34463105f0d3949d78e562a039e4469548b609395de5a4fd43c46ca9fd6ee29ada5e");
        s.update(msg, 0, msg.length);
        if (!Arrays.equals(s.digest(), output)) {
            throw new RuntimeException();
        }
    }
}

BTW, when n > 200, the state buffer in SHA3.java does not have enough bytes and the last line of implDigest() throws an exception. It probably needs to squeeze multiple times to fullfill the output.
Comments
[jdk21u-fix-request] Approval Request from Dmitry Chuyko The change improves NIST CAVP conformance for SHAKE256 algorithm. The test provided in the bug description currently fails on JDK 21 and passes after patching. SHA3 part is clean, SHAKE128 part is removed. Testing: provided "A2" test, test/jdk/java/security jtreg.
06-09-2024

I've removed SHAKE128 from 21u PR.
29-08-2024

The JDK 21 backport ought not to include the new SHAKE128 implementation.
22-08-2024

A pull request was submitted for review. Branch: master URL: https://git.openjdk.org/jdk21u-dev/pull/902 Date: 2024-08-08 14:09:26 +0000
08-08-2024

Changeset: fcb4df26 Author: Weijun Wang <weijun@openjdk.org> Date: 2023-11-20 23:28:48 +0000 URL: https://git.openjdk.org/jdk/commit/fcb4df26f18c7a10e4511f4be948c887f14ee093
20-11-2023

I'll also take this chance to add the SHAKE128 implementation.
20-11-2023

A pull request was submitted for review. URL: https://git.openjdk.org/jdk/pull/16698 Date: 2023-11-16 20:45:29 +0000
16-11-2023