JDK-6575075 : DES\CBC\NOPADDING works improperly in some cases
  • Type: Bug
  • Component: security-libs
  • Sub-Component: javax.crypto:pkcs11
  • Affected Version: 6
  • Priority: P4
  • Status: Closed
  • Resolution: Duplicate
  • OS: solaris_10
  • CPU: sparc
  • Submitted: 2007-06-28
  • Updated: 2011-02-16
  • Resolved: 2008-02-04
Related Reports
Duplicate :  
Description
FULL PRODUCT VERSION :
java version "1.6.0_01"
Java(TM) SE Runtime Environment (build 1.6.0_01-b06)
Java HotSpot(TM) Server VM (build 1.6.0_01-b06, mixed mode)


ADDITIONAL OS VERSION INFORMATION :
SunOS morpheus 5.10 Generic_118844-26 i86pc i386 i86pc


A DESCRIPTION OF THE PROBLEM :
On the Solaris 10 x86 implementation, calling Cipher.update(buffer) with a buffer that does not contain a full block (i.e. size % 8 != 0) will cause the decryption to fail to perform correctly. It works correctly in the Windows 32 bit and Solaris 8 SPARC implementations.

STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
Run the test program included below

EXPECTED VERSUS ACTUAL BEHAVIOR :
EXPECTED -
Output on Win32:
e7 9e 0e 8e 79 9a fd 0d
e7 9e 0e 8e 79 9a fd 0d
e7 9e 0e 8e 79 9a fd 0d


ACTUAL -
Output on Solaris 10 x86:

e7 9e 0e 8e 79 9a fd 0d
c0 02 06 dd 20 62 12 31
Exception in thread "main" java.security.ProviderException: update() failed
        at sun.security.pkcs11.P11Cipher.implUpdate(P11Cipher.java:460)
        at sun.security.pkcs11.P11Cipher.engineUpdate(P11Cipher.java:391)
        at sun.security.pkcs11.P11Cipher.engineUpdate(P11Cipher.java:380)
        at javax.crypto.Cipher.update(DashoA13*..)
        at atest.DESFail.decrypt(DESFail.java:70)
        at atest.DESFail.main(DESFail.java:20)
Caused by: sun.security.pkcs11.wrapper.PKCS11Exception: CKR_BUFFER_TOO_SMALL
        at sun.security.pkcs11.wrapper.PKCS11.C_DecryptUpdate(Native Method)
        at sun.security.pkcs11.P11Cipher.implUpdate(P11Cipher.java:453)
        ... 5 more


ERROR MESSAGES/STACK TRACES THAT OCCUR :
See Actual Result

REPRODUCIBILITY :
This bug can be reproduced always.

---------- BEGIN SOURCE ----------
package atest;

import java.util.Arrays;

import javax.crypto.Cipher;
import javax.crypto.spec.IvParameterSpec;
import javax.crypto.spec.SecretKeySpec;

public class DESFail {
    private static final String DES_KEY = "deadbeefdeadbeef";
    
    public static void main(String[] args) throws Exception {
        // good on Solaris 10 x86
        System.out.println(new DESFail().decrypt(false, false, DES_KEY));
        
        // bad on Solarix 10 x86, good on Win32 and Solaris 8 SPARC
        System.out.println(new DESFail().decrypt(true, false, DES_KEY));

        // exception on Solarix 10 x86, good on Win32 and Solaris 8 SPARC
        System.out.println(new DESFail().decrypt(true, true, DES_KEY));
    }

    public String decrypt(boolean fail, boolean bufferFail, String key) throws Exception {
        byte[] codeBytesMaster = new byte[] {
                1, 2, 3, 4, 5, 6, 7, 8,
                9, 10, 11, 12, 13, 14, 15, 16
        };
        byte[] buffer = new byte[1024*1024];
        byte[] keyBytes = hexKeyToBytes(key);
        
        byte[] ivBytes = new byte[8];
        IvParameterSpec iv = new IvParameterSpec(ivBytes);
    
        Cipher cipher = Cipher.getInstance("DES/CBC/NoPadding");
        cipher.init(Cipher.DECRYPT_MODE, new SecretKeySpec(keyBytes, "DES"), iv);
            
        // "read" the first buffer of info:
        int first = 1048544 + (fail ? 1 : 0);
        for (int idx = 0; idx < first; idx++) {
            buffer[idx] = codeBytesMaster[idx % 16];
        }
        {
            byte[] partA = cipher.update(buffer, 0, first);
        }
        
        // "read" the second buffer of info:
        for (int idx = 0; idx < first; idx++) {
            buffer[idx] = codeBytesMaster[(idx + (fail ? 1 : 0)) % 16];
        }
        {
            byte[] partB = cipher.update(buffer, 0, first);
        }
        
        // "read" the third buffer of info:
        for (int idx = 0; idx < first; idx++) {
            buffer[idx] = codeBytesMaster[(idx + (fail ? 2 : 0)) % 16];
        }
        {
            byte[] partC = cipher.update(buffer, 0, first);
        }
        
        // "read" the fourth buffer of info
        int second = (fail ? 53781 : 53784);
        for (int idx = 0; idx < second; idx++) {
            buffer[idx] = codeBytesMaster[(idx + (fail ? 3 : 0)) % 16];
        }
        byte[] partD;
        if (bufferFail) {
            // this throws an exception
            partD = cipher.update(buffer, 0, second);
        } else {
            // this demonstrates decryption failure/corruption
            for (int loop = 0; loop < 3361; loop++) {
                partD = cipher.update(buffer, loop * 16, 16);
            }
            partD = cipher.update(buffer, 53776, 8);
        }
        
        int ofs = (bufferFail ? 53776 : 0);
        
        return String.format("%02x %02x %02x %02x %02x %02x %02x %02x ",
                partD[ofs + 0], partD[ofs + 1], partD[ofs + 2], partD[ofs + 3],
                partD[ofs + 4], partD[ofs + 5], partD[ofs + 6], partD[ofs + 7]);
    }
    
    public static byte[] hexKeyToBytes(String key) throws Exception {
        byte[] kb = new byte[8];
        if (key.length() != 16) {
            throw new Exception("Key is the wrong length (" + key.length() + " != 16)");
        }
        for (int loop = 0; loop < 8; loop++) {
            kb[loop] = (byte) Integer.parseInt(key.substring(loop * 2, loop * 2 + 2), 16);
        }
        return kb;
    }

}

---------- END SOURCE ----------

CUSTOMER SUBMITTED WORKAROUND :
First case of test program -- only Cipher.update() buffers of data that are complete

Comments
EVALUATION Since the fix has already went into jdk7 and openjdk6 through 4898484, it's probably cleaner to just close this one.
04-02-2008

EVALUATION Received some more information from the submitter. After further investigation, it appears that the cause of problem is in p11_crypt.c. Although PKCS#11 spec states that it's ok to use the same buffer as both the input and output, this can lead to data corruption for some PKCS#11 impls especially when there are some previously buffered input bytes. At some point, it produced more bytes than it read in and thus overwrote/corrupted the input data as a result. As for the CKR_BUFFER_TOO_SMALL error code, it's also relevant to the same input/output buffer usage - When malloc'ing the buffer, it's based on the input size and does not consider the output size at all. Thus, when the internally buffered input bytes accumulated to as much as blocksize, the output size would be bigger than the input size and thus leads to the CKR_BUFFER_TOO_SMALL problem. In jdk7, this issue has been addressed as a by-product of fixing 4898484 which has been integrated in both jdk7 and openjdk6. Leave the bug status in "Cause Known" state for now. If It's decided later to backport 4898484, then I'll close this one.
13-11-2007

EVALUATION Cannot reproduce on Solaris10 x86 machine using 6.0 or 7. Have sent an email to the original submitter for more information.
16-10-2007