United StatesChange Country, Oracle Worldwide Web Sites Communities I am a... I want to...
Bug ID: JDK-6578658 Request for raw RSA (NONEwithRSA) Signature support in SunMSCAPI
JDK-6578658 : Request for raw RSA (NONEwithRSA) Signature support in SunMSCAPI

Details
Type:
Bug
Submit Date:
2007-07-10
Status:
Closed
Updated Date:
2011-06-22
Project Name:
JDK
Resolved Date:
2011-06-22
Component:
security-libs
OS:
generic,windows_xp
Sub-Component:
javax.crypto
CPU:
x86,generic
Priority:
P3
Resolution:
Fixed
Affected Versions:
6,6u20,6u21
Fixed Versions:

Related Reports
Backport:
Duplicate:
Relates:
Relates:

Sub Tasks

Description
A DESCRIPTION OF THE REQUEST :
Requesting support for doing raw RSA signing with SunMSCAPI - at present only SHA-1, MD5 and MD2 signing is possible.

This would include the SunMSCAPI provider having the following property or similar:

Signature.NONEwithRSA=sun.security.mscapi.RSASignature$NONE

Alternatively, the SunMSCAPI cipher class could be changed to support encrypting with private keys, although i suspect that the underlying MSCAPI doesn't allow that.

JUSTIFICATION :
NONEwithRSA Signing is required by JSSE for client authentication enabled SSL. At the moment because SunMSCAPI doesn't include a NONEwithRSA property, the default NONEwithRSA signature class is being used. This class merely wraps the cipher class - and the MSCAPI cipher class doesn't support encrypting with private keys. A "Bad Key" exception results.

See: http://forum.java.sun.com/thread.jspa?threadID=5188515&tstart=0

EXPECTED VERSUS ACTUAL BEHAVIOR :
EXPECTED -
After setting a client application to use the Windows key and trust stores in the normal manner, and given that the server and client trust each other's certificates, using JSSE to connect to a server using SSL with client authentication should work.
ACTUAL -
The following error is thrown during the client verification stage, on the client end:

Exception in thread "main" javax.net.ssl.SSLException: java.security.ProviderException: java.security.KeyException: Bad Key.
	at com.sun.net.ssl.internal.ssl.Alerts.getSSLException(Alerts.java:190)
	at com.sun.net.ssl.internal.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1520)
	at com.sun.net.ssl.internal.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1487)
	at com.sun.net.ssl.internal.ssl.SSLSocketImpl.handleException(SSLSocketImpl.java:1470)
	at com.sun.net.ssl.internal.ssl.SSLSocketImpl.handleException(SSLSocketImpl.java:1396)
	at com.sun.net.ssl.internal.ssl.AppOutputStream.write(AppOutputStream.java:64)
	at sun.nio.cs.StreamEncoder.writeBytes(StreamEncoder.java:202)
	at sun.nio.cs.StreamEncoder.implFlushBuffer(StreamEncoder.java:272)
	at sun.nio.cs.StreamEncoder.implFlush(StreamEncoder.java:276)
	at sun.nio.cs.StreamEncoder.flush(StreamEncoder.java:122)
	at java.io.OutputStreamWriter.flush(OutputStreamWriter.java:212)
	at SunMSCAPITest.main(SunMSCAPITest.java:37)
Caused by: java.security.ProviderException: java.security.KeyException: Bad Key.

	at sun.security.mscapi.RSACipher.doFinal(RSACipher.java:242)
	at sun.security.mscapi.RSACipher.engineDoFinal(RSACipher.java:266)
	at javax.crypto.Cipher.doFinal(DashoA13*..)
	at java.security.Signature$CipherAdapter.engineSign(Signature.java:1225)
	at java.security.Signature$Delegate.engineSign(Signature.java:1128)
	at java.security.Signature.sign(Signature.java:522)
	at com.sun.net.ssl.internal.ssl.RSASignature.engineSign(RSASignature.java:149)
	at java.security.Signature$Delegate.engineSign(Signature.java:1128)
	at java.security.Signature.sign(Signature.java:522)
	at com.sun.net.ssl.internal.ssl.HandshakeMessage$CertificateVerify.<init>(HandshakeMessage.java:1216)
	at com.sun.net.ssl.internal.ssl.ClientHandshaker.serverHelloDone(ClientHandshaker.java:715)
	at com.sun.net.ssl.internal.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:197)
	at com.sun.net.ssl.internal.ssl.Handshaker.processLoop(Handshaker.java:511)
	at com.sun.net.ssl.internal.ssl.Handshaker.process_record(Handshaker.java:449)
	at com.sun.net.ssl.internal.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:817)
	at com.sun.net.ssl.internal.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1029)
	at com.sun.net.ssl.internal.ssl.SSLSocketImpl.writeRecord(SSLSocketImpl.java:621)
	at com.sun.net.ssl.internal.ssl.AppOutputStream.write(AppOutputStream.java:59)
	... 6 more
Caused by: java.security.KeyException: Bad Key.

	at sun.security.mscapi.RSACipher.encryptDecrypt(Native Method)
	at sun.security.mscapi.RSACipher.doFinal(RSACipher.java:222)
	... 23 more

---------- BEGIN SOURCE ----------
import java.io.BufferedReader;
import java.io.InputStreamReader;
import java.io.OutputStreamWriter;
import java.io.Writer;
import java.net.Socket;

import javax.net.ssl.SSLSocketFactory;

public class SunMSCAPITest {
	
	// Website to access
	public static final String TARGET_HTTPS_SERVER = "localhost";
	public static final int TARGET_HTTPS_PORT = 8443;

	public static void main(String[] args) throws Exception {
		
		System.setProperty("javax.net.ssl.keyStoreProvider","SunMSCAPI");
		System.setProperty("javax.net.ssl.keyStoreType","Windows-MY");
		System.setProperty("javax.net.ssl.trustStoreProvider","SunMSCAPI");
		System.setProperty("javax.net.ssl.trustStoreType","Windows-ROOT");

		Socket socket = SSLSocketFactory.getDefault().createSocket(TARGET_HTTPS_SERVER,
				TARGET_HTTPS_PORT);
		
		try {
			Writer out = new OutputStreamWriter(socket.getOutputStream(),
					"ISO-8859-1");
			out.write("GET / HTTP/1.1\r\n");
			out.write("Host: " + TARGET_HTTPS_SERVER + ":" + TARGET_HTTPS_PORT
					+ "\r\n");
			out.write("Agent: SSL-TEST\r\n");
			out.write("\r\n");
			out.flush();
			BufferedReader in = new BufferedReader(new InputStreamReader(socket
					.getInputStream(), "ISO-8859-1"));
			String line = null;
			while ((line = in.readLine()) != null) {
				System.out.println(line);
			}
		} finally {
			socket.close();
		}
	}
}
---------- END SOURCE ----------

CUSTOMER SUBMITTED WORKAROUND :
At the moment this can be worked around by setting both the client and server applications to use one of the other signature types instead of NONEwithRSA:

i.e.
Provider p = Security.getProvider("SunMSCAPI");
p.setProperty("Signature.NONEwithRSA","sun.security.mscapi.RSASignature$SHA1");

This isn't always possible to do on the server end though, and breaks SSL compatibility.

                                    

Comments
EVALUATION

I offered to help Vinnie with this, but haven't been able to figure this out.

My original thought was that Windows 7 might have been enhanced so that Ciphers can take either public or private keys for encryption, but that is not the case. I was able to get certain PrivateKeys to initialize a Cipher object on XP and Windows 7.

Here is a small code snippet that shows a simplified test case.

import java.security.*;
import java.util.Enumeration;
import javax.crypto.Cipher;

public class Main {

    public static void main(String[] args) throws Exception {

        Provider[] p = Security.getProviders();

        KeyStore ks = KeyStore.getInstance("Windows-MY", "SunMSCAPI");
        ks.load(null, null);

        Enumeration e = ks.aliases();

        PrivateKey privateKey = null;

        PublicKey publicKey = null;

        while (e.hasMoreElements()) {
            String alias = (String) e.nextElement();
            if (ks.isKeyEntry(alias)) {
                System.out.println("got it: " + alias);
                privateKey = (PrivateKey) ks.getKey(alias, null);
                publicKey = (PublicKey) ks.getCertificate(alias).getPublicKey();
            }
        }

        System.out.println(privateKey);

        Signature sig = Signature.getInstance("NONEwithRSA", "SunMSCAPI");
        sig.initSign(privateKey);
        sig.update((byte)0x15);
        byte [] ba = sig.sign();
    }
}

For creating keypairs/certs, I used one of three methods:

1)  The MS SDK makecert utility.  (FAIL)

% makecert c:/tmp/cert.cer -a sha1 \
    -n "CN=Sun JSSE TestService, O=Java, C=US" -ss My

2)  keytool  (PASS)

[wetmore@dualcongas] 338 >keytool -genkey -keyalg rsa -keystore foo.p12 -storety
pe pkcs12
Enter keystore password:
Re-enter new password:
What is your first and last name?
  [Unknown]:  Sun JSSE TestService
What is the name of your organizational unit?
  [Unknown]:  Java
What is the name of your organization?
  [Unknown]:  Sun
What is the name of your City or Locality?
  [Unknown]:  SCA
What is the name of your State or Province?
  [Unknown]:  CA
What is the two-letter country code for this unit?
  [Unknown]:  US
Is CN=Brad, OU=Java, O=Sun, L=SCA, ST=CA, C=US correct?
  [no]:  yes

3) The interoperability test suite test certs that were created for TLS 1.2 during JDK 7 development.  (PASS)

% cd <JDK7>/jdk/test/closed/sun/security/ssl/sanity/compatibility/helpers/certs
% sh ./generate.sh

I thought maybe it maybe had something to do with  keyusages, but 2) doesn't produce keyusages, so I'm not seeing something.

Only 1) failed to run, it failed with:

[wetmore@dualcongas] 317 >java Main
got it: Sun JSSE Test Service
RSAPrivateKey [size=1024 bits, type=Signature, container=JoeSoft]
Exception in thread "main" java.security.ProviderException: java.security.KeyExc
eption: Bad Key.

        at sun.security.mscapi.RSACipher.doFinal(RSACipher.java:242)
        at sun.security.mscapi.RSACipher.engineDoFinal(RSACipher.java:266)
        at javax.crypto.Cipher.doFinal(DashoA13*..)
        at java.security.Signature$CipherAdapter.engineSign(Signature.java:1225)

        at java.security.Signature$Delegate.engineSign(Signature.java:1128)
        at java.security.Signature.sign(Signature.java:522)
        at Main.main(Main.java:37)
Caused by: java.security.KeyException: Bad Key.

        at sun.security.mscapi.RSACipher.encryptDecrypt(Native Method)
        at sun.security.mscapi.RSACipher.doFinal(RSACipher.java:222)
        ... 6 more

To do cert management while testing, it is best to use certmgr rather than using the cert manager in IE.  If you use IE, go into the Internet Options, then content, certificates, personal certificates.

Originally, I was thinking that Ciphers wouldn't allow initialization with PrivateKeys, so we might want to consider rewriting this using CryptSignMessage, if there is an option for NoPadding.  That is, we simply have a regular Cipher *AND* Signature.  But at this point, I had no problems using either 2)/3), so I'm not sure where the problem really lies.

Also, when looking at this, please note the second webbug, I think the problems described are different, and should be investigated further.  This seems to be true, I was getting different values on each run.

    public static void main(String[] args) throws Exception {

        Provider[] p = Security.getProviders();

        KeyStore ks = KeyStore.getInstance("Windows-MY", "SunMSCAPI");
        ks.load(null, null);

        Enumeration e = ks.aliases();

        PrivateKey privateKey = null;

        PublicKey publicKey = null;

        while (e.hasMoreElements()) {
            String alias = (String) e.nextElement();
            if (ks.isKeyEntry(alias)) {
                System.out.println("got it: " + alias);
                privateKey = (PrivateKey) ks.getKey(alias, null);
                publicKey = (PublicKey) ks.getCertificate(alias).getPublicKey();
            }
        }

        System.out.println(privateKey);

        Signature sig1 = Signature.getInstance("NONEwithRSA", "SunMSCAPI");
        sig1.initSign(privateKey);
        sig1.update((byte)0x15);
        byte [] ba = sig1.sign();
        System.out.println("Output Length: " + ba.length*8);
        for (byte b : ba) {
            System.out.print(b + " ");
        }
        System.out.println();

        Signature sig2 = Signature.getInstance("NONEwithRSA", "SunMSCAPI");
        sig2.initSign(privateKey);
        sig2.update((byte)0x15);
        byte [] ba2 = sig2.sign();
        System.out.println("Output Length: " + ba2.length*8);
        for (byte b : ba2) {
            System.out.print(b + " ");
        }
        System.out.println();

        // using the first available
        sig1 = Signature.getInstance("NONEwithRSA");
        sig1.initVerify(publicKey);
        sig1.update((byte)0x15);
        if (sig1.verify(ba)) {
            System.out.println("Verify PASSED");
        } else {
            System.out.println("Verify FAILED");
        }

        sig1 = Signature.getInstance("NONEwithRSA");
        sig1.initVerify(publicKey);
        sig1.update((byte)0x15);
        if (sig1.verify(ba2)) {
            System.out.println("Verify PASSED");
        } else {
            System.out.println("Verify FAILED");
        }
    }
}

which outputs:

[wetmore@dualcongas] 330 >java Main
got it: CN=Brad,OU=Java,O=Sun,L=SCA,ST=CA,C=US
RSAPrivateKey [size=1024 bits, type=Exchange, container={A18137D3-228C-42C9-A81C
-AC29F1AC4CA7}]
Output Length: 1024
59 33 -5 -115 103 -97 -123 -41 38 -93 53 -52 20 39 -51 -114 97 39 53 -1 32 48 32
 -20 28 2 89 113 123 -84 -31 -68 -24 -107 -93 66 -76 -80 -47 -81 -91 79 115 -35
110 -51 -44 94 -112 114 -126 -49 -127 38 -61 -48 -89 -28 57 -74 -117 95 49 44 -4
2 -84 27 94 -90 -33 23 -7 -35 -63 -35 -15 -59 66 16 -15 12 105 -106 -86 17 122 -
4 121 -87 40 -63 -121 119 4 -3 17 112 50 -120 72 -35 -12 10 61 -34 84 111 3 -59
50 103 88 118 -102 -37 -24 65 3 -6 -3 -31 -80 -74 119 -59 -35 -119 -97
Output Length: 1024
102 62 11 -53 -78 -31 -41 -114 -127 70 -78 17 84 71 16 -32 125 27 -75 -94 -50 -2
9 -51 -98 106 39 -92 113 -78 -99 -107 -113 12 -39 98 -19 -68 -91 -6 84 99 104 -1
23 -77 -123 -52 -45 83 -6 118 21 -56 58 92 -80 27 -62 12 -12 34 -28 -24 -96 -38
73 -118 -54 35 -20 -64 -76 -26 -10 -96 77 -72 44 -74 -89 69 -91 13 -122 -89 -22
-64 72 -23 28 -11 84 -62 -90 -85 -91 -80 60 77 -55 114 -34 -10 105 -53 -66 -122
-82 -66 12 -96 63 -65 -1 -49 89 -45 -48 -93 -108 94 -120 103 22 53 -25 69 102 -2
3
Verify FAILED
Verify FAILED

Note the signatures are indeed different.

At this point, I need to move on.
                                     
2011-02-15



Hardware and Software, Engineered to Work Together