JDK-6448723 : SSLContext + X509KeyManager implementation cause SSLHandshakeException: no cipher suites in common
  • Type: Bug
  • Component: security-libs
  • Sub-Component: javax.net.ssl
  • Affected Version: 6u2
  • Priority: P3
  • Status: Closed
  • Resolution: Other
  • OS: linux_redhat_8.0
  • CPU: x86
  • Submitted: 2006-07-13
  • Updated: 2022-06-01
  • Resolved: 2022-06-01
Description
Reproducable with all 1.5.0_0X and 1.6.0._0X.

In project GlassFish (https://glassfish.dev.java.net), the HTTP front Component named Grizzly is implemented using NIO. When initializing the SSLContext:

            context = SSLContext.getInstance(protocol);
 
            // Configure SSL session timeout and cache size
            configureSSLSessionContext(context.getServerSessionContext());

            context.init(getKeyManagers(algorithm,
                                        (String) attributes.get("keyAlias")),
                         getTrustManagers(),
                         new SecureRandom());

where getKeyManagers is implemented has:

    /**
     * Gets the initialized key managers.
     */
    protected KeyManager[] getKeyManagers(String algorithm,
                                          String keyAlias)
                throws Exception {

        KeyManager[] kms = null;

        String keystorePass = getKeystorePassword();

        KeyStore ks = getKeystore(keystorePass);
        if (keyAlias != null && !ks.isKeyEntry(keyAlias)) {
            throw new IOException(sm.getString("jsse.alias_no_key_entry", keyAlias));
        }

        KeyManagerFactory kmf = KeyManagerFactory.getInstance(algorithm);
        kmf.init(ks, keystorePass.toCharArray());

        kms = kmf.getKeyManagers();
        if (keyAlias != null) {
            for(int i=0; i<kms.length; i++) {
                kms[i] = new JSSEKeyManager((X509KeyManager)kms[i], keyAlias);
            }
        }

where JSSEKeyManager is an implementation of:

public final class JSSEKeyManager implements X509KeyManager.

Any SSL+ NIO request will throws the following exception:

Caused by: javax.net.ssl.SSLHandshakeException: no cipher suites in common
        at com.sun.net.ssl.internal.ssl.Alerts.getSSLException(Alerts.java:150)
        at com.sun.net.ssl.internal.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1352)
        at com.sun.net.ssl.internal.ssl.Handshaker.fatalSE(Handshaker.java:176)
        at com.sun.net.ssl.internal.ssl.Handshaker.fatalSE(Handshaker.java:164)
        at com.sun.net.ssl.internal.ssl.ServerHandshaker.chooseCipherSuite(ServerHandshaker.java:639)
        at com.sun.net.ssl.internal.ssl.ServerHandshaker.clientHello(ServerHandshaker.java:450)
        at com.sun.net.ssl.internal.ssl.ServerHandshaker.processMessage(ServerHandshaker.java:178)
        at com.sun.net.ssl.internal.ssl.Handshaker.processLoop(Handshaker.java:495)
        at com.sun.net.ssl.internal.ssl.Handshaker$1.run(Handshaker.java:437)        at java.security.AccessController.doPrivileged(Native Method)
        at com.sun.net.ssl.internal.ssl.Handshaker$DelegatedTask.run(Handshaker.java:932) 


This is produced when interacting with SSLEngine.unwrap() after a successful socketChannel.read()

>     211         inputBB.flip();
>     212         SSLEngineResult result = sslEngine.unwrap(inputBB, byteBuffer);

the result = SSLEngineResult.STATUS = OK, result.getHandshakeStatus() return NEED_TASK but the result.byteProduced() == 0. Hence when latter calling:

>         Runnable runnable;
>         while ((runnable = sslEngine.getDelegatedTask()) != null) {
>             runnable.run();
>         }
>         return sslEngine.getHandshakeStatus();

This is where the handshake fail and the exception is thrown. The ssl debug only have:

[#|2006-07-13T12:04:59.685-0400|INFO|sun-appserver-pe9.1|javax.enterprise.system.stream.out|_ThreadID=12;_ThreadName=httpSSLWorkerThread-8181-0;|
Cipher Suites: [SSL_DHE_RSA_WITH_3DES_EDE_CBC_SHA, SSL_DHE_DSS_WITH_3DES_EDE_CBC_SHA, SSL_RSA_WITH_3DES_EDE_CBC_SHA, SSL_DHE_DSS_WITH_RC4_128_SHA, SSL_RSA_WITH_RC4_128_SHA, SSL_RSA_WITH_RC4_128_MD5, SSL_DHE_DSS_EXPORT1024_WITH_RC4_56_SHA, SSL_RSA_EXPORT1024_WITH_RC4_56_SHA, SSL_DHE_DSS_EXPORT1024_WITH_DES_CBC_SHA, SSL_RSA_EXPORT1024_WITH_DES_CBC_SHA, Unknown 0x0:0x61, Unknown 0x0:0x60, SSL_DHE_RSA_WITH_DES_CBC_SHA, SSL_DHE_DSS_WITH_DES_CBC_SHA, SSL_RSA_WITH_DES_CBC_SHA, SSL_DHE_RSA_EXPORT_WITH_DES40_CBC_SHA, SSL_DHE_DSS_EXPORT_WITH_DES40_CBC_SHA, SSL_RSA_EXPORT_WITH_DES40_CBC_SHA, SSL_RSA_EXPORT_WITH_RC2_CBC_40_MD5, SSL_RSA_EXPORT_WITH_RC4_40_MD5]|#]

[#|2006-07-13T12:04:59.685-0400|INFO|sun-appserver-pe9.1|javax.enterprise.system.stream.out|_ThreadID=12;_ThreadName=httpSSLWorkerThread-8181-0;|
Compression Methods:  { |#]

[#|2006-07-13T12:04:59.685-0400|INFO|sun-appserver-pe9.1|javax.enterprise.system.stream.out|_ThreadID=12;_ThreadName=httpSSLWorkerThread-8181-0;|0|#]

[#|2006-07-13T12:04:59.685-0400|INFO|sun-appserver-pe9.1|javax.enterprise.system.stream.out|_ThreadID=12;_ThreadName=httpSSLWorkerThread-8181-0;| }|#]

[#|2006-07-13T12:04:59.685-0400|INFO|sun-appserver-pe9.1|javax.enterprise.system.stream.out|_ThreadID=12;_ThreadName=httpSSLWorkerThread-8181-0;|
***|#]

[#|2006-07-13T12:04:59.686-0400|INFO|sun-appserver-pe9.1|javax.enterprise.system.stream.out|_ThreadID=12;_ThreadName=httpSSLWorkerThread-8181-0;|
httpSSLWorkerThread-8181-0, fatal error: 40: no cipher suites in common
javax.net.ssl.SSLHandshakeException: no cipher suites in common|#]

[#|2006-07-13T12:04:59.686-0400|INFO|sun-appserver-pe9.1|javax.enterprise.system.stream.out|_ThreadID=12;_ThreadName=httpSSLWorkerThread-8181-0;|
httpSSLWorkerThread-8181-0|#]

[#|2006-07-13T12:04:59.686-0400|INFO|sun-appserver-pe9.1|javax.enterprise.system.stream.out|_ThreadID=12;_ThreadName=httpSSLWorkerThread-8181-0;|, SEND TLSv1 ALERT:  |#]

[#|2006-07-13T12:04:59.686-0400|INFO|sun-appserver-pe9.1|javax.enterprise.system.stream.out|_ThreadID=12;_ThreadName=httpSSLWorkerThread-8181-0;|fatal, |#]

[#|2006-07-13T12:04:59.686-0400|INFO|sun-appserver-pe9.1|javax.enterprise.system.stream.out|_ThreadID=12;_ThreadName=httpSSLWorkerThread-8181-0;|description = handshake_failure|#]

[#|2006-07-13T12:04:59.687-0400|INFO|sun-appserver-pe9.1|javax.enterprise.system.stream.out|_ThreadID=12;_ThreadName=httpSSLWorkerThread-8181-0;|
httpSSLWorkerThread-8181-0, WRITE: TLSv1 Alert, length = 2|#] 

If I implements X509ExtendedKeyManager instead

public final class JSSEKeyManager extends X509ExtendedKeyManager

everything works fine. I know the docs of  X509KeyManager states:

>  * <LI> obtain the corresponding key material for given aliases.
>  * </UL>
>  * <P>
>  * Note: the X509ExtendedKeyManager should be used in favor of this
>  * class.
>  *
>  * @since 1.4
>  * @version 1.16

but this is needs to be changed to must for NIO, because the implementation try to invoke:

    public String chooseEngineClientAlias(String[] keyType,
                                          Principal[] issuers,
                                          SSLEngine engine){
        return delegate.chooseClientAlias(keyType,issuers,null);
    }


    /**
     * Choose an alias to authenticate the server side of an SSLEngine connection
     * given the public key type and the list of certificate issuer authorities
     * recognized by the peer (if any).
     * @return Alias name for the desired key
     **/
    public String chooseEngineServerAlias(String keyType,
                                          Principal[] issuers,
                                          SSLEngine engine){
        return serverKeyAlias;
    }

on an interface (X509KeyManager impl) that doesn"t have those methods.

The SSLEngine doesn't mention this information at all, which is causing unexpected result. This needs to be documented or some kind of better exception is thrown.

Comments
EVALUATION null socket or engine means that implementations of this interface are free to select an alias applicable to any socket/engine. I would prefer to change the documentation or provide a friendly exception message, rather than cast (null SSLEngine) to use the methods for sockets.
14-02-2011

EVALUATION It is intentional that X509ExtendedKeyManager must be used with an SSLEngine. However, we need to change the documentation to make that clear and at a minimum provide better error diagnostics. Probably we should also change the SunJSSE SSLContext such that if chooseEngineServerAlias() is called on a KeyManager that does not implement X509ExtendedKeyManager, we simply call the method for sockets (chooseServerAlias) and pass null as the value of the SSLSocket. This is allowed by the specification of chooseServerAlias() and most KeyManager implementations never consult the SSLSocket/SSLEngine object anyway.
14-07-2006

WORK AROUND Do not implements X509KeyManager but extends X509ExtendedKeyManager
13-07-2006