J2SE Version (please include all output from java -version flag):
java version "1.6.0_10"
Java(TM) SE Runtime Environment (build 1.6.0_10-b33)
Java HotSpot(TM) Client VM (build 11.0-b15, mixed mode, sharing)

Does this problem occur on J2SE 5.0.x or 6.0?  Yes / No (pick one)
Yes

Operating System Configuration Information (be specific):
Client: Microsoft Windows XP [Version 5.1.2600]
Server: Redhat Linux Fedora Core release 3 (Heidelberg)

Bug Description:
Selector.select() may hang while key SelectionKey is interested in OP_CONNECT

When this problem occurs I can see using jconsole that the thread calling select() is hung here:
sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:274)
sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:256)
sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:137)
sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
   - locked sun.nio.ch.Util$1@139422b
   - locked java.util.Collections$UnmodifiableSet@2720f6
   - locked sun.nio.ch.WindowsSelectorImpl@e50f2c
sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
sun.nio.ch.SelectorImpl.select(SelectorImpl.java:84)

Using jmap and jhat I can see that the Selector has:
cancelledKeys empty
selectedKeys empty
keys 1 key interested in OP_CONNECT

The code below is a very cut down version of the code where this situation came about.
The code below should either cause an exception or write some lines of text followed by Completed, it should never hang forever at the selector.select() line (it is acceptable if it blocks for a while - but it has now been hung for more than an hour).
Note: I have never managed to reproduce the problem with the code below, however it includes everything relevant including synchronization, and the actual code almost never causes the problem.

When the problem did occur it had already managed to connect and disconnect once to the same address, and while this client was hung other clients running from the same computer were able to connect afterwards with the original client still hung.
It is not known if a call to 'channel.finishConnect()' would succeed or fail with the hung client as it is still hung on the selector.select(); line.

In theory the problem should be able to reproduced with any server not running on the client machine (if it is running there selector.select() would never get called).

import static java.lang.Math.*;
import static java.lang.System.*;
import static java.nio.channels.SelectionKey.*;
import java.io.*;
import java.net.*;
import java.nio.channels.*;
import java.util.*;

public class Test {
  private static final int CONNECTIONS = 10;
  private static final String HOSTNAME = "www.sun.com";
  private static final int PORT = 80;
 
  private static int connected = 0;
  private static InetSocketAddress address;
  private static Selector selector;

  private static void connectionDone(SelectableChannel aChannel) {
    connected++;
    out.println("Connected " + connected + " out of " + CONNECTIONS);
    try {
      aChannel.close();
    }
    catch (IOException ex) {
      out.println(ex.getMessage());
    }
    if (connected == CONNECTIONS) {
      out.println("Completed");
      exit(0);
    }
  }

  private static synchronized void createConnection() throws IOException {
    SocketChannel channel = SocketChannel.open();
    channel.configureBlocking(false);
    if (channel.connect(address)) {
      connectionDone(channel);
    }
    else {
      selector.wakeup();
      channel.register(selector, OP_CONNECT, new Object());
    }
  }

  private static synchronized void processSelectedKeys(Set<SelectionKey> aSetOfKeys)
      throws IOException {
    for (SelectionKey key : aSetOfKeys) {
      SelectableChannel channel = key.channel();
      if (channel instanceof SocketChannel) {
        if (((SocketChannel)channel).finishConnect()) {
          connectionDone(channel);
        }
      }
    }
    aSetOfKeys.clear();
  }

  public static void main(String[] args) {
    try {
      address = new InetSocketAddress(HOSTNAME, PORT);
      selector = Selector.open();
      new Thread() {
        @Override
        public void run() {
          try {
            while (true) {
              out.println("Selecting");
              selector.select();
              Set<SelectionKey> keySet = selector.selectedKeys();
              out.println("Selected Keys Size: " + keySet.size());
              processSelectedKeys(keySet);
            }
          }
          catch (IOException ex) {
            out.println(ex.getMessage());
            exit(1);
          }
        }
      }.start();
      for (int i = 0; i < CONNECTIONS; i++) {
        try {
          Thread.sleep((long)(random() * 10));
        }
        catch (InterruptedException ex) {
        }
        createConnection();
      }
    }
    catch (IOException ex) {
      out.println(ex.getMessage());
      exit(1);
    }
  }
}