JDK-8065994 : HTTP Tunnel connection to NTLM proxy reauthenticates instead of using keep-alive
  • Type: Bug
  • Component: core-libs
  • Sub-Component: java.net
  • Affected Version: 7u40
  • Priority: P3
  • Status: Closed
  • Resolution: Fixed
  • Submitted: 2014-11-26
  • Updated: 2016-12-13
  • Resolved: 2015-01-23
The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.
JDK 7 JDK 8 JDK 9
7u80Fixed 8u60Fixed 9 b48Fixed
Related Reports
Relates :  
Relates :  
Description
Submitter uses HttpURLConnection to connect from a Swing GUI to an application server. 
The connection supports tunneling HTTPS through an NTLM proxy. 

The GUI uses a local copy of the Java JRE, and uses the HttpURLConnection class 

The connection should use keep-alive by default since the client, proxy and server are all using HTTP/1.1. A single socket should be opened from the GUI and multiple HTTP requests should use the same socket. 

Before Java 1.7.0_40, the HTTPS tunnel through the NTLM proxy would do NTLM authentication on the initial CONNECT request, but not on the following HTTP requests. The GUI would open a single socket and reuse it. 

In Java 1.7.0_40 and all later JRE versions, including JRE for 1.7.0_72 and 1.8.0_20, each request results in the GUI sending a new CONNECT request to the proxy server, doing the NTLM authentication (successfully), and then creating a new tunnel and new socket to the server. 

The problem this causes is that the requests take much longer because it has to create a new socket, and do the SSL handshake with the server. The request time for the second and subsequent requests should normally be around 100ms, but because of the problem, they take 3-4 seconds. 

The HTTP headers between the GUI and the proxy servers appear identical in all the JRE versions, but the Java client's behavior is different. 

The last good JRE is 1.7.0_25. All versions before that work also OK. 
1.6.0_26 
1.7.0.17 
1.7.0_21 
1.7.0_25 

Tested all versions after that that were available for download, and none of them work. 
1.7.0_40 
1.7.0_45 
1.7.0_51 
1.7.0_67 
1.7.0_71 
1.7.0_72 
1.8.0_20 
Comments
BPR request removed at the request of Support - fix ready for next Update.
10-02-2015

Nightly results are good. SQE OK to take the fix to PSU15_02.
02-02-2015

I've attaching a new instrumented trace file which identifies the bug in this issue (7u80_broken_caching.txt) The Proxy equals test while retrieving a HttpsClient from the keep alive cache is failing on obtaining a cached reference in the NTML scenario. The sockaddress comparison fails, since the previously cached HttpsClient contains a Proxy with unresolved SocketAddress, while the subsequent attempt to use the same HttpsClient contains a Proxy with an unresolved address. [1st] *** new HttpsClient request with proxy = HTTP @ 10.6.160.94:8080 [2nd] FINEST: Looking for HttpClient for URL https://xx.xxx.com/qoms151/rfqtunnel and proxy value of HTTP @ /10.6.160.94:8080 Note the '/' before the IP address which appears to indicate a resolves address. I've identified the two calling sites that create the Proxy objects with both the resolved and unresolved address scenarios : Proxy constructor with unresolved SocketAddress : *** New Proxy constructor with type = HTTP and SocketAddress of:10.6.160.94:8080 java.lang.Throwable at java.net.Proxy.<init>(Proxy.java:97) at sun.net.spi.DefaultProxySelector$2.run(DefaultProxySelector.java:308) at sun.net.spi.DefaultProxySelector$2.run(DefaultProxySelector.java:200) at java.security.AccessController.doPrivileged(Native Method) at sun.net.spi.DefaultProxySelector.select(DefaultProxySelector.java:199) at sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:926) at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:177) at sun.net.www.protocol.https.HttpsURLConnectionImpl.connect(HttpsURLConnectionImpl.java:153) Proxy constructor with resolved SocketAddress : *** New Proxy constructor with type = HTTP and SocketAddress of:/10.6.160.94:8080 java.lang.Throwable at java.net.Proxy.<init>(Proxy.java:97) at sun.net.www.protocol.https.HttpsClient.newHttpProxy(HttpsClient.java:208) at sun.net.www.protocol.https.HttpsClient.New(HttpsClient.java:326) at sun.net.www.protocol.https.HttpsClient.New(HttpsClient.java:317) at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.proxiedConnect(AbstractDelegateHttpsURLConnection.java:149) at sun.net.www.protocol.http.HttpURLConnection.doTunneling(HttpURLConnection.java:1782) at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:183) at sun.net.www.protocol.https.HttpsURLConnectionImpl.connect(HttpsURLConnectionImpl.java:153) HttpsClient creates Proxy with resolved SocketAddress : 200 try { 201 saddr = java.security.AccessController.doPrivileged(new 202 java.security.PrivilegedExceptionAction<InetSocketAddress>() { 203 public InetSocketAddress run() { 204 return new InetSocketAddress(phost, pport); 205 }}); 206 } catch (java.security.PrivilegedActionException pae) { 207 } 208 return new Proxy(Proxy.Type.HTTP, saddr); 209 } HttpClient does not : 216 if (proxyHost == null || proto == null) 217 return Proxy.NO_PROXY; 218 int pport = proxyPort < 0 ? getDefaultPort(proto) : proxyPort; 219 InetSocketAddress saddr = InetSocketAddress.createUnresolved(proxyHost, pport); 220 return new Proxy(Proxy.Type.HTTP, saddr); For fix, I'm proposing we simply let HttpsClient.newHttpProxy call into the HttpClient.newHttpProxy method. An alternative would be for us to start comparing IP address of Proxy in Http(s)Client classes, but keeping Proxy constructors the same via use of unresolved addresses should mean we're comparing like with like. (and keep alive cache works as a result)
22-01-2015

behavioural changes seen around the sun.net.www.protocol.http.HttpURLConnection writeRequests and sun.net.www.protocol.http.HttpURLConnection getInputStream methods and how the keep alive parameter gets passed. 7u25 (post initial SSL connection) Nov 20, 2014 6:51:51 PM sun.net.www.protocol.http.HttpURLConnection writeRequests FINE: sun.net.www.MessageHeader@375c56d37 pairs: {GET /qoms151/rfqtunnel HTTP/1.1: null}{Cache-Control: no-cache}{accept: *}{Pragma: no-cache}{User-Agent: Java/1.7.0_25}{Host: qt.fxall.com}{Proxy-Connection: keep-alive} Nov 20, 2014 6:51:51 PM sun.net.www.protocol.http.HttpURLConnection getInputStream FINE: sun.net.www.MessageHeader@f48530a5 pairs: {null: HTTP/1.1 200 OK}{Date: Thu, 20 Nov 2014 23:51:51 GMT}{Server: Jetty(8.1.8.v20121106)}{Content-Length: 0}{Content-Type: text/plain; charset=utf-8} Nov 20, 2014 6:51:51 PM HTTPKeepAliveTester main 7u72 (post initial SSL connection) Nov 20, 2014 6:48:47 PM sun.net.www.protocol.http.HttpURLConnection writeRequests FINE: sun.net.www.MessageHeader@1f074cb7 pairs: {GET /qoms151/rfqtunnel HTTP/1.1: null}{Cache-Control: no-cache}{accept: *}{Pragma: no-cache}{User-Agent: Java/1.7.0_72}{Host: qt.fxall.com}{Connection: keep-alive} Nov 20, 2014 6:48:48 PM sun.net.www.protocol.http.HttpURLConnection getInputStream FINE: sun.net.www.MessageHeader@11907607 pairs: {null: HTTP/1.1 200 OK}{Date: Thu, 20 Nov 2014 23:48:48 GMT}{Server: Jetty(8.1.8.v20121106)}{Content-Length: 0}{Keep-Alive: timeout=45}{Connection: Keep-Alive}{Content-Type: text/plain; charset=utf-8} Nov 20, 2014 6:48:48 PM HTTPKeepAliveTester main Proxy-Connection gets set in 7u25 while 7u72 has Connection. Investigating whether JDK-8009251 is a factor here.
26-11-2014