United StatesChange Country, Oracle Worldwide Web Sites Communities I am a... I want to...
JDK-6358532 : HttpURLConnection.disconnect doesn???t really do the job

Details
Type:
Bug
Submit Date:
2005-12-02
Status:
Resolved
Updated Date:
2011-12-20
Project Name:
JDK
Resolved Date:
2006-06-21
Component:
core-libs
OS:
solaris_9
Sub-Component:
java.net
CPU:
sparc
Priority:
P4
Resolution:
Fixed
Affected Versions:
1.3.1
Fixed Versions:

Related Reports
Backport:
Backport:
Backport:

Sub Tasks

Description
Customer(ISV) have a new version of our product that is currently targeted for release in
2006 that will be using 1.5.  One of the challenges that customer face is that the exis-
ting product, using 1.3 will continue to be in use at the end-user sites for some
time.  Customer is requesting a fix in 1.5 and 1.3 if at all possible. 

Customer has provided suggested fix. 

Description 

In our server side application, we issue number of outgoing HTTP/S requests to 
external systems beyond our control. Some of these systems are sluggish in their 
response than others, which affects the responsiveness of our application.  We 
would like to be able to abandon the HTTP/S requests issued to these systems if 
they fail to respond within a certain time limit (that is configurable on our side).
If we let the ???main??? thread servicing the user in our application issue the
 HTTP request to the external system, we can???t respond to the user request until
 the external system returns the response or an error, since the thread tends to
 get stuck in socket.read.  So our strategy has been to spin a separate thread (
which can be a new thread or a worker thread from a pool) to issue the HTTP request
and have the ???main??? thread wait for a specified time. If the response is not
received within the specified time limit, the main thread abandons the HTTP request
and returns a response to the user. However, this approach doesn???t scale well
for two reasons. 
1. For each outgoing HTTP request, we must spin a new thread. This quickly cause
s a thread pile up under load. If we use a bounded (in size) thread pool we simply 
pile the number of outstanding requests. 
2. The thread that is spun for the HTTP request continues waiting on socket IO 
socket.read even though no one cares about the response after the time limit. 
So, we would like a deterministic way to ???interrupt??? the thread waiting on socket
 IO in an HttpURLConnection. Thread.interrupt() is not affective as designed to
 interrupt an IO. So we decided to try invoking the HttpURLConnection.disconnect
() method from another thread to ???interrupt??? the thread that is stuck on the Socket
IO. It turns out that it does close the socket causing the thread to return 
with an IOException. However even though the sun.net.www.http.HttpClient.closeSe
rver() method invoked from HttpURLConnection.disconnect() does successfully close
the connection, the thread is usually stuck in the sun.net.www.http.HttpClient
.parseHTTP() method which catches the exception and re-opens the connection to the
server after one failure. 
The problem is that the HttpClient doesn???t distinguish between the ???client-initiatd"
socket close versus a socket close by the peer. The ???retry-once??? semantics
works well for socket resets by the peer, but does not work when the client its
elf wants to abandon the request. Since the HttpURLConnection.disconnect()implem-
entation sets the reference to the HttpClient to null after invoking closeServer
() there is no way to close the socket again.  I have considered the following 
fix (context diff follows). 

*** c:/jdk1.3.1_13/src/share/classes/sun/net/www/http/HttpClient.java Wed Nov 16
 12:31:27 
2005 
--- c:/eclipse/workspace/HttpClientTest/src/sun/net/www/http/HttpClient.java Wed
 Nov 16 
12:50:36 2005 
*************** 
*** 32,38 **** 
PosterOutputStream poster = null; 
// if we've had one io error 
! boolean failedOnce = false; 
/** regexp pool of hosts for which we should connect directly, not Proxy 
* these are intialized from a property. 
--- 32,38 ---- 
PosterOutputStream poster = null; 
// if we've had one io error 
! volatile boolean failedOnce = false; 
/** regexp pool of hosts for which we should connect directly, not Proxy 
* these are intialized from a property. 
*************** 
*** 815,820 **** 
--- 815,828 ---- 
} catch (Exception e) {} 
} 
+ /* Use to abandon the HttpRequest */ 
+ public void abandon() { 
+ try { 
+ failedOnce = true; 
+ closeServer(); 
+ } catch (Exception e) {} 
+ } 
+ 
/** 
* @return the proxy host being used for this client, or null 
* if we're not going through a proxy 
The changes are indicated using ! and + in the first field. The rest of the lines
are for context. 
1. I made the field ???failedOnce??? volatile to allow changes by one thread (the 
interrupter) to be visible to another thread (the interrupted) without excessive
synchronization. 
2. I added another method called abandon() that sets failedOnce = true before in
voking 
closeServer(). I couldn???t add ???failedOnce??? in closeServer() itself because it would
prevent the ???retry-once??? semantics. 
*** 
c:/jdk1.3.1_13/src/share/classes/sun/net/www/protocol/http/HttpURLConnection.java
 
Wed Nov 16 12:31:28 2005 
--- 
c:/eclipse/workspace/HttpClientTest/src/sun/net/www/protocol/http/HttpURLConnect
ion 
.java Wed Nov 16 12:49:47 2005 
*************** 
*** 858,864 **** 
ProgressData.pdata.unregister(pe); 
} 
if (http != null) { 
! http.closeServer(); 
// poster = null; 
http = null; 
connected = false; 
--- 858,864 ---- 
ProgressData.pdata.unregister(pe); 
} 
if (http != null) { 
! http.abandon(); 
// poster = null; 
http = null; 
connected = false; 

In the HttpURLConnection class, I simply call ???abandon??? instead of ???closeServer???
to distinguish between ???client initiated??? close (semantically equivalent to 
abandon) and a error triggered close of the socket that ???closeServer??? serves today.
JSSE Extension While I was able to fix this for the JDK classes, we do use JSSE
for HTTPS connections and run in to the same behavior there. However, the Http
sURLConnection and HttpsClient classes in JSSE do not have any inheritance relat-
ionship with the JDK classes and have the same issue. Since we don???t have the code
for JSSE, I???m not able to provide a code diff. I suspect the fix would be similar.

                                    

Comments
EVALUATION

The issue is that the programmer is using HttpURLConnection.disconnect to asynchronously terminate a HttpURLConnection that is in the process of reading or waiting on the response headers. Currently we throw a NullPointerException.

There is a window where we are waiting on the response headers where if an IOException is thrown (in our case from the async disconnect) we will resend the request. This is clearly not correct. The fix is to simply not resend the request in this situation.
                                     
2006-06-09
EVALUATION

This bug is not as applicable to tiger or mustang as you URLConnection.setReadTimeout() to specify a timeout for when reading. Nevertheless if you call disconnect asynchronously (even with tiger or mustang) you could possibly run into this issue. Therefore it may cause problems for existing pre tiger apps when upgrading to a later jdk.
                                     
2006-06-12



Hardware and Software, Engineered to Work Together