JDK-8240275 : Occasional errors in HttpURLConnection due to race with GC
  • Type: Bug
  • Component: core-libs
  • Sub-Component: java.net
  • Affected Version: 8,11,15
  • Priority: P4
  • Status: Resolved
  • Resolution: Fixed
  • Submitted: 2020-02-27
  • Updated: 2022-12-12
  • Resolved: 2022-12-07
The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.
JDK 20
20 b27Fixed
Related Reports
Relates :  
Relates :  
Description
ADDITIONAL SYSTEM INFORMATION :
Reproduced on:
RHEL6, java-1.8.0-openjdk-1.8.0.232.b09-1.el6_10.x86_64
RHEL7, java-1.8.0-openjdk-1.8.0.242.b08-0.el7_7.x86_64
Ubuntu 18.04, OpenJDK 11.0.6+10-1ubuntu1~18.04.1
Ubuntu 18.04, OpenJDK from master branch of https://github.com/openjdk/jdk
RHEL7, IBM java 1.8.0_221

A DESCRIPTION OF THE PROBLEM :
When using HttpURLConnection or HttpsURLConnection, the underlying HttpClient object is normally added to the KeepAliveCache for reuse when the response body has been fully read.  However, if the response body is never fully read (for example, if a 201 response code is returned by the server and the code that is using HttpURLConnection reads the "Location" header but doesn't bother reading the response body), then the underlying HttpClient object is added to the KeepAliveCache by the MeteredStream/KeepAliveStream finalizer when the HttpURLConnection/HttpClient/InputStream/etc is garbage collected.

Unfortunately, "Resurrecting" the HttpClient object from the MeteredStream/KeepAliveStream finalizer like this is unreliable and leads to intermittent failures in subsequent HttpURLConnection calls.  For example, GC may identify that both the MeteredStream and Socket underlying an HttpURLConnection are ready to be finalized, and may finalize the MeteredStream before the Socket causing the HttpClient object to be "resurrected" and added to the KeepAliveCache.  If a new HttpURLConnection is created (in another thread that is running in parallel with GC) after this happens and before GC gets around to finalizing the Socket, the new HttpURLConnection may reuse this HttpClient.  If GC then gets around to finalizing the Socket while the new HttpURLConnection is actively using it, the Socket finalizer will close the underlying connection while it is in use and cause an error.  If sun.net.http.retryPost is set to "false" then this error may propagate up to user-level code.  Alternatively, if the above condition is combined with conditions that trigger the bug described in https://bugs.openjdk.java.net/browse/JDK-8209178 then such errors can lead to hung connections after the HttpClient automatically retries the interrupted connection.

I've posted a proposed fix for this at https://github.com/PaulSD/jdk/commit/9a3fb5d8f6ec31bfcdf8a25588383d46014811fa
This fix works by adding each HttpClient object to a new data structure in KeepAliveCache immediately after it is created, then using a WeakReference/ReferenceQueue on the HttpURLConnection/HttpsURLConnection to detect when the associated HttpURLConnection/HttpsURLConnection is garbage collected so the KeepAliveCache can either drop the HttpClient reference or move it to the "idle and available for reuse" data structure as appropriate.  This eliminates the current "object resurrection" behavior and prevents GC from finalizing the HttpClient or any underlying objects while they are still in use.

STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
Download reproduction script/code from: https://github.com/PaulSD/jdk/tree/test
./test

EXPECTED VERSUS ACTUAL BEHAVIOR :
EXPECTED -
Garbage collector did not collect in expected order within 3 tries, giving up
ACTUAL -
Erroneous behavior has been successfully reproduced: Underlying Socket is being closed after request and before response in main thread

---------- BEGIN SOURCE ----------
https://github.com/PaulSD/jdk/blob/test/Test.java
---------- END SOURCE ----------

CUSTOMER SUBMITTED WORKAROUND :
Ensure that all code that uses HttpURLConnection or HttpsURLConnection always calls `myhttpurlconnection.getInputStream().close();` before abandoning the HttpURLConnection/HttpsURLConnection object.

FREQUENCY : occasionally



Comments
Changeset: 27bbe7be Author: Daniel JeliƄski <djelinski@openjdk.org> Date: 2022-12-07 08:56:10 +0000 URL: https://git.openjdk.org/jdk/commit/27bbe7be2c43a22e8cf55aa403d8018346ae3e37
07-12-2022

A pull request was submitted for review. URL: https://git.openjdk.org/jdk/pull/11474 Date: 2022-12-02 08:31:25 +0000
02-12-2022

The observations on both Oracle Linux and Windows 10: JDK 8: Fail. JDK 11: Fail. JDK15: Fail. ILW=MML=P4
02-03-2020