JDK-8212926 : HttpClient does not retrieve files with large sizes over HTTP/1.1
  • Type: Bug
  • Component: core-libs
  • Sub-Component: java.net
  • Affected Version: 11,11.0.1
  • Priority: P4
  • Status: Closed
  • Resolution: Fixed
  • Submitted: 2018-10-24
  • Updated: 2018-11-21
  • Resolved: 2018-10-25
The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.
JDK 11
11.0.2Fixed
Description
ADDITIONAL SYSTEM INFORMATION :
$ java -version
java version "11.0.1" 2018-10-16 LTS
Java(TM) SE Runtime Environment 18.9 (build 11.0.1+13-LTS)
Java HotSpot(TM) 64-Bit Server VM 18.9 (build 11.0.1+13-LTS, mixed mode)

$ [System.Environment]::OSVersion.Version
Major  Minor  Build  Revision
-----  -----  -----  --------
10     0      17134  0

A DESCRIPTION OF THE PROBLEM :
When using the new HttpClient it dosen't download files whose filesize in byte is reported bigger than Integer.MAX_VALUE most of the time if using HTTP 1.1.

1) A file with size of 2147483648 (= Integer.MAX_VALUE + 1) reported in the content-length header just dosen't invoke any methode besides the onSubscribe methode of a HttpResponse.BodySubscriber.

2) Interestingly a file with a size of 4294967294 (= Integer.MAX_VALUE - Integer.MIN_VALUE  - 1) reported in the content-length header downloads normally.

3) However a file with a size of 4294967296 (= Integer.MAX_VALUE - Integer.MIN_VALUE + 1) downloads a file with the size of exactly 0 bytes.

For some reson using HTTP 2 works for at least the first case and downloads that file correctly, I don't have the resources to easily test the other cases using HTTP 2.

STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
1. Create Http Client that will use HTTP 1.1 (for the example file I provided this is important as cloudflare will server up HTTP 2 if we don't specificly request HTTP 1.1)
2. Create a request to download a file with one of the specified sizes (for example: https://map.usbcraft.net/file/usbpc-downloads/java_http_client.fail (content-length = 2147483648))
3. Try to download from that url into a file with a HttpResponse.BodyHandlers.ofFile()

EXPECTED VERSUS ACTUAL BEHAVIOR :
EXPECTED -
It should download the file normally into the specified file on the hard drive.
ACTUAL -
The file on the Hard drive is created, but dosen't get any content and remains at 0 bytes size. 
Looking in the Task Manager shows that java is downloading with full speed, but I don't know what is happening with that data.

For some reason it also sometimes happend that the CPU usage shot up from ~5% to ~30% and network usage stopped.

---------- BEGIN SOURCE ----------
import java.io.IOException;
import java.net.URI;
import java.net.http.HttpClient;
import java.net.http.HttpRequest;
import java.net.http.HttpResponse;
import java.nio.file.Path;

public class Main {
    public static void main(String[] args) throws IOException, InterruptedException {
        HttpClient client = HttpClient.newBuilder()
                .version(HttpClient.Version.HTTP_1_1)
                .build();

        HttpRequest request = HttpRequest.newBuilder()
                .uri(URI.create("https://map.usbcraft.net/file/usbpc-downloads/java_http_client.fail"))
                .build();

        client.send(request, HttpResponse.BodyHandlers.ofFile(Path.of("./test.bin")));
    }
}
---------- END SOURCE ----------

FREQUENCY : always



Comments
Fix Request We would like to backport this to 11 as it is basic functionality to support downloading of files > 4Gb using a fixed content length. The fix is very simple and was caused by the use `int` where `long` should have been used for HTTP/1.1 content-lengths. A regression test is included. The fix is pushed to jdk/jdk and the patch applies cleanly to jdk11u.
25-10-2018