JDK-8050983 : Misplaced parentheses in sun.net.www.http.HttpClient break HTTP PUT streaming
  • Type: Bug
  • Component: core-libs
  • Sub-Component: java.net
  • Affected Version: 7u60,9
  • Priority: P4
  • Status: Resolved
  • Resolution: Fixed
  • OS: windows_7
  • CPU: x86_64
  • Submitted: 2014-07-17
  • Updated: 2016-05-27
  • Resolved: 2014-11-14
The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.
JDK 7 JDK 8 JDK 9
7u80Fixed 8u40Fixed 9 b42Fixed
Related Reports
Duplicate :  
Relates :  
Relates :  
Description
FULL PRODUCT VERSION :
java version "1.7.0_60"
Java(TM) SE Runtime Environment (build 1.7.0_60-b19)
Java HotSpot(TM) 64-Bit Server VM (build 24.60-b09, mixed mode)

ADDITIONAL OS VERSION INFORMATION :
Microsoft Windows [Version 6.1.7601]
Linux x.example.com 3.14.8-100.fc19.x86_64 #1 SMP Mon Jun 16 21:53:59 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux

A DESCRIPTION OF THE PROBLEM :
Parentheses are misplaced in sun.net.www.http.HttpClient.parseHTTP() and parseHTTPHeader().  When a streaming PUT request fails with an IOException, the PUT request is retried with an empty body.  Like streaming POST, streaming PUT requests must not be retried.

Current jre code:
        } catch (IOException e) {
            closeServer();
            cachedHttpClient = false;
            if (!failedOnce && requests != null) {
                failedOnce = true;
                if (getRequestMethod().equals("CONNECT") ||
                    (httpuc.getRequestMethod().equals("POST") &&
                    (!retryPostProp || streaming))) {
                    // do not retry the request
                }  else {
                    // try once more
                    openServer();
                    if (needsTunneling()) {
                        httpuc.doTunneling();
                    }
                    afterConnect();
                    writeRequests(requests, poster);
                    return parseHTTP(responses, pi, httpuc);
                }
            }

Incorrect parentheses:
                if (getRequestMethod().equals("CONNECT") ||
                    (httpuc.getRequestMethod().equals("POST") &&
                    (!retryPostProp || streaming))) {

Corrected parentheses:
                if (getRequestMethod().equals("CONNECT") ||
                    (httpuc.getRequestMethod().equals("POST") &&
                        !retryPostProp) ||
                    streaming) {

Note that this is also the cause of bug 6944020.

STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
Send a large streaming PUT, e.g. 5MB.  Repeat many times over a typical internet connection where errors occasionally occur.  Occasionally an IOException occurs and an empty body is PUT on the retry.

EXPECTED VERSUS ACTUAL BEHAVIOR :
EXPECTED -
Streaming PUT should throw an IOException and not retry with an empty body.
ACTUAL -
An empty body is sent with the PUT instead of the correct content.

REPRODUCIBILITY :
This bug can be reproduced always.

---------- BEGIN SOURCE ----------
import java.io.BufferedReader;
import java.io.BufferedWriter;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.io.IOException;
import java.io.OutputStream;
import java.io.OutputStreamWriter;
import java.net.HttpURLConnection;
import java.net.MalformedURLException;
import java.net.ServerSocket;
import java.net.Socket;
import java.net.SocketTimeoutException;
import java.net.URL;

public class Put {
    // changing the method or the streaming avoids the issue
    static final String METHOD = "PUT";
    static final boolean STREAMING = true;

    public static void main(String[] args) {
        try {
            int port = 8000;
            String path = "/testput";
            URL url = new URL("http://localhost:" + port + path);
            byte[] buf = new byte[5000];

            new MyServer(port).start();

            for (int i = 0; i < 3; i++) {
                try {
                    HttpURLConnection conn = (HttpURLConnection)
                        url.openConnection();

                    conn.setRequestMethod(METHOD);
                    conn.setRequestProperty("Content-Type",
                                            "application/octet-stream");
                    conn.setConnectTimeout(2000);
                    conn.setReadTimeout(2000);
                    if (STREAMING) {
                        conn.setFixedLengthStreamingMode(buf.length);
                    }
                    conn.setDoOutput(true);
                    conn.getOutputStream().write(buf);

                    int code = conn.getResponseCode();
                    System.out.println("Client received response: "
                                       + code + "\n");
                } catch (IOException e) {
                    e.printStackTrace();
                }
            }
        } catch (MalformedURLException e) {
        }
    }

    public static void readFully(BufferedReader reader) throws IOException {
        String line;
        int contentLength = -1;
        int bytesRead = 0;

        do {
            line = reader.readLine();
            if (line == null) {
                throw new IOException("read failed");
            }

            System.out.println(line); 
            if (line.toLowerCase().startsWith("content-length: ")) {
                contentLength = Integer.parseInt(line.substring(16));
            }
        } while (line.length() > 0);

        if (contentLength > 0) {
            char[] buffer = new char[contentLength];

            while (bytesRead < contentLength) {
                int remaining = contentLength - bytesRead;
                int n = 0;

                try {
                    n = reader.read(buffer, bytesRead, remaining);
                } catch (SocketTimeoutException e) {
                    System.out.println("Server read timeout");
                }

                if (n > 0) {
                    bytesRead += n;
                } else {
                    break;
                }
            }
        }

        System.out.println("Server received " + bytesRead + " bytes\n");
    }

    static class MyServer extends Thread {
        private int port;

        MyServer(int port) {
            this.port = port;
            this.setDaemon(true);
        }

        public void run() {
            try {
                ServerSocket serverSocket = new ServerSocket(port);

                while (true) {
                    Socket clientSocket = serverSocket.accept();
                    
                    clientSocket.setSoTimeout(1000);
                    new ServerThread(clientSocket).start();
                }
            } catch (IOException e) {
                e.printStackTrace();
            }
        }
    }

    static class ServerThread extends Thread {
        private Socket socket;

        ServerThread(Socket socket) {
            this.socket = socket;
        }

        public void run() {
            try {
                InputStream in = socket.getInputStream();
                BufferedReader reader = new BufferedReader(
                    new InputStreamReader(in));
                OutputStream out = socket.getOutputStream();
                BufferedWriter writer = new BufferedWriter(
                    new OutputStreamWriter(out));
                String response = "HTTP/1.1 200 OK\nContent-Length: 0";

                readFully(reader);
                writer.write(response);
                writer.close();
            } catch (IOException e) {
                e.printStackTrace();
            }
        }
    }
}

---------- END SOURCE ----------

CUSTOMER SUBMITTED WORKAROUND :
Apache HttpClient is an alternative to java.net.HttpURLConnection.


Comments
The description is correct. The solution is to no retry in any case of a streaming body ( which is not buffered ). JDK-6672144 introduced the original change, but it was mistakenly limited to POST only requests.
07-11-2014