JDK-8209178 : Proxied HttpsURLConnection doesn't send BODY when retrying POST request
  • Type: Bug
  • Component: core-libs
  • Sub-Component: java.net
  • Affected Version: 8,11,12
  • Priority: P4
  • Status: Resolved
  • Resolution: Fixed
  • OS: windows_7
  • CPU: x86_64
  • Submitted: 2018-08-03
  • Updated: 2022-12-16
  • Resolved: 2019-10-01
The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.
Other JDK 11 JDK 13 JDK 14 JDK 8
,shenandoah8u332Resolved 11.0.18-oracleFixed 13.0.2Fixed 14 b17Fixed 8u321Fixed
Related Reports
Relates :  
Relates :  
Description
ADDITIONAL SYSTEM INFORMATION :
Reproduced on Windows 7
but also occurs on linux servers

Reproduced with : 
- Oracle JDK 1.8.0_172
- OpenJDK 10.0.2

A DESCRIPTION OF THE PROBLEM :
As described here : https://bugs.java.com/bugdatabase/view_bug.do?bug_id=6382788, the JDK is able to retry http/https requests when the server doesn't respond with a correct HTTP response.

When it occurs on a POST request : the body of the POST is lost during the retry process. Only the request headers are sent, leaving the remote server waiting for the body.

This bug is linked to this former bug, when it was the request header that was lost :
https://bugs.java.com/bugdatabase/view_bug.do?bug_id=8025710

Please note that the JDK handles this kind of retries in a least two places : 
- sun.net.www.http.HttpClient#parseHTTP
- sun.net.www.http.HttpClient#parseHTTPHeader


In my case, the cause of the retry is a very specific timeout behaviour on a cached TLS connection :
I described it in another bug report : internal review ID : 9056455

STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
The testcase I described in the bug report "internal review ID : 9056455" is able to reproduced the behaviour reliably. 

But I guess it is possible to simulate the problem more easily 
( I don't know if the problem is specific to HTTPS, but ssl debug are valuable to observe the problem )
- set up an HTTPS server responding to POST requests but which close abrutly the connection without any response one out of two times
- Set up a forward proxy usable for calling the previous server
- make a HTTPS POST call (with a body) with HttpsURLConnection to the previous server through the proxy (at the moment where the request will first be closed abrutly by the server)

EXPECTED VERSUS ACTUAL BEHAVIOR :
EXPECTED -
Transparent retry of the POST will work and the server will correctly received the body on the second try

JDK logs on sun.net.www. and ssl debug logs shows that two TLS connection was made  and that the body was sent in each case.
ACTUAL -
The JDK tries to make a transparent retry of the POST but only send the headers and not the body on this retry :  
The server wait for the body (as the requet contains a non-null Content-Length) and the client call eventually fail with a read timeout

JDK logs on sun.net.www. and ssl debug logs shows that two TLS connection was made but that the body was not sent on the second call


During the retry we have : 
(...)
main, WRITE: TLSv1.2 Alert, length = 26
[Raw write]: length = 31
0000: 15 03 03 00 1A 00 00 00   00 00 00 00 05 AE EA C8  ................
0010: 41 BB E8 1A 3A 64 08 8D   57 4A 3D 63 0A 34 A9     A...:d..WJ=c.4.
main, called closeSocket(false)
main, called close()
main, called closeInternal(true)
août 03, 2018 7:49:47 PM sun.net.www.protocol.http.HttpURLConnection sendCONNECTRequest
PRÉCIS: sun.net.www.MessageHeader@14fc5f045 pairs: {CONNECT xxxxxxxx.com:443 HTTP/1.1: null}{User-Agent: Java/1.8.0_172}{Host: api.orange.com}{Accept: text/html, image/gif, image/jpeg, *; q=.2, */*; q=.2}{Proxy-Connection: keep-alive}
août 03, 2018 7:49:47 PM sun.net.www.protocol.http.HttpURLConnection doTunneling
PRÉCIS: sun.net.www.MessageHeader@6e2829c71 pairs: {null: HTTP/1.1 200 Connection established}
Allow unsafe renegotiation: false
Allow legacy hello messages: true
(...)

Here the stack from the timeout : 
(...)
Padded plaintext before ENCRYPTION:  len = 399
0000: 50 4F 53 54 20 2F 6F 70   65 6E 69 64 63 6F 6E 6E  POST /openidconn
0010: 65 63 74 2F 66 72 2F 76   31 2F 74 6F 6B 65 6E 20  ect/fr/v1/token 
0020: 48 54 54 50 2F 31 2E 31   0D 0A 43 6F 6E 74 65 6E  HTTP/1.1..Conten
0030: 74 2D 54 79 70 65 3A 20   61 70 70 6C 69 63 61 74  t-Type: applicat
0040: 69 6F 6E 2F 78 2D 77 77   77 2D 66 6F 72 6D 2D 75  ion/x-www-form-u
0050: 72 6C 65 6E 63 6F 64 65   64 0D 0A 63 68 61 72 73  rlencoded..chars
0060: 65 74 3A 20 75 74 66 2D   38 0D 0A 41 75 74 68 6F  et: utf-8..Autho
0070: 72 69 7A 61 74 69 6F 6E   3A 20 42 61 73 69 63 20  rization: Basic 
   (deleted)
00C0: 53 77 3D 3D 0D 0A 43 61   63 68 65 2D 43 6F 6E 74 xxxx..Cache-Cont
00D0: 72 6F 6C 3A 20 6E 6F 2D   63 61 63 68 65 0D 0A 50  rol: no-cache..P
00E0: 72 61 67 6D 61 3A 20 6E   6F 2D 63 61 63 68 65 0D  ragma: no-cache.
00F0: 0A 55 73 65 72 2D 41 67   65 6E 74 3A 20 4A 61 76  .User-Agent: Jav
0100: 61 2F 31 2E 38 2E 30 5F   31 37 32 0D 0A 48 6F 73  a/1.8.0_172..Hos
0110: 74 3A    (deleted)                                               63 6F  t: api.xxxxxx.co
0120: 6D 0D 0A 41 63 63 65 70   74 3A 20 74 65 78 74 2F  m..Accept: text/
0130: 68 74 6D 6C 2C 20 69 6D   61 67 65 2F 67 69 66 2C  html, image/gif,
0140: 20 69 6D 61 67 65 2F 6A   70 65 67 2C 20 2A 3B 20   image/jpeg, *; 
0150: 71 3D 2E 32 2C 20 2A 2F   2A 3B 20 71 3D 2E 32 0D  q=.2, */*; q=.2.
0160: 0A 43 6F 6E 6E 65 63 74   69 6F 6E 3A 20 6B 65 65  .Connection: kee
0170: 70 2D 61 6C 69 76 65 0D   0A 43 6F 6E 74 65 6E 74  p-alive..Content
0180: 2D 4C 65 6E 67 74 68 3A   20 36 35 0D 0A 0D 0A     -Length: 65....
main, WRITE: TLSv1.2 Application Data, length = 423
[Raw write]: length = 428
0000: 17 03 03 01 A7 00 00 00   00 00 00 00 01 54 A3 9B  .............T..
0010: 86 64 95 19 C0 EA 32 A4   6E DA 58 DA C8 5C 3C 35  .d....2.n.X..\<5
0020: D3 4E FC D8 D4 60 9E 9F   A7 BC B4 10 74 9F A4 12  .N...`......t...
0030: 10 EF 3F C4 C9 5C 3B 82   A1 AA E4 FC D7 11 72 59  ..?..\;.......rY
0040: 35 91 17 5C D5 A9 6A FC   88 DA C7 18 51 F9 82 0A  5..\..j.....Q...
0050: 50 26 EC 57 E0 B1 6B 4B   BA FB DC 25 CA 30 35 8C  P&.W..kK...%.05.
0060: FC 5D 4B 6B 62 7A 74 86   20 97 D6 74 7D 1A DC 17  .]Kkbzt. ..t....
0070: A9 F2 DE B7 3D 3C 07 93   A8 8F CB 85 03 2F 5F 3C  ....=<......./_<
0080: 31 1A 74 82 9E D0 52 81   2E FB 90 AD 36 B1 DC 66  1.t...R.....6..f
0090: EA 4E 83 B5 B8 93 2F D6   4E A1 42 81 40 65 72 BD  .N..../.N.B.@er.
00A0: 5B 30 27 CB 3B C8 33 9A   00 84 A5 68 A6 13 32 C3  [0'.;.3....h..2.
00B0: 21 F5 72 2B EB B2 B2 CB   5D 17 DD 08 49 8A C1 FD  !.r+....]...I...
00C0: 0E 09 70 D0 26 F8 8C 9A   45 CF 55 99 7A 92 F3 A3  ..p.&...E.U.z...
00D0: DB CC E9 80 F8 A6 76 46   C8 DA 33 A9 3E A3 6B A9  ......vF..3.>.k.
00E0: 65 18 BA D9 DD 25 BC AE   21 F2 2B 6A 9F CE 68 45  e....%..!.+j..hE
00F0: 61 D1 04 6B 77 10 BE 40   55 31 7D D1 6E E1 02 E2  a..kw..@U1..n...
0100: EF 61 61 02 FF 8D A3 CB   72 E1 18 98 0F A9 81 57  .aa.....r......W
0110: 7F 61 AC 5B A1 71 BC B0   D8 D7 39 78 D4 AF 98 C1  .a.[.q....9x....
0120: 39 CE D0 7B FE 75 E6 6F   3C 64 E1 13 19 89 01 1E  9....u.o<d......
0130: 08 B8 A9 E7 79 54 EA BB   32 88 EB 2D 9E B3 3E 4D  ....yT..2..-..>M
0140: FA 2C 0B 00 84 7D 3C 04   6A CB 91 4E 43 C3 A5 83  .,....<.j..NC...
0150: 81 8F D6 AC 45 6F BB 53   91 A7 D5 AF 80 FB 65 0E  ....Eo.S......e.
0160: 5C 4D DB BF 95 FA C4 98   9F 95 6B 4C D6 3E 8A E1  \M........kL.>..
0170: F5 72 23 FC 15 36 39 E1   DF EF A2 7C E5 58 71 80  .r#..69......Xq.
0180: 84 26 39 93 C9 5B A7 FF   80 9F 4B 64 3C A0 60 48  .&9..[....Kd<.`H
0190: 8D 4C 29 A1 9B 33 29 7F   78 58 B0 46 A3 2D 47 DC  .L)..3).xX.F.-G.
01A0: 4A F7 74 4C C5 28 CD 5A   F7 3C 96 4F              J.tL.(.Z.<.O
main, handling exception: java.net.SocketTimeoutException: Read timed out
main, called close()
main, called closeInternal(true)
main, SEND TLSv1.2 ALERT:  warning, description = close_notify
Padded plaintext before ENCRYPTION:  len = 2
0000: 01 00                                              ..
main, WRITE: TLSv1.2 Alert, length = 26
[Raw write]: length = 31
0000: 15 03 03 00 1A 00 00 00   00 00 00 00 02 8A 6C 98  ..............l.
0010: 8F 35 34 06 0F 52 77 F6   08 9B 36 DF 46 B4 F3     .54..Rw...6.F..
main, called closeSocket(true)
main, called close()
main, called closeInternal(true)

java.net.SocketTimeoutException: Read timed out

	at java.net.SocketInputStream.socketRead0(Native Method)
	at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
	at java.net.SocketInputStream.read(SocketInputStream.java:171)
	at java.net.SocketInputStream.read(SocketInputStream.java:141)
	at sun.security.ssl.InputRecord.readFully(InputRecord.java:465)
	at sun.security.ssl.InputRecord.read(InputRecord.java:503)
	at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:983)
	at sun.security.ssl.SSLSocketImpl.readDataRecord(SSLSocketImpl.java:940)
	at sun.security.ssl.AppInputStream.read(AppInputStream.java:105)
	at java.io.BufferedInputStream.fill(BufferedInputStream.java:246)
	at java.io.BufferedInputStream.read1(BufferedInputStream.java:286)
	at java.io.BufferedInputStream.read(BufferedInputStream.java:345)
	at sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:735)
	at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:678)
	at sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:848)
	at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:678)
	at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1587)
	at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1492)
	at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:480)
	at sun.net.www.protocol.https.HttpsURLConnectionImpl.getResponseCode(HttpsURLConnectionImpl.java:347)
	at TestCase.testCaseMethod

---------- BEGIN SOURCE ----------
Here the test case from bug report "internal review ID : 9056455"

It make two request, but depending on the test setup, one request may be sufficient to reproduce (cf Steps to Reproduce) 

package testcase;

import org.apache.commons.io.IOUtils;
import org.junit.Test;

import javax.net.ssl.HttpsURLConnection;
import java.io.InputStream;
import java.io.OutputStream;
import java.net.InetSocketAddress;
import java.net.Proxy;
import java.net.SocketAddress;
import java.net.URL;
import java.nio.charset.StandardCharsets;

public class TestCaseHttpsURLConnection {

	@Test
	public void testCaseMethod() throws Exception {

		URL url = new URL( "https://some.tls.server/xxx/yyy");

		SocketAddress sa = new InetSocketAddress("some.corporate.proxy",8080);
		Proxy proxy = new Proxy(Proxy.Type.HTTP,sa);

		int nb = 0;
		while(++nb <= 2) {
			try {

				System.out.println("***********  REQUEST n° " + nb + "  ******************");

				HttpsURLConnection urlConnection = (HttpsURLConnection) url.openConnection(proxy);

				urlConnection.setConnectTimeout(1000);
				urlConnection.setReadTimeout(15000);
				urlConnection.setDoInput(true);
				urlConnection.setDoOutput(true);
				urlConnection.setRequestMethod("POST");
				urlConnection.setUseCaches(false);

				urlConnection.setRequestProperty( "Content-Type", "application/x-www-form-urlencoded");
				urlConnection.setRequestProperty( "charset", "utf-8");
				urlConnection.setRequestProperty("Authorization", "Basic xxxxxxxxxx==");

				urlConnection.setRequestProperty("Connection", "keep-alive");

				String urlParameters  = "param1=a&param2=b&param3=c";
				byte[] postData       = urlParameters.getBytes( StandardCharsets.UTF_8 );

				OutputStream outputStream = urlConnection.getOutputStream();
				outputStream.write(postData);
				outputStream.close();

				int responseCode = urlConnection.getResponseCode();

				System.out.printf("UrlConnection - ResponseCode : %s%n",responseCode);

				String output;
				InputStream inputStream;
				if(responseCode < 400) {
					inputStream = urlConnection.getInputStream();
				} else {
					inputStream = urlConnection.getErrorStream();
				}
				output = IOUtils.toString(inputStream); // consumint the inputstream is also important to make the httpsclient cacheable
				inputStream.close();  // Important to make the client httpscacheable
//				urlConnection.disconnect(); // NO ...

				System.out.printf("Response body : %s%n",output);

			} finally {
				try {
					Thread.sleep(35000);
				} catch (InterruptedException e) {
					e.printStackTrace();
				}
			}
		}
	}

}

---------- END SOURCE ----------

CUSTOMER SUBMITTED WORKAROUND :
In my case : disable keepalive connection which triggered the retry condition too often
Other workaround : disable retries on POST with proprety sun.net.http.retryPost to have a fail fast behaviour (but losing retries capability)

FREQUENCY : always



Comments
Fix Request (8u) Backport for parity with Oracle 8u331. JDK11u patch does not apply cleanly, 8u patch has been reviewed and new test passes.
17-01-2022

8u code review: https://mail.openjdk.java.net/pipermail/jdk8u-dev/2022-January/014498.html
17-01-2022

Thanks Christoph Repeating that with link to review thread Fix Request (11u) Applies almost cleanly to 11u minimal testcase change were needed: one time server.getAddress().getAddress() were changed to server.getAddress().getHostName review thread link - https://mail.openjdk.java.net/pipermail/jdk-updates-dev/2019-November/002095.html
14-11-2019

Hi Vladimir, I corrected the labes for jdk11u and jdk13u fix requests. (You were using jdk11 and 13 without "u"). As for the 11u change: Can you please create a webrev and post on jdk-updates-dev, even though it's just a minor modification? Backports without review only work for copyright/whitespace changes.
11-11-2019

Fix Request (11u) Applies almost cleanly to 11u minimal testcase change were needed: one time server.getAddress().getAddress() were changed to server.getAddress().getHostName()
11-11-2019

Fix Request (13u) Applies cleanly to 13u
11-11-2019

URL: https://hg.openjdk.java.net/jdk/jdk/rev/94ca05133eb2 User: dfuchs Date: 2019-10-01 11:25:45 +0000
01-10-2019

below patch will resolve this issue. ################################# -- a/src/java.base/share/classes/sun/net/www/http/HttpClient.java Wed May 15 13:54:43 2019 +0530 +++ b/src/java.base/share/classes/sun/net/www/http/HttpClient.java Mon Jul 15 19:37:31 2019 +0530 @@ -851,8 +851,10 @@ openServer(); if (needsTunneling()) { MessageHeader origRequests = requests; + PosterOutputStream origPoster = poster; httpuc.doTunneling(); requests = origRequests; + poster = origPoster; } afterConnect(); writeRequests(requests, poster);
15-07-2019

I'm switching this from security-libs/javax.net.ssl. I believe TLS is not causing the problem and the exception is caused by http-client. The TLS read times out because it is not receiving any data and as mentioned by Chris Hagerty above, there are features along this line.
01-10-2018

Generally, non-idempotent requests should not be retried automatically. The HTTP URL protocol handler has an implementation specific system property to enable automatic retry of POST requests; can be set on the command line as follows: `-Dsun.net.http.retryPost=true`.
20-09-2018

Output using test case provided by submitter in above comment (same output in JDK 10.0.2 and JDK-11): Server is: /127.0.0.1:58523 Verifying communication with server Proxy started Verifying communication with proxy Tunnel: Waiting for client Tunnel: Client accepted Tunnel: Reading request line Tunnel: Request status line: CONNECT 127.0.0.1:58523 HTTP/1.1 Tunnel: Reading header: User-Agent: Java/10.0.2 Tunnel: Reading header: Host: 127.0.0.1:58523 Tunnel: Reading header: Accept: text/html, image/gif, image/jpeg, *; q=.2, */*; q=.2 Tunnel: Reading header: Proxy-Connection: keep-alive Tunnel: Reading header: Tunnel: Sending HTTP/1.1 200 OK Tunnel: Starting tunnel pipes Tunnel: Done - waiting for next client Tunnel: Waiting for client HTTP-Dispatcher - received request on : /foo/x HTTP-Dispatcher - received request headers : {Content-type=[application/x-www-form-urlencoded], Accept=[text/html, image/gif, image/jpeg, *; q=.2, */*; q=.2], Connection=[keep-alive], Charset=[utf-8], Host=[127.0.0.1:58523], Pragma=[no-cache], Content-length=[26], User-agent=[Java/10.0.2], Cache-con trol=[no-cache]} HTTP-Dispatcher - received request body : param1=a&param2=b&param3=c HTTP-Dispatcher - closing connection unexpectedly ... Tunnel: Client accepted Tunnel: Reading request line Tunnel: Request status line: CONNECT 127.0.0.1:58523 HTTP/1.1 Tunnel: Reading header: User-Agent: Java/10.0.2 Tunnel: Reading header: Host: 127.0.0.1:58523 Tunnel: Reading header: Accept: text/html, image/gif, image/jpeg, *; q=.2, */*; q=.2 Tunnel: Reading header: Proxy-Connection: keep-alive Tunnel: Reading header: Tunnel: Sending HTTP/1.1 200 OK Tunnel: Starting tunnel pipes Tunnel: Done - waiting for next client Tunnel: Waiting for client HTTP-Dispatcher - received request on : /foo/x HTTP-Dispatcher - received request headers : {Content-type=[application/x-www-form-urlencoded], Accept=[text/html, image/gif, image/jpeg, *; q=.2, */*; q=.2], Connection=[keep-alive], Charset=[utf-8], Host=[127.0.0.1:58523], Pragma=[no-cache], Content-length=[26], User-agent=[Java/10.0.2], Cache-con trol=[no-cache]} Stopping proxy Proxy stopped Server stopped Exception in thread "main" java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(SocketInputStream.java:116) at java.base/java.net.SocketInputStream.read(SocketInputStream.java:171) at java.base/java.net.SocketInputStream.read(SocketInputStream.java:141) at java.base/sun.security.ssl.SSLSocketInputRecord.read(SSLSocketInputRecord.java:425) at java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(SSLSocketInputRecord.java:65) at java.base/sun.security.ssl.SSLSocketImpl.bytesInCompletePacket(SSLSocketImpl.java:918) at java.base/sun.security.ssl.AppInputStream.read(AppInputStream.java:144) at java.base/java.io.BufferedInputStream.fill(BufferedInputStream.java:252) at java.base/java.io.BufferedInputStream.read1(BufferedInputStream.java:292) at java.base/java.io.BufferedInputStream.read(BufferedInputStream.java:351) at java.base/sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:746) at java.base/sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:689) at java.base/sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:859) at java.base/sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:689) at java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1604) at java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1509) at java.base/java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:527) at java.base/sun.net.www.protocol.https.HttpsURLConnectionImpl.getResponseCode(HttpsURLConnectionImpl.java:329) at JDK8209178.callHttpsServerThroughProxy(JDK8209178.java:176) at JDK8209178.test(JDK8209178.java:145) at JDK8209178.main(JDK8209178.java:118)
17-09-2018

Additional Information from submitter : The bug has been marked "Cannot Reproduce" too quickly I think. The provided test case, was only met to be the client part of the test : you need a faulty https server to reproduce the problem in the retry mechanism I didn't know how to make a complete test case but I find an existing test in openjdk that I could use as a basis http://hg.openjdk.java.net/jdk10/jdk10/jdk/file/0220dcbe2106/test/java/net/httpclient/ProxyTest.java So here is below a complete test that reproduce the bug. The https server close the connection abrutly the first time it is called. The jdk HttpsClient try to make a retry but creating a new proxy tunnel and resend the request but the bug triggers and the body is not sent As the server tries to read the body it waits for it and never responds. So the client timeouts ... ************** /* * Copyright (c) 2017, Oracle and/or its affiliates. All rights reserved. * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. * * This code is free software; you can redistribute it and/or modify it * under the terms of the GNU General Public License version 2 only, as * published by the Free Software Foundation. * * This code is distributed in the hope that it will be useful, but WITHOUT * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License * version 2 for more details (a copy is included in the LICENSE file that * accompanied this code). * * You should have received a copy of the GNU General Public License version * 2 along with this work; if not, write to the Free Software Foundation, * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA. * * Please contact Oracle, 500 Oracle Parkway, Redwood Shores, CA 94065 USA * or visit www.oracle.com if you need additional information or have any * questions. */ import java.io.BufferedReader; import java.io.File; import java.io.FileInputStream; import java.io.IOException; import java.io.InputStream; import java.io.InputStreamReader; import java.io.OutputStream; import java.io.OutputStreamWriter; import java.io.PrintWriter; import java.io.Writer; import java.net.InetAddress; import java.net.InetSocketAddress; import java.net.MalformedURLException; import java.net.ProtocolException; import java.net.Proxy; import java.net.ServerSocket; import java.net.Socket; import java.net.URI; import java.net.URISyntaxException; import java.nio.charset.StandardCharsets; import java.security.KeyManagementException; import java.security.KeyStore; import java.security.KeyStoreException; import java.security.NoSuchAlgorithmException; import java.security.NoSuchProviderException; import java.security.UnrecoverableKeyException; import java.security.cert.CertificateException; import java.util.HashMap; import java.util.StringTokenizer; import javax.net.ssl.HostnameVerifier; import javax.net.ssl.HttpsURLConnection; import javax.net.ssl.KeyManagerFactory; import javax.net.ssl.SSLContext; import javax.net.ssl.SSLSession; import javax.net.ssl.TrustManagerFactory; import com.sun.net.httpserver.HttpContext; import com.sun.net.httpserver.HttpExchange; import com.sun.net.httpserver.HttpHandler; import com.sun.net.httpserver.HttpServer; import com.sun.net.httpserver.HttpsConfigurator; import com.sun.net.httpserver.HttpsParameters; import com.sun.net.httpserver.HttpsServer; import java.security.cert.X509Certificate; import sun.security.tools.keytool.CertAndKeyGen; import sun.security.x509.X500Name; /** * DERIVED FROM http://hg.openjdk.java.net/jdk10/jdk10/jdk/file/0220dcbe2106/test/java/net/httpclient/ProxyTest.java * @author Arrault Fabien (farrault@ippon.fr) */ public class ProxyTest { static { // System.setProperty("javax.net.debug","all"); // No ... both client and server are logging ... try { HttpsURLConnection.setDefaultHostnameVerifier(new HostnameVerifier() { public boolean verify(String hostname, SSLSession session) { return true; } }); SSLContext.setDefault(new TestSSLContext().get()); } catch (Exception ex) { throw new ExceptionInInitializerError(ex); } } static final String RESPONSE = "<html><body><p>Hello World!</body></html>"; static final String PATH = "/foo/"; static HttpServer createHttpsServer() throws IOException, NoSuchAlgorithmException { HttpsServer server = com.sun.net.httpserver.HttpsServer.create(); HttpContext context = server.createContext(PATH); context.setHandler(new HttpHandler() { boolean simulateError = true; @Override public void handle(HttpExchange he) throws IOException { System.out.printf("%s - received request on : %s%n",Thread.currentThread().getName(),he.getRequestURI()); System.out.printf("%s - received request headers : %s%n",Thread.currentThread().getName(),new HashMap(he.getRequestHeaders())); InputStream requestBody = he.getRequestBody(); String body = ProxyTest.toString(requestBody); System.out.printf("%s - received request body : %s%n",Thread.currentThread().getName(),body); if(simulateError) { simulateError = false; System.out.printf("%s - closing connection unexpectedly ... %n",Thread.currentThread().getName(),he.getRequestHeaders()); he.close(); // try not to respond anything the first time ... return; } he.getResponseHeaders().add("encoding", "UTF-8"); he.sendResponseHeaders(200, RESPONSE.length()); he.getResponseBody().write(RESPONSE.getBytes(StandardCharsets.UTF_8)); he.close(); } }); server.setHttpsConfigurator(new Configurator(SSLContext.getDefault())); server.bind(new InetSocketAddress(InetAddress.getLoopbackAddress(), 0), 0); return server; } public static void main(String[] args) throws IOException, URISyntaxException, NoSuchAlgorithmException, InterruptedException { HttpServer server = createHttpsServer(); server.start(); try { new ProxyTest().test(server); } finally { server.stop(0); System.out.println("Server stopped"); } } public void test(HttpServer server /*, HttpClient.Version version*/) throws IOException, URISyntaxException, NoSuchAlgorithmException, InterruptedException { System.out.println("Server is: " + server.getAddress().toString()); System.out.println("Verifying communication with server"); URI uri = new URI("https:/" + server.getAddress().toString() + PATH + "x"); TunnelingProxy proxy = new TunnelingProxy(server); proxy.start(); try { System.out.println("Proxy started"); Proxy p = new Proxy(Proxy.Type.HTTP, InetSocketAddress.createUnresolved("localhost", proxy.getAddress().getPort())); System.out.println("Verifying communication with proxy"); callHttpsServerThroughProxy(uri, p); } finally { System.out.println("Stopping proxy"); proxy.stop(); System.out.println("Proxy stopped"); } } private void callHttpsServerThroughProxy(URI uri, Proxy p) throws IOException, MalformedURLException, ProtocolException { HttpsURLConnection urlConnection = (HttpsURLConnection) uri.toURL().openConnection(p); urlConnection.setConnectTimeout(1000); urlConnection.setReadTimeout(3000); urlConnection.setDoInput(true); urlConnection.setDoOutput(true); urlConnection.setRequestMethod("POST"); urlConnection.setUseCaches(false); urlConnection.setRequestProperty("Content-Type", "application/x-www-form-urlencoded"); urlConnection.setRequestProperty("charset", "utf-8"); urlConnection.setRequestProperty("Connection", "keep-alive"); String urlParameters = "param1=a&param2=b&param3=c"; byte[] postData = urlParameters.getBytes(StandardCharsets.UTF_8); OutputStream outputStream = urlConnection.getOutputStream(); outputStream.write(postData); outputStream.close(); int responseCode = urlConnection.getResponseCode(); System.out.printf(" ResponseCode : %s%n", responseCode); String output; InputStream inputStream= (responseCode < 400) ? urlConnection.getInputStream() : urlConnection.getErrorStream(); output = toString(inputStream); inputStream.close(); System.out.printf(" Output from server : %s%n", output); if (responseCode == 200) { // OK ! } else { throw new RuntimeException("Bad response Code : " + responseCode); } } static class TunnelingProxy { final Thread accept; final ServerSocket ss; final boolean DEBUG = false; final HttpServer serverImpl; TunnelingProxy(HttpServer serverImpl) throws IOException { this.serverImpl = serverImpl; ss = new ServerSocket(); accept = new Thread(this::accept); } void start() throws IOException { ss.bind(new InetSocketAddress(InetAddress.getLoopbackAddress(), 0)); accept.start(); } // Pipe the input stream to the output stream. private synchronized Thread pipe(InputStream is, OutputStream os, char tag) { return new Thread("TunnelPipe("+tag+")") { @Override public void run() { try { try { int c; while ((c = is.read()) != -1) { os.write(c); os.flush(); // if DEBUG prints a + or a - for each transferred // character. if (DEBUG) System.out.print(tag); } is.close(); } finally { os.close(); } } catch (IOException ex) { if (DEBUG) ex.printStackTrace(System.out); } } }; } public InetSocketAddress getAddress() { return new InetSocketAddress(ss.getInetAddress(), ss.getLocalPort()); } // This is a bit shaky. It doesn't handle continuation // lines, but our client shouldn't send any. // Read a line from the input stream, swallowing the final // \r\n sequence. Stops at the first \n, doesn't complain // if it wasn't preceded by '\r'. // String readLine(InputStream r) throws IOException { StringBuilder b = new StringBuilder(); int c; while ((c = r.read()) != -1) { if (c == '\n') break; b.appendCodePoint(c); } if (b.codePointAt(b.length() -1) == '\r') { b.delete(b.length() -1, b.length()); } return b.toString(); } public void accept() { Socket clientConnection = null; try { while (true) { System.out.println("Tunnel: Waiting for client"); Socket previous = clientConnection; try { clientConnection = ss.accept(); } catch (IOException io) { if (DEBUG) io.printStackTrace(System.out); break; } finally { // we have only 1 client at a time, so it is safe // to close the previous connection here if (previous != null) previous.close(); } System.out.println("Tunnel: Client accepted"); Socket targetConnection = null; InputStream ccis = clientConnection.getInputStream(); OutputStream ccos = clientConnection.getOutputStream(); Writer w = new OutputStreamWriter(ccos, "UTF-8"); PrintWriter pw = new PrintWriter(w); System.out.println("Tunnel: Reading request line"); String requestLine = readLine(ccis); System.out.println("Tunnel: Request status line: " + requestLine); if (requestLine.startsWith("CONNECT ")) { // We should probably check that the next word following // CONNECT is the host:port of our HTTPS serverImpl. // Some improvement for a followup! // Read all headers until we find the empty line that // signals the end of all headers. while(!requestLine.equals("")) { System.out.println("Tunnel: Reading header: " + (requestLine = readLine(ccis))); } // Open target connection targetConnection = new Socket( serverImpl.getAddress().getAddress(), serverImpl.getAddress().getPort()); // Then send the 200 OK response to the client System.out.println("Tunnel: Sending " + "HTTP/1.1 200 OK\r\n\r\n"); pw.print("HTTP/1.1 200 OK\r\nContent-Length: 0\r\n\r\n"); pw.flush(); } else { // This should not happen. throw new IOException("Tunnel: Unexpected status line: " + requestLine); } // Pipe the input stream of the client connection to the // output stream of the target connection and conversely. // Now the client and target will just talk to each other. System.out.println("Tunnel: Starting tunnel pipes"); Thread t1 = pipe(ccis, targetConnection.getOutputStream(), '+'); Thread t2 = pipe(targetConnection.getInputStream(), ccos, '-'); t1.start(); t2.start(); // We have only 1 client... wait until it has finished before // accepting a new connection request. // System.out.println("Tunnel: Waiting for pipes to close"); // t1.join(); // t2.join(); System.out.println("Tunnel: Done - waiting for next client"); } } catch (Throwable ex) { try { ss.close(); } catch (IOException ex1) { ex.addSuppressed(ex1); } ex.printStackTrace(System.err); } } void stop() throws IOException { ss.close(); } } static class Configurator extends HttpsConfigurator { public Configurator(SSLContext ctx) { super(ctx); } @Override public void configure (HttpsParameters params) { params.setSSLParameters (getSSLContext().getSupportedSSLParameters()); } } public static class TestSSLContext { SSLContext ssl; public TestSSLContext() throws Exception { init(); } private void init() throws Exception { CertAndKeyGen keyGen=new CertAndKeyGen("RSA","SHA1WithRSA",null); keyGen.generate(1024); //Generate self signed certificate X509Certificate[] chain=new X509Certificate[1]; chain[0]=keyGen.getSelfCertificate(new X500Name("CN=ROOT"), (long)365*24*3600); char[] passphrase = "passphrase".toCharArray(); KeyStore ks = KeyStore.getInstance("JKS"); ks.load(null, passphrase); // must be "initialized" ... ks.setKeyEntry("server", keyGen.getPrivateKey(), passphrase, chain); KeyManagerFactory kmf = KeyManagerFactory.getInstance("SunX509"); kmf.init(ks, passphrase); TrustManagerFactory tmf = TrustManagerFactory.getInstance("SunX509"); tmf.init(ks); ssl = SSLContext.getInstance("TLS"); ssl.init(kmf.getKeyManagers(), tmf.getTrustManagers(), null); } public SSLContext get() { return ssl; } } // ############################################################################################### private static String toString(InputStream inputStream) throws IOException { StringBuilder sb = new StringBuilder(); BufferedReader bufferedReader = new BufferedReader(new InputStreamReader(inputStream, StandardCharsets.UTF_8)); int i = bufferedReader.read(); while (i != -1) { sb.append((char) i); i = bufferedReader.read(); } bufferedReader.close(); return sb.toString(); } }
28-08-2018

I ran this test on an outside tls server through a corporate proxy and had not exception thrown. Issue may only manifest itself with the tester's proxy and/or tls server. The old webbug doesn't exist when I used the first link, here is the JBS version. https://bugs.openjdk.java.net/browse/JDK-6382788
15-08-2018