ADDITIONAL SYSTEM INFORMATION :
Windows 10
JDK 17
A DESCRIPTION OF THE PROBLEM :
When HttpClient sends an Expect:100-Continue header via http 2 protocol. If the server responds with an 417[Expectation Code] or any other error code besides code 100. The client hangs forever. Issue does not exist when using http 1.1 protocol
STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
1) Create an very basic HttpServer which does not parse any bytes but instead sends http 2 frames at each stage of the request.
2)Send an POST request with expectContinue enabled
3)Send one request to first upgrade the client first to http 2 and then on the 2nd request send POST with expectContinue enabled. Client exits normally if response code is 100 but hangs forever if code is anything else
EXPECTED VERSUS ACTUAL BEHAVIOR :
EXPECTED -
HttpClient should finish the Expect 100 request either as an normal response or should throw an exception if response code is not 100
ACTUAL -
Client hangs forever when Expect : 100-Continue response code is not 100
---------- BEGIN SOURCE ----------
The problem does not persist with http 1.1 protocol hence we need to first upgrade the protocol on the client and server to http 2 after exchanging Connection: Upgrade & 101 Switching protocol packets and then on the 2nd request we send an PUT request with an expect 100 header enabled.
To keep this example as simple as possible an very oversimplified version of HPack was implemented and the server does not attempt to parse any bytes of the client but from debugging experience i create stages where each stage decides what response to send to the client.
The most basic version of HPACK
private static final class OverSimplifiedHPack
{
/*
0 1 2 3 4 5 6 7
+---+---+---+---+---+---+---+---+
| 0 | 0 | 0 | 0 | Index (4+) |
+---+---+-----------------------+
| H | Value Length (7+) |
+---+---------------------------+
| Value String (Length octets) |
+-------------------------------+
*/
private static byte[] encode_literal_non_indexed_headers(String[] index_values)throws Exception
{
try(ByteArrayOutputStream encoding=new ByteArrayOutputStream();
ByteArrayOutputStream buffer=new ByteArrayOutputStream())
{
for(int i=0;i<=index_values.length-2;i+=2)
{
String value=index_values[i+1];
byte[]
header_index=encodePrefix(encoding,Integer.parseInt(index_values[i]),(byte)4),
valueBytes=value.getBytes(),
valueLength=encodePrefix(encoding,valueBytes.length,(byte)7);
encoding.reset();
encoding.write(header_index[0]&0b00001111);
if(header_index.length>1){encoding.write(header_index,1,header_index.length-1);}
encoding.write(valueLength[0]&0b01111111);
if(valueLength.length>1){encoding.write(valueLength,1,valueLength.length-1);}
encoding.write(valueBytes);
buffer.write(encoding.toByteArray());
}
return buffer.toByteArray();
}
}
private static byte[] encodePrefix(ByteArrayOutputStream encoding,int value,byte prefix)throws Exception
{
encoding.reset();
int max=(int)Math.pow(2,prefix)-1;
if(value<max){encoding.write(value);}
else
{
encoding.write(max);
value-=max;
while(value>128)
{
encoding.write(((value%128)+128));
value/=128;
}
encoding.write(value);
}
return encoding.toByteArray();
}
}
The HttpServer
The server gives some predefined responses and is designed to parse exactly 2 requests.
1)The 1st request is an random get request. This is designed to 1st upgrade the client to Http 2
2)The 2nd request is the actual Expect 100 header where there is one important boolean flag
boolean doContinue=true;
Setting this flag to true writes 100 which produces expected result on the client side. Setting to false however writes code 417 indicating expectation failed which hangs the client
8 = index of the header :status in the static encoding table of HPACK[RFC 7541]
28 = index of the header content-length in the static encoding table of HPACK[RFC 7541]
public static void main2(String[] args)throws Exception
{
try(ServerSocket socket=new ServerSocket(5000))
{
try(Socket client=socket.accept())
{
try(InputStream input=client.getInputStream();
OutputStream output=client.getOutputStream())
{
byte[] array=new byte[8196];
int
stage=1,
requestID=0;
/*Please change this variable for both test cases*/
boolean doContinue=true;
while(input.read(array)>0)
{
if(stage==1)
{
try(ByteArrayOutputStream serverPreface=new ByteArrayOutputStream())
{
serverPreface.write
(
(
"HTTP/1.1 101 Switching Protocols\r\n"+
"Connection:Upgrade\r\n"+
"Upgrade:h2c\r\n\r\n"
).getBytes()
);
byte[] settings={};
serverPreface.write
(
new byte[]
{
(byte)((settings.length>>16) & 0xFF),
(byte)((settings.length>>8) & 0xFF),
(byte)(settings.length & 0xFF),
(byte)0x4,
(byte)0x1,
0,0,0,0
}
);
serverPreface.write(settings);
output.write(serverPreface.toByteArray());
output.flush();
}
if(requestID==0){stage=3;}
}
else if(stage==2)
{
//if we are at the 2nd request skip any frames which is not an Header Frame
if(requestID==1 && array[3]!=0x1){continue;}
try(ByteArrayOutputStream frameBytes=new ByteArrayOutputStream())
{
byte[] headerBytes=OverSimplifiedHPack.encode_literal_non_indexed_headers
(
new String[]
{
"8",doContinue?"100":"407",
"28","0"
}
);
frameBytes.write
(
new byte[]
{
(byte)((headerBytes.length>>16) & 0xFF),
(byte)((headerBytes.length>>8) & 0xFF),
(byte)(headerBytes.length & 0xFF),
(byte)0x1,
(byte)0x4,
0,0,0,3
}
);
frameBytes.write(headerBytes);
output.write(frameBytes.toByteArray());
output.flush();
}
if(doContinue){stage=3;}
else{stage=-1;}
}
else if(stage==3)
{
//for the 2nd request we respond only after the client has sent us all the header and
//data frames. For any other frame types we don't respond
if(requestID==1 && !(array[3]==0x1 || array[3]==0x0)){continue;}
try(ByteArrayOutputStream frameBytes=new ByteArrayOutputStream())
{
byte[] content=(requestID==0?"Request 1 Done":"Test Complete").getBytes();
byte[] headerBytes=OverSimplifiedHPack.encode_literal_non_indexed_headers
(
new String[]
{
"8","202",
"28",""+content.length
}
);
frameBytes.write
(
new byte[]
{
(byte)((headerBytes.length>>16) & 0xFF),
(byte)((headerBytes.length>>8) & 0xFF),
(byte)(headerBytes.length & 0xFF),
(byte)0x1,
(byte)0x4,
0,0,0,(byte)(requestID==0?1:3)
}
);
frameBytes.write(headerBytes);
frameBytes.write
(
new byte[]
{
(byte)((content.length>>16) & 0xFF),
(byte)((content.length>>8) & 0xFF),
(byte)(content.length & 0xFF),
(byte)0x0,
(byte)0x1,
0,0,0,(byte)(requestID==0?1:3)
}
);
frameBytes.write(content);
output.write(frameBytes.toByteArray());
output.flush();
}
if(requestID==0)
{
requestID=1;
stage=2;
}
else{stage=-1;}
}
}
}
}
}
}
Output
When doContinue=true
Status code: 200
Headers: {:status=[200], content-length=[14]}
Body: Request 1 Done
=======================
Status code: 202
Headers: {:status=[202], content-length=[13]}
Body: Test Complete
When do Continue = false
Status code: 200
Headers: {:status=[200], content-length=[14]}
Body: Request 1 Done
=======================
/*HANGS FOREVER*/
---------- END SOURCE ----------
CUSTOMER SUBMITTED WORKAROUND :
No workaround. Issue does not exist for http 1.1. When upgrading to version 2 response 100 must be written before the 101 Switching protocol response and then both test cases work else client throws error.
FREQUENCY : always