JDK-4209283 : Compression across a socket causes problems
  • Type: Bug
  • Component: core-libs
  • Sub-Component: java.util
  • Affected Version: 1.1.7
  • Priority: P2
  • Status: Closed
  • Resolution: Duplicate
  • OS: windows_nt
  • CPU: x86
  • Submitted: 1999-02-05
  • Updated: 1999-02-08
  • Resolved: 1999-02-08
Related Reports
Duplicate :  
Description
A licensee needs to know if compression should work across a socket stream.

This doesn't seem to be a dup of 4030767 as the root cause of this problem
is compression.

Customer has reported problem as: Server hangs when creating ObjectInputStream
using compression across a socket stream

The problem arises when compression is used. The server hangs when trying to
create the ObjectInputStream. The JDK API docs say
that the ObjectInputStream constructor will block until it reads the stream
header.  (Using the debugger you can see it trying to inflate the
protocol for STREAM_MAGIC and STREAM_VERSION)

The ObjectInputStream in the server is built as follows:

    new  ObjectInputStream(
         new InflaterInputStream(
             new BufferedInputStream(socket.getInputStream())))

The ObjectOutputStream in the client is built in a similar way:

    new  ObjectOutputStream(
         new DeflaterOutputStream(
             new BufferedOutputStream(socket.getOutputStream())))

If you don't use the Inflater/Deflater streams (i.e., no compression), or if I
write to and read from a file, this works just fine. But when you use
compression across a socket stream, the ObjectInputStream blocks on
construction.

Should it work with compression across a socket stream?

Have tried various combinations of flush() and finish() with no joy. Only case
where you can get it to work is if stream is closed from client, which is not
acceptable as you would have to create a new stream for every object you wish
to send/receive.

The problem is reproducible on the JDK1.1.7 FCS, and 1.2
To see it when it works,

java TestSocketServer N 5000 1

java TestSocketClient 9.20.2.110 5000 1 N

This will send one object from the client to the server
(substitute your ip address).  The client will sleep at the
end before returning for a while, this is to better
demonstate the server hang when not working.

To see it failing (the 'y' instead of 'n' indicates
to use compression)

java TestSocketServer Y 5000 1

java TestSocketClient 9.20.2.110 5000 1 Y

The client sleeps as normal but the server hangs.

Warning when the client finishes sleeping the following
exception occurs on the server

Under 1.1.7 you get @
Exception while creating input stream.
java.io.EOFException: Unexpected end of ZLIB input stream


Under 1.2 you get :
 java.io.StreamCorruptedException: Caught EOFException while reading the stream header

That is why I increased the sleep on the client otherwise
this happened immediately.

There is a readme.1st in the test case bundle, the above
is an abbreviated version of how to run with the extra
warning on the exception and below is copied from the readme:

The problem arises when compression is used. The server hangs when trying to
create the ObjectInputStream. The JDK API docs say that the ObjectInputStream
constructor will block until it reads the stream header.

The ObjectInputStream in the server is built as follows:

    new  ObjectInputStream(
         new InflaterInputStream(
             new BufferedInputStream(socket.getInputStream())))

The ObjectOutputStream in the client is built in a similar way:


    new  ObjectOutputStream(
         new DeflaterOutputStream(
             new BufferedOutputStream(socket.getOutputStream())))

If I don't use the Inflater/Deflater streams (i.e., no compression), or if I
write to and read from a file, this works just fine. But when I use compression
across a socket stream, the ObjectInputStream blocks on construction.

Tried it with jit off but still fails.

The attached test case hangs under NT as well.

>From what I can see 4 bytes of protocol are transmitted from the client
to the server (magic number and version) the code being something like this:

ObjectInputStream->readStreamHeader()  calls readShort
DataInputStream->readShort() calls read
BufferedInputStream->read() calls fill
BufferedInputStream->fill() calls read
InflaterInputStream->read() calls inflate(native) this checks the 2 byte
buffer and returns Z_STREAM_ERROR, inf.needsInput it true so fill() is
called.

*I don't think the protocol should be inflated.*

I'm pretty convinced that this is the problem.

I've tried a few tests, in particular faking the protocol so the server
proceeds onto receiving the object stream and all seems ok.

One of the things I would of like to of tried is for the readStreamHeader()
method
to directly call the native method java_net_SocketInputStream_socketRead()
(from SocketInputStream->socketRead).  But this involved changing quite a few
things from private to public and I thought there would of been a far easier
fix than this.

Closing down the stream in the testcase after data has been sent fixes the
problem (but is not a solution).

Comments
EVALUATION If anything, this is probably an issue relating to the java.util.zip compression stream classes. I'm recategorizing it. I'm guessing this is a case of user error perhaps compounded by incomplete docs on the blocking/chunking behavior of the compression classes. More data may need to be written on the DeflatorOutputStream in order to push through the next chunk of compresses data so that it can be inflated at the other end and supply enough data so that ObjectInputStream can read it's magic header bytes. Just a guess. I'm not really familiar with the compression algorithm details. jeff.nisewanger@Eng 1999-02-05 Right now, java.util.zip does not support partial flush in the deflator. Deflator calls zlib deflate with Z_NO_FLUSH flag. As a result, the compressor will not emit enough data for the decompressor to inflate, causing the decompressor to hang. This feature is under consideration for our next release. Duplicate of 4206909. ###@###.### 1999-02-08
08-02-1999