JDK-6525563 : Memory leak in ObjectOutputStream
  • Type: Bug
  • Component: core-libs
  • Sub-Component: java.io:serialization
  • Affected Version: 6
  • Priority: P4
  • Status: Open
  • Resolution: Unresolved
  • OS: windows_xp
  • CPU: x86
  • Submitted: 2007-02-15
  • Updated: 2013-05-01
Related Reports
Relates :  
Description
FULL PRODUCT VERSION :
java version "1.6.0"
Java(TM) SE Runtime Environment (build 1.6.0-b105)
Java HotSpot(TM) Client VM (build 1.6.0-b105, mixed mode, sharing)

ADDITIONAL OS VERSION INFORMATION :
Microsoft Windows XP [Wersja 5.1.2600]

EXTRA RELEVANT SYSTEM CONFIGURATION :
To speed up the fault I have limited maximum heap size for VM with -Xmx20m
option.

Be sure to have enough free disk space for output file. During the test with
-Xmx20m option the out.dat is around 14Mb size.

A DESCRIPTION OF THE PROBLEM :
I have written a simple application that sends a lot (1500 per second) serialized small objects over the ObjectOutputStream via TCP connection.
During the testing I have discovered memory leaks and application crashed due to "out of memory".

First I have used a method ObjectOutputStream.writeObject(o).
This of course casued growing:
    private final HandleTable handles;
    private final ReplaceTable subs;
in ObjectOutputStream. This has prevented my objects to be garbage collected after sending and that was my intention.

My test does not need to keep cashed objects so I have changed implementation to to ObjectOutputStream.writeUnshared(o).

My expectation was that this will end gowring tables in ObjectOutputStream.
Unfortunately  handles table have been still growing but my objects have been garbage collected. I have classified this as "memory leak".

Looking deeper into the problem the I have found possible reason.
The order of metthod calls is as follows:
ObjectOutputStream.writeUnshared(obj)
--- writeObject0(obj, true);
------ writeOrdinaryObject(obj, desc, unshared);
--------- handles.assign(unshared ? null : obj);
This last call is really:
    handles.assign(null);

Here you have a code for  ObjectOutputStream.HandleTable.assign(Object)
	int assign(Object obj) {
	    if (size >= next.length) {
		growEntries();
	    }
	    if (size >= threshold) {
		growSpine();
	    }
	    insert(obj, size);
	    return size++;
	}

What you can see here is that size of handle table is growing even if  this function is called with "null" argument. Increasing handles table is causing VM to run out of memory after some time.


STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
0. make all following steps in one directory
1. put the source in TestObjectOutputStream.java file
2. compile source:
	javac TestObjectOutputStream.java
3. run test with output file name as its argument:
	java TestObjectOutputStream out.dat
   or to get faster result limit max heap size:
	java -Xmx20m TestObjectOutputStream out.dat


EXPECTED VERSUS ACTUAL BEHAVIOR :
EXPECTED -
Note that I didn't manage to get to expected behavior due to the
OutOfMemoryError fault.
   1. file out.dat should grow until disklimit
   2. finally application should end with exception

ACTUAL -
z:\java>java -Xmx20m TestObjectOutputStream out.dat
Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
        at java.io.ObjectOutputStream$HandleTable.growEntries(Unknown
Source)
        at java.io.ObjectOutputStream$HandleTable.assign(Unknown Source)
        at java.io.ObjectOutputStream.writeOrdinaryObject(Unknown Source)
        at java.io.ObjectOutputStream.writeObject0(Unknown Source)
        at java.io.ObjectOutputStream.writeUnshared(Unknown Source)
        at TestObjectOutputStream.main(TestObjectOutputStream.java:36)

For exact line numbers I have run it in NetBeans IDE 5.5 and result is:
Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
        at
java.io.ObjectOutputStream$HandleTable.growEntries(ObjectOutputStream.java:2
308)
        at
java.io.ObjectOutputStream$HandleTable.assign(ObjectOutputStream.java:2237)
        at
java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1389)
        at
java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1150)
        at
java.io.ObjectOutputStream.writeUnshared(ObjectOutputStream.java:393)
        at
testobjectoutputstream.TestObjectOutputStream.main(TestObjectOutputStream.ja
va:44)

REPRODUCIBILITY :
This bug can be reproduced always.

---------- BEGIN SOURCE ----------
import java.io.*;

class Data implements Serializable{
    int d=0;
}

public class TestObjectOutputStream {
    
    public TestObjectOutputStream() {
    }
    
    public static void main(String[] args) {
        ObjectOutputStream out=null;
        try {
            out = new java.io.ObjectOutputStream(
                    new java.io.FileOutputStream(args[0]));
            while (true){
                out.writeUnshared(new Data());
            }
        } catch (FileNotFoundException ex) {
            ex.printStackTrace();
        } catch (IOException ex) {
            ex.printStackTrace();
        }
    }
}
---------- END SOURCE ----------

CUSTOMER SUBMITTED WORKAROUND :
Call from time to time:
   ObjectOutputStream.reset()
This will clear handles.

Anyway you can never be sure that you will reset it on time in live telecom system.

Comments
EVALUATION I suppose that it is not inconceivable that ObjectOutputStream's handle table data structure could be reworked so that it did not grow in proportion to the number of handles allocated for objects written "unshared"-- but that would likely be at the performance expense, in space and time, of normal usage, and it's not obvious that the same could be done for ObjectInputStream's handle table. Despite the use of writeUnshared, the usage described in this report is pretty much reminiscent of that described in 4363937, 4075901, and related closed reports. (And note that as soon as the written Data class has references to other objects-- not just primitive data-- in its serialized form, those objects will require non-empty handles unless they are also all written "unshared".) As described in the previously cited reports, the proper way to write a continuing series of objects that are not intended to represent a single consistent object graph (with integrity of references, etc.) is to invoke ObjectOutputStream.reset() in between the writes.
15-02-2007