JDK-6404711 : BitSet serialization optimization defeats explicit capacity management
  • Type: Bug
  • Component: core-libs
  • Sub-Component: java.util
  • Affected Version: 6
  • Priority: P3
  • Status: Closed
  • Resolution: Fixed
  • OS: solaris_10
  • CPU: sparc
  • Submitted: 2006-03-27
  • Updated: 2010-04-02
  • Resolved: 2006-04-14
The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.
JDK 6
6 b81Fixed
Related Reports
Relates :  
Description
FULL PRODUCT VERSION :
java version "1.6.0-beta"
Java(TM) 2 Runtime Environment, Standard Edition (build 1.6.0-beta-b59g)
Java HotSpot(TM) Client VM (build 1.6.0-beta-b59g, mixed mode, sharing)


ADDITIONAL OS VERSION INFORMATION :
Windos XP

A DESCRIPTION OF THE PROBLEM :
Since 1.6, serializing & deserializing a BitSet will change its memory footprint. Serializing will trim any excess space at the end which results in less data to save, which is good.

After deserialization the new BitSet will have a smaller size than the original one, which can be a problem.

The problem is that a set() which is within the original size of the BitSet might trigger a reallocation in the serialized/deserialized version. This can result in a BitSet which is nearly twice the size of the original untrimmed BitSet and can even lead to an OutOfMemory.


STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :

Compile and run the below test case:

>>javac BitSetBug.java
>>java BitSetBug read
>>java BitSetBug write


import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.io.ObjectInputStream;
import java.io.ObjectOutputStream;
import java.util.BitSet;


public class BitSetBug
{
    private final static int size = 8 * 1024 * 1024;
    private final static String fileName = "bitset";

    public static void main(String[] args)
    throws Exception
    {
        if ("read".equals(args[0])) {
            read();
        } else if ("write".equals(args[0])) {
            write();
        }        
    }
    
    public static void write()
    throws Exception
    {        
        // create a bitset & set bit in second to last word
        BitSet bitset = new BitSet(size);
        bitset.set(size - 65);

        System.out.println("1: originalBitset size: " + bitset.size());
        
        // serialize bitset
        FileOutputStream fileOut = new FileOutputStream(fileName);
        ObjectOutputStream objectOut = new ObjectOutputStream(fileOut);
        objectOut.writeUnshared(bitset);
        objectOut.flush();
        fileOut.close();
        
        System.out.println("2: originalBitset size: " + bitset.size());
        
        // set bit in last word
        bitset.set(size - 1);

        System.out.println("3: originalBitset size: " + bitset.size());
    }
    
    public static void read()
    throws Exception
    {
        // deserialize bitset
        FileInputStream fileIn = new FileInputStream(fileName);
        ObjectInputStream objectIn = new ObjectInputStream(fileIn);
        
        BitSet bitset = (BitSet)objectIn.readObject();

        System.out.println("4: deserializedBitset size: " + bitset.size());

        // set bit in last word
        bitset.set(size - 1);

        System.out.println("5: deserializedBitset size: " + bitset.size());
    }
}

Output on 1.5:
---------------

> > /usr/java/jre1.5.0_06/bin/java -classpath . BitSetBug write
1: originalBitset size: 8388608
2: originalBitset size: 8388608
3: originalBitset size: 8388608
> > /usr/java/jre1.5.0_06/bin/java -classpath . BitSetBug read
4: deserializedBitset size: 8388608
5: deserializedBitset size: 8388608

Output on 1.6:
--------------
> > /mnt/support1/jdk1.6.0/bin/java -classpath . BitSetBug write
1: originalBitset size: 8388608
2: originalBitset size: 8388544
3: originalBitset size: 16777088
> > /mnt/support1/jdk1.6.0/bin/java -classpath . BitSetBug read
4: deserializedBitset size: 8388544
5: deserializedBitset size: 16777088

Note how on 1.6, the bitset grows to almost 16Mb (megabit) instead of staying at 8Mb instead.


EXPECTED VERSUS ACTUAL BEHAVIOR :
EXPECTED -
Expectation was that, like in 1.5, the deserialized BitSet would stay at 32MB size if sets where done within original bounds.
ACTUAL -
Result is that the deserialized BitSet can grow to nearly double the original memory footprint.

REPRODUCIBILITY :
This bug can be reproduced always.

Release Regression From : 5.0u6
The above release value was the last known release where this 
bug was known to work. Since then there has been a regression.

Comments
EVALUATION After some thought, I've decided to respect the user's wishes when setting the capacity in the constructor. The idea is to add a "sticky size" flag.
06-04-2006

EVALUATION The current behavior is an intentional result of the changes for 4963875: RFE: Reduction of space used by instances of java.util.BitSet The submitter's scenario is a reasonable one, and was not considered during implementation of 4963875. There are a number of improvements we can make to the current BitSet: - make trimToSize public - make the backing array grow by 1.5, not 2 We could add a field to remember the former size, but that would make everyone pay for an optimization for a very few. I'm not sure that would be right.
27-03-2006

WORK AROUND BitSet.trimToSize() is not public, but users can make a copy with a desired capacity by BitSet deserialized = ... BitSet copy = new BitSet(desiredCapacity); copy.setOr(deserialized); deserialized = null; // gc
27-03-2006