FULL PRODUCT VERSION :
java version "1.6.0_04"
Java(TM) SE Runtime Environment (build 1.6.0_04-b12)
Java HotSpot(TM) Server VM (build 10.0-b19, mixed mode)


FULL OS VERSION :
SunOS gold01 5.10 Generic_125101-07 i86pc i386 i86pc Solaris

A DESCRIPTION OF THE PROBLEM :
We are using WeakReferences to manage distributed object cache. When program no longer references given object, we rely on WeakReference getting enqueued at some point so we can send remote system information that client is not longer interested in object updates. Unfortunately, there seems to be a big change of behaviour in CMS between 1.6.0_02 and 1.6.0_04 - with _02, weak references were cleared and enqueued within one or two CMS runs after they stopped being referenced, with _04 it seems we have to wait till we run out of memory (and full gc) for this to happen.

THE PROBLEM WAS REPRODUCIBLE WITH -Xint FLAG: Did not try

THE PROBLEM WAS REPRODUCIBLE WITH -server FLAG: Yes

STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
Run the program attached below with following parameters

java -server -Xmx300m -XX:+UseConcMarkSweepGC -XX:+CMSIncrementalMode -XX:CMSIncrementalDutyCycleMin=5 -XX:+CMSIncrementalPacing  -XX:+PrintGCDetails -XX:+UseParNewGC -XX:+ParallelRefProcEnabled -cp . WeakTest

with jdk 1.6.0_02 and 1.6.0_04.


EXPECTED VERSUS ACTUAL BEHAVIOR :
With 1.6.0_02, as soon as first CMS is finished, some of "Removed X nnn" messages start to appear and then they continue to appear quite frequently. On my machine, it takes around 700 adds to reach the stage where references are starting to be removed almost in realtime.

With 1.6.0_04, I'm not able to spot even single removal of weak reference for a long, long time (after that I have stopped testing - I suppose that on reaching out of memory condition, full gc will clear them finally).

Please note that I do not mean refList growing - it is obvious leak (very small) to hold the weak references referenced. Problem is with queue removal thread never being woken up by queue.


REPRODUCIBILITY :
This bug can be reproduced always.

---------- BEGIN SOURCE ----------
import java.lang.ref.*;
import java.util.ArrayList;

public class WeakTest {

  static Object[] refArray;
  
  public static void main(String[] args) throws Exception {
    
    Object[][][] start= new Object[1000][1000][2];
    
    final ReferenceQueue queue = new ReferenceQueue();
    
    ArrayList<Reference> refList = new ArrayList<Reference>();
    
    
    int counter = 0;
    
    new Thread() {
      @Override public void run() {
        while ( true ) {
          try {
            Reference r = queue.remove();
            System.out.println("Removed " + r);
          } catch (Exception exc) {
            exc.printStackTrace();
          }
        }
      }
    }.start();
    
    refArray = new Object[100];
    
    
    while ( true ) {
      Integer[][] arr = new Integer[1000][];
      for ( int i =0; i < arr.length; i++) {
        arr[i] = new Integer[1000];
      }

      final int fcounter = counter++;

      
      refArray[fcounter%refArray.length] = arr[0];
      
      WeakReference reference = new WeakReference(arr[0],queue) {
        @Override public String toString() {
          return "X " + fcounter;
        }
      };
      
      
      refList.add(reference);
      
      
      if ( fcounter % 100 == 99 ) {
        System.out.println("Added " + fcounter + " references");
        Thread.sleep(10000);
      }
      
    }
  }
}

---------- END SOURCE ----------

CUSTOMER SUBMITTED WORKAROUND :
Invoke full gc with System.gc() once per some time [not acceptable]
Stay with 1.6.0_02

Release Regression From : 6u2
The above release value was the last known release where this 
bug was not reproducible. Since then there has been a regression.

Release Regression From : 6u2
The above release value was the last known release where this 
bug was not reproducible. Since then there has been a regression.