FULL PRODUCT VERSION :
/opt/SUNXjdk/1.6/bin/java -fullversion
java full version "1.6.0-ea-b49"


ADDITIONAL OS VERSION INFORMATION :
SunOS snake 5.10 Generic_118844-08 i86pc i386 i86pc


EXTRA RELEVANT SYSTEM CONFIGURATION :
W1100z

A DESCRIPTION OF THE PROBLEM :
Under Solaris x86 Xtoolkit the use of an icon on a Frame causes the GC
subsystem to fail e.g. no GC of a disposed Frame object.  Thus if you
extend Frame and store lots of data you will have a massive leak. I
assume that this issue also exists under Linux as XToolkit is the
default (note it is my understanding that it will also be the default
for J2SE 5.0 and 6.0 in the near future).

IMPACTS
    J2SE 5.0 and J2SE 6.0 Solaris x86 explicitly using Xtoolkit
    J2SE 5.0 and J2SE 6.0 ALL Linux varients

MACHINE ENVIRONMENT
    uname -a
    SunOS snake 5.10 Generic_118844-08 i86pc i386 i86pc

JVM J2SE versions 1.5 and 1.6 both fail exactly the same (1.6 must be at rev
1.6.0-ea-b49).

    java -version
    java version "1.5.0_04"
    Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_04-b05)
    Java HotSpot(TM) Client VM (build 1.5.0_04-b05, mixed mode)

    java -version
    java version "1.6.0-ea"
    Java(TM) 2 Runtime Environment, Standard Edition (build 1.6.0-ea-b45) *1
    -and-
    Java(TM) 2 Runtime Environment, Standard Edition (build 1.6.0-ea-b48) *1
    -and-
    Java(TM) 2 Runtime Environment, Standard Edition (build 1.6.0-ea-b49)

*1 Special note for JVM J2SE 1.6, prior to 1.6.0-ea-b49 e.g. versions
   1.6.0-ea-b45 to 1.6.0-ea-b48 the leak occurs regardless of the NOICON
   argument (which eliminates the leak for 1.5.0_04-b05 and 1.6.0-ea-b49),
   thus the August 25, 2005 mustang build fixed only part of the Frame
   leak i.e. Frames with no icons, however the leak still exists for Frames
   with icons.


EXPECTED BEHAVIOUR (good run, Solaris 10 X86, with MToolKit, currently the default)
java -Xms32m -Xmx32m  XtoolkitIconLeak
HEAP USED =    392896, ACTIVITY = new Frame()
HEAP USED =   5453544, ACTIVITY = Frame.dispose()
HEAP USED =    403872, ACTIVITY = new Frame()
HEAP USED =   5404656, ACTIVITY = Frame.dispose()
HEAP USED =    403800, ACTIVITY = new Frame()
HEAP USED =   5404760, ACTIVITY = Frame.dispose()
HEAP USED =    403904, ACTIVITY = new Frame()
HEAP USED =   5404864, ACTIVITY = Frame.dispose()


UNEXPEDTED BEHAVIOUR bad run, e.g. use -Dawt.toolkit=sun.awt.X11.XToolkit
java -Xms32m -Xmx32m -Dawt.toolkit=sun.awt.X11.XToolkit XtoolkitIconLeak
HEAP USED =    533976, ACTIVITY = new Frame()
HEAP USED =   5567728, ACTIVITY = Frame.dispose()
HEAP USED =   5567880, ACTIVITY = new Frame()
HEAP USED =  10539600, ACTIVITY = Frame.dispose()
HEAP USED =  10571208, ACTIVITY = new Frame()
HEAP USED =  15542360, ACTIVITY = Frame.dispose()
HEAP USED =  15572952, ACTIVITY = new Frame()
HEAP USED =  20464112, ACTIVITY = Frame.dispose()

WORKAROUND, DO NOT USE Frame.setIconImage()
java -Xms32m -Xmx32m -Dawt.toolkit=sun.awt.X11.XToolkit XtoolkitIconLeak NOICON
Not installing icons on frame via Frame.setIconImage()
HEAP USED =    550080, ACTIVITY = new Frame()
HEAP USED =   5551792, ACTIVITY = Frame.dispose()
HEAP USED =   5551696, ACTIVITY = new Frame()
HEAP USED =   5552144, ACTIVITY = Frame.dispose()
HEAP USED =   5551984, ACTIVITY = new Frame()
HEAP USED =   5553192, ACTIVITY = Frame.dispose()
HEAP USED =   5553032, ACTIVITY = new Frame()
HEAP USED =   5553672, ACTIVITY = Frame.dispose()


Note I have tried to submit this bug here before and also under my support contract but
nothing seems to happen and the BUG is never viewable in the BUG database.  A related
issue was recently fixed e.g. BUG 6257260 "Memory leak on closing JFrame" , however
I am sure if an ICON is added to the Frame(s) created the leak still remains.

STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
1) Run the program

java -Xms32m -Xmx32m -Dawt.toolkit=sun.awt.X11.XToolkit XtoolkitIconLeak

2) hit the button, then close the resulting frame

3) repeat 2) above (about 7 times), until all 32MB is used

EXPECTED VERSUS ACTUAL BEHAVIOR :
EXPECTED -
The heap should have hovered around 5MB
ACTUAL -
The heap grew by 5MB locked up in each Frame that is created, it is never freed on Frame.dispose() call.   This seems to happen due to a weak reference to the ICON that somehow remains deep in the AWT or X11 code.



ERROR MESSAGES/STACK TRACES THAT OCCUR :
HEAP USED =    476136, ACTIVITY = new Frame()
HEAP USED =   5499744, ACTIVITY = Frame.dispose()
HEAP USED =   5475064, ACTIVITY = new Frame()
HEAP USED =  10502552, ACTIVITY = Frame.dispose()
HEAP USED =  10502816, ACTIVITY = new Frame()
HEAP USED =  15484248, ACTIVITY = Frame.dispose()
HEAP USED =  15508176, ACTIVITY = new Frame()
HEAP USED =  20419272, ACTIVITY = Frame.dispose()
HEAP USED =  20443136, ACTIVITY = new Frame()
HEAP USED =  25446552, ACTIVITY = Frame.dispose()
HEAP USED =  25445792, ACTIVITY = new Frame()
HEAP USED =  30425696, ACTIVITY = Frame.dispose()
HEAP USED =  30449576, ACTIVITY = new Frame()
Exception in thread "AWT-EventQueue-0" java.lang.OutOfMemoryError: Java heap space
        at XtoolkitIconLeak.action(XtoolkitIconLeak.java:151)
        at java.awt.Component.handleEvent(Component.java:6153)
        at java.awt.Window.postEvent(Window.java:2110)
        at java.awt.Component.postEvent(Component.java:4643)
        at java.awt.Component.dispatchEventImpl(Component.java:4362)
        at java.awt.Component.dispatchEvent(Component.java:4180)
        at java.awt.EventQueue.dispatchEvent(EventQueue.java:599)
        at java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:273)
        at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:183)
        at java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:173)
        at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:168)
        at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:160)
        at java.awt.EventDispatchThread.run(EventDispatchThread.java:121)


REPRODUCIBILITY :
This bug can be reproduced always.

---------- BEGIN SOURCE ----------
import java.awt.*;
import java.awt.image.*;
import java.awt.event.*;

public class XtoolkitIconLeak extends Frame {

    static boolean S_allowIcon = true;

    static int S_offset = 100;
    boolean S_etype = true;
    Button S_b = null;
    byte[] S_bigDummyData = null;

    public XtoolkitIconLeak(String name, boolean etype /* if true then exit app on windowClose() */){
        S_etype = etype;
        setTitle(name);
        /*--------------------------------------------------------------
         * Under Solaris 10 X86 if we intall an ICON in the frame, GC fails
         * and we leak 5M per window iteration (e.g. new frame and close)
         *   -Dawt.toolkit=sun.awt.X11.XToolkit
         * or
         *   unsetenv AWT_TOOLKIT
         */
        Toolkit kit = Toolkit.getDefaultToolkit();
        Image icon = kit.getImage("/opt/WebApps/Crome/local_root/crome/default/data/images/cr_icon.gif");
        if (S_allowIcon) {
            setIconImage(icon);
        }
        /*--------------------------------------------------------------------*/
        if (S_etype == true) {
            // only the primary window has a new frame button
            setBounds(0,0,250,100);
            setLayout(new GridLayout(1,1));
            S_b = new Button("create new frame");
            add(S_b);
        }
        addWindowListener(new WindowAdapter() {
            public void windowClosing(WindowEvent we){
                dispose();
                doGC(4,"Frame.dispose()");
                if (S_etype == true) {
                    // only the primary window invokes an application exits
                    System.exit(0);
                }
            }
        });
        setVisible(true);
    }

    public void doGC(int passes, String activity)
    {

        Runtime rt = Runtime.getRuntime();
        for (int i = 0; i< passes; i++) {
            System.gc();
            rt.gc();
            rt.runFinalization();
        }
        long free   = rt.freeMemory();
        long total  = rt.totalMemory();
        long used   = total - free;
        String sused = "" + used;
        while (sused.length() < 9) sused = " " + sused ;
        System.out.println("HEAP USED = " + sused + ", ACTIVITY = "  + activity);
    }

    public boolean action(Event event, Object arg)
    {
        if (event.target.equals(S_b)) {
            doGC(4,"new Frame()");
            XtoolkitIconLeak popup = new XtoolkitIconLeak("new sub frame",false);
            S_offset = S_offset + 22;
            popup.setBounds(S_offset, S_offset, 250,100);
            popup.S_bigDummyData = new byte[1000*1000*5];
        }
        return true;
    }

    public static void main(String args[]){
        System.out.println("awt.toolkit="+System.getProperty("awt.toolkit"));
        if (args.length > 0 && args[0].equalsIgnoreCase("NOICON")) {
            XtoolkitIconLeak.S_allowIcon = false;
            System.out.println("Not installing icons on frame via Frame.setIconImage()");
        }
        XtoolkitIconLeak Xtest = new XtoolkitIconLeak("main frame",true);
    }
}

---------- END SOURCE ----------

CUSTOMER SUBMITTED WORKAROUND :
Do not install an ICON, e.g. used the argument NOICON to the source code supplied as follows:

java -Xms32m -Xmx32m -Dawt.toolkit=sun.awt.X11.XToolkit XtoolkitIconLeak NOICON

this merely never calls Frame.setIconImage()