Bug ID: JDK-8041572 [macosx] huge native memory leak in AWTWindow.m

Type: Bug
Component: client-libs
Sub-Component: java.awt
Affected Version: 7u66,8,9

Priority: P2
Status: Resolved
Resolution: Fixed
OS: os_x
CPU: x86

Submitted: 2014-04-23
Updated: 2015-02-02
Resolved: 2014-04-25

JDK 7	JDK 8	JDK 9
7u75Fixed	8u11Fixed	9 b15Fixed

There is a huge native memory leak in awt code on Mac OS X. It was first identified by running NetBeans on JDK 8. The leak can be also reproduced using SwingSet2 with the following steps:
1) Run SwingSet2 demo on JDK 8u5 or JDK 9
2) move mouse over the SwingSet2 window - for example make circles over the window

the leak can be seen Activity Monitor and in details in vmmap output. The following output is captured, while moving mouse over  SwingSet2 window.
bobik:~ thurka$ jps
33784 SwingSet2.jar
33786 Jps
bobik:~ thurka$ date; vmmap 33784 | grep ^DefaultMallocZone
Wed Apr 23 09:53:52 CEST 2014
DefaultMallocZone_0x109d33000      48.2M      78842      32.8M     68%
bobik:~ thurka$ date; vmmap 33784 | grep ^DefaultMallocZone
Wed Apr 23 09:54:12 CEST 2014
DefaultMallocZone_0x109d33000     239.2M    3646723     217.8M     91%
bobik:~ thurka$ date; vmmap 33784 | grep ^DefaultMallocZone
Wed Apr 23 09:54:32 CEST 2014
DefaultMallocZone_0x109d33000     466.2M    7873686     437.3M     93%
bobik:~ thurka$ date; vmmap 33784 | grep ^DefaultMallocZone
Wed Apr 23 09:55:11 CEST 2014
DefaultMallocZone_0x109d33000     928.2M   16488624     883.2M     95%
bobik:~ thurka$ date; vmmap 33784 | grep ^DefaultMallocZone
Wed Apr 23 09:55:44 CEST 2014
DefaultMallocZone_0x109d33000       1.3G   23483264       1.2G     95%

As you can see DefaultMallocZone of SwingSet2 application raised to 1G in two minutes.

SQE OK to take the fix to CPU15_01
30-10-2014
7u Critical Request Template: -Justification: it's a huge mem leak in native part, without fix every java application with UI based on AWT will leak on Mac platform. And the leak is very big that and appears on every mouse movement. -Risk Analysis: low risk -Webrev: http://cr.openjdk.java.net/~mcherkas/8051263/webrev.00/ -Testing: it can be easily tested manually: run SwingSet2, move cursor over the SwingSet2 window. Without the fix amount of consumed memory is growing fast, with the fix application's mem amount doesn't change. -Back ports (done/to-be-done) : all backports were done - FX Impact: no impact
29-10-2014
Backport to 7u pending code review. I plan to add a (7u / 15_01 specific) critical request as soon as this code review is done.
29-10-2014
Tom�� H��rka added a comment - 2014-10-24 05:59 8u and 9 are already fixed (AFAICT). This memory leak was not reproducible in JDK 7 at the time it was filed against JDK 8u and JDK 9, but it looks like some backports to JDK 7u66 enabled this issue in JDK 7u66.
24-10-2014
Added critical watch for 15_01 for the 7u backport. We will work to propose this is fixed for 15_01 ASAP.
24-10-2014
SQE is OK to take this fix. Note: SQE makes this decision because we were told this is in Oracle's business interests. Otherwise I do not see why this should go into 8u11 instead of 8u20
12-05-2014
It's hard to provide a regression test because the memory leak is native and can't be detected without external tools or crashing the OS.
08-05-2014
Everything is ready - backports of both required fixes are the same as main fixes. As soon as inclusion into CPU approved I'll push it to 8u-cpu.
07-05-2014
I still interested in the fix is intergreated into 8uCPU. SQE would prefer to have nightly testing completed with the fix. BTW. Critical request label means the fix is ready and all the backports are ready.
07-05-2014
Critical request justification: Issue impact: prevents adoption of Netbeans 8, critically big memory leak. Fix risks: low, the fix is small and mac os x only. Must be back ported together with JDK-8041987 Webrev: http://cr.openjdk.java.net/~pchelko/9/8041572/webrev/ 9 changeset: http://hg.openjdk.java.net/jdk9/client/jdk/rev/63e1be3350fa
07-05-2014
The back port is the same as the original fix. This fix must be back ported together with JDK-8041987. I'll add the critical request label there too.
28-04-2014
Are the backports to 8u11 and 7u56 ready?
28-04-2014
Netbeans reports that due to this bug they are having to tell their customers to stay off JDK 8 and use JDK 7 instead. It would be good to fix this as soon as possible.
24-04-2014
Ok Petr, you can fix it this way. My main motivation for the fix with 'autorelease' method was simplicity. With explicit 'release' you have to cover all possible exits from getTopmostWindowUnderMouseID method.
23-04-2014
Thank you for the contribution, Tom��s. I'll send the fix for the review in a moment. But I think that here it's better to explicitly release the reference, because this method could be called on Appkit and the objects would leave until the next Runloop pass which is an unnecessary waste.
23-04-2014
The fix easy - windows variable just needs to be autoreleased. See attached patch. I have a custom JDK build with above patch and I was able to verify via vmmap output that the leak in getTopmostWindowUnderMouseID method is fixed by this patch.
23-04-2014
The issue is reproducible with 8 and 9. The fix is as simple as releasing the NSMutableArray upon exit from the getTopmostWindowUnderMouseID method.
23-04-2014
screenshot showing allocation stack trace of leaked objects.
23-04-2014
Running SwingSet2 demo under Instruments revealed that local variable NSMutableArray *windows in AWTWindow getTopmostWindowUnderMouseID is not properly released. See attached screenshot from Instruments.
23-04-2014