JDK-7151427 : Potential memory leak in error handling code in X11SurfaceData.c
  • Type: Bug
  • Component: client-libs
  • Sub-Component: 2d
  • Affected Version: 7
  • Priority: P4
  • Status: Resolved
  • Resolution: Fixed
  • OS: linux
  • CPU: x86
  • Submitted: 2012-03-06
  • Updated: 2013-11-27
  • Resolved: 2012-03-27
The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.
JDK 6 JDK 7 JDK 8
6u71Fixed 7u40Fixed 8 b32Fixed
Description
FULL PRODUCT VERSION :
java version "1.7.0_01"
Java(TM) SE Runtime Environment (build 1.7.0_01-b08)
Java HotSpot(TM) Client VM (build 21.1-b02, mixed mode)

ADDITIONAL OS VERSION INFORMATION :
Linux zhouyx-workstation 3.0.0-16-generic #28-Ubuntu SMP Fri Jan 27 17:50:54 UTC 2012 i686 i686 i386 GNU/Linux

A DESCRIPTION OF THE PROBLEM :

In X11SurfaceData.c, when memory allocation failed, the error handling code does not free the memory already allocated.

eg.
In following piece of code, if "shminfo->shmid = shmget(IPC_PRIVATE, height * img->bytes_per_line, PC_CREAT|0777); "  failed,  both "img" and "shminfo" are not freed.
 
////////////////////////////////////////////////////////////
    shminfo = malloc(sizeof(XShmSegmentInfo));
    if (shminfo == NULL) {
        return NULL;
    }
    memset(shminfo, 0, sizeof(XShmSegmentInfo));

    img = XShmCreateImage(awt_display, xsdo->configData->awt_visInfo.visual,
                          xsdo->depth, ZPixmap, NULL, shminfo,
                          width, height);
    if (img == NULL) {
        free((void *)shminfo);
        return NULL;
    }
    shminfo->shmid =
        shmget(IPC_PRIVATE, height * img->bytes_per_line, IPC_CREAT|0777);
    if (shminfo->shmid < 0) {
        J2dRlsTraceLn1(J2D_TRACE_ERROR,
                       "X11SD_SetupSharedSegment shmget has failed: %s",
                       strerror(errno));
        return NULL;
    }
////////////////////////////////////////////






STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
It is not easy to reproduce, we observed the memory leak only on one machine because it exists in error handling code and app rarely goes there.

However, this bug is obvious when looking at the code.

EXPECTED VERSUS ACTUAL BEHAVIOR :
EXPECTED -
All allocated memory should be freed in error handling code.
ACTUAL -
Some allocated memory are not freed.

REPRODUCIBILITY :
This bug can be reproduced occasionally.

Comments
EVALUATION Fixed in changeset URL: http://hg.openjdk.java.net/jdk8/2d/jdk/rev/b1af41b86f9f Email thread here : http://mail.openjdk.java.net/pipermail/2d-dev/2012-March/002397.html
12-03-2012