JDK-6897303 : awt_image_ImagingLib_transformBI ends up calling Runtime.gc although thread is s
  • Type: Bug
  • Component: client-libs
  • Sub-Component: java.awt
  • Affected Version: 1.4.2_21
  • Priority: P2
  • Status: Closed
  • Resolution: Fixed
  • OS: solaris_2.5.1
  • CPU: sparc
  • Submitted: 2009-11-02
  • Updated: 2011-03-04
  • Resolved: 2010-02-08
The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.
Other Other Other
1.4.2_25-rev b04Fixed 1.4.2_26-revFixed 1.4.2_27Fixed
Related Reports
Relates :  
JDK-4928472 - Should not call GC when AWT encounters an out of memory error
Description
SUN AWT Native code enters awt_image_ImagingLib_transformBI but ends up calling java/lang/Runtime.gc although thread is still in critical via GetPrimitiveArrayCritical - this breaks the JNI spec.

For JRockit, this causes a deadlock since there will be contention for the GC lock. GC thread is waiting for image thread to exit critical, but image thread calls java/lang/Runtime.gc() even though its still in critical.

Seems to involve the awt_NewHandler::handler(size_t) which is described as:


// Called when malloc or operator new runs out of memory. We try to
// compact the heap by initiating a Java GC. If the amount of free
// memory available after this operation increases, then we return
// (1) to indicate that malloc or operator new should retry the
// allocation. Returning (0) indicates that the allocation should fail.

I cannot reproduce this issue, I only have a .mdmp file from a customer. I can get this stack trace:

5d17f4d8 58b116a9 java!java/lang/Runtime.gc()V (Runtime.java) (+0x6f bytes)
5d17f508 005bf556 java!jrockit/vm/RNI.c2java(IIIII)V (RNI.java) (+0x49 bytes)
5d17f530 5b545378 jvm!stubsCallC2JavaStub+0x26 [d:\tendril4\work\install\src_pb_jrockitq_5462931\s\jvm\src\jvm\code\gen\lir\stubs_x86.c @ 66]
5d17f630 004c76a7 jvm!call_java+0xcd [d:\tendril4\work\install\src_pb_jrockitq_5462931\s\jvm\src\jvm\native\jni\jnimethod.c @ 234]
5d17f658 004ad173 jvm!jniCallIntMethodV+0x37 [d:\tendril4\work\install\src_pb_jrockitq_5462931\s\jvm\src\jvm\native\jni\jnimethod.c @ 498]
5d17f698 6c9808d5 jvm!xjniCallIntMethodV+0x43 [d:\tendril4\work\install\src_pb_jrockitq_5462931\s\jvm\src\jvm\native\jni\jni.c @ 199]
5d17f700 77bbd0a2 awt!Java_sun_awt_windows_WCanvasPeer_resetTargetGC+0xcb
5d17f718 6c9611ea msvcrt!malloc+0x82
5d17f738 6c94936d awt!Java_sun_awt_image_ImagingLib_init+0x182e5
5d17f764 6c9480a0 awt!Java_sun_awt_image_ImagingLib_init+0x468
5d17f7f8 6d2ba599 awt!Java_sun_awt_image_ImagingLib_transformBI+0x1e4
5d17f800 6d2ba376 java!sun/awt/image/ImagingLib.transformBI(Ljava/awt/image/BufferedImage;Ljava/awt/image/BufferedImage;[DI)I (ImagingLib.java) (+0xe9 bytes)
5d17f858 6d2b9f89 java!sun/awt/image/ImagingLib.filter(Ljava/awt/image/BufferedImageOp;Ljava/awt/image/BufferedImage;Ljava/awt/image/BufferedImage;)Ljava/awt/image/BufferedImage; (ImagingLib.java:202) (+0x206 bytes)
5d17f888 6d2b6ced java!java/awt/image/AffineTransformOp.filter(Ljava/awt/image/BufferedImage;Ljava/awt/image/BufferedImage;)Ljava/awt/image/BufferedImage; (AffineTransformOp.java:262) (+0x3b9 bytes)
5d17f8e8 6d2b5433 java!sun/java2d/pipe/DrawImage.transformImage(Lsun/java2d/SunGraphics2D;Ljava/awt/Image;IILjava/awt/image/BufferedImageOp;Ljava/awt/geom/AffineTransform;Ljava/awt/Color;)V (DrawImage.java:304) (+0xa8d bytes)
5d17f9a0 6d2b4fff java!sun/java2d/pipe/DrawImage.scaleImage(Lsun/java2d/SunGraphics2D;Ljava/awt/Image;IIIILjava/awt/Color;)Z (DrawImage.java:109) (+0x1e3 bytes)
5d17fa08 59a8d093 java!sun/java2d/pipe/DrawImage.scaleImage(Lsun/java2d/SunGraphics2D;Ljava/awt/Image;IIIILjava/awt/Color;Ljava/awt/image/ImageObserver;)Z (DrawImage.java:774) (+0x5f bytes)
5d17fa48 6d2b51f5 java!sun/java2d/SunGraphics2D.drawImage(Ljava/awt/Image;IIIILjava/awt/Color;Ljava/awt/image/ImageObserver;)Z (SunGraphics2D.java:2716) (+0x103 bytes)
5d17fa90 6d2b508e java!sun/awt/image/ImageRepresentation.drawToBufImage(Ljava/awt/Graphics;Lsun/awt/image/Image;IIIILjava/awt/Color;Ljava/awt/image/ImageObserver;)Z (ImageRepresentation.java:758) (+0x145 bytes)
5d17fad0 6d2b4f99 java!sun/java2d/pipe/DrawImage.scaleImage(Lsun/java2d/SunGraphics2D;Ljava/awt/Image;IIIILjava/awt/Color;Ljava/awt/image/ImageObserver;)Z (DrawImage.java:781) (+0xee bytes)
5d17fb10 59a8d093 java!sun/java2d/pipe/ValidatePipe.scaleImage(Lsun/java2d/SunGraphics2D;Ljava/awt/Image;IIIILjava/awt/Color;Ljava/awt/image/ImageObserver;)Z (ValidatePipe.java:162) (+0x79 bytes)
5d17fb48 59a8cf89 java!sun/java2d/SunGraphics2D.drawImage(Ljava/awt/Image;IIIILjava/awt/Color;Ljava/awt/image/ImageObserver;)Z (SunGraphics2D.java:2716) (+0x103 bytes)
5d17fb90 59a4324a java!sun/java2d/SunGraphics2D.drawImage(Ljava/awt/Image;IIIILjava/awt/image/ImageObserver;)Z (SunGraphics2D.java:2669) (+0x29 bytes)

Since i dont have the symbols for awt.dll, the:

awt!Java_sun_awt_image_ImagingLib_init references on the stack is wrong (only export symbols).

But it seems that somewhere from the entry awt!Java_sun_awt_image_ImagingLib_transformBI, several calls to *(env)->GetPrimitiveArrayCritical and *(env)->ReleasePrimitiveArrayCritical exist.

Somewhere in the code path I would assume there is an allocation being made to the heap which fails and as such triggers the code in awt_NewHandler::handler(size_t) EVEN though the code is in nested critical. This breaks the JNI specification.

I have also tried the 1.4.2_21 distribution of the JDK but the problem is still there.

If i can get the symbol files for either (or both) for awt.dll version:

1.4.2_21: Tue May 05 22:19:49 2009 (4A009F65)

1.4.2_17: Sat Feb 09 04:17:43 2008 (47AD1B57)

I could probably give you more information about which code path potentially have the unmatched *(env)->GetPrimitiveArrayCritical and *(env)->ReleasePrimitiveArrayCritical pair.

Summary:

Entry into Java_sun_awt_image_ImagingLib_transformBI ends up eventually calling java/lang/Runtime.gc() through awt_NewHandler::handler(size_t) EVEN though the thread is in critical. This breaks the JNI specification and causes different kinds of problems, deadlocking most prevalent.

Cannot reproduce, but I could extract more information if symbols for awt.dll (win_ia32) is provided by SUN.

Thank you
Markus Gronlund
Oracle JRockit Sustaining Engineering

Release Regression From : 1.4.2_21
The above release value was the last known release where this 
bug was not reproducible. Since then there has been a regression.
I guess problem seems to be not reproducible, submitter wants to see more with symbols.

http://jre.sfbay/java/re/j2sdk/1.4.2_21/promoted/fcs/b03/j4b/bundles/windows-i586/

this usually contains the symbols files as requested.
j2sdkfb-1_4_2_21-fcs-bin-b03-windows-i586-05_may_2009.zip
Comments
EVALUATION Concerning the stack trace, the submitter did note his stack trace was bogus due to lack of symbols, and I think the critical section isn't being taken lexically in the body of transformBI, but in one of the functions named "..._init" in the trace, that are actually called from transformBI. I haven't tracked down which one but there are certainly candidates. The line if ((status = (*sMlibFns[MLIB_AFFINE].fptr)(dst, src, mtx, filter, MLIB_EDGE_SRC_EXTEND) != MLIB_SUCCESS)) is one such candidate.
04-11-2009
EVALUATION The call to GC is from a C++ handler AWT install for awt.dll on windows. The code is in src/windows/native/sun/windows/awt_new.cpp It was fixed in JDk 1.6 as 4928472 : Should not call GC when AWT encounters an out of memory error This basically emptied out the method and throws OOME instead. The code in awt_ImagingLib.c is an innocent bystander here as it presumably just calls malloc, not expecting that to do any JNI work! Whilst we could perhaps move allocations outside of critical blocks here, that doesn't change that this is a bad thing for awt_new.cpp to be doing. I fact there's no indication in 4928472 that the problem was seen in the context of any imaging ops. transformBI is in medialib glue code in src/share/native/sun/awt/medialib/awt_ImagingLib.c And whilst transformBI does call Get/ReleasePrimitiveArrayCritical() I don't see malloc's - although I do see some calls to free(). The smoking gun has to include something like that. And the stack trace in the email is a bit questionable since transformBI does not call init(). : >> 5d17f738 6c94936d awt!Java_sun_awt_image_ImagingLib_init+0x182e5 >> 5d17f764 6c9480a0 awt!Java_sun_awt_image_ImagingLib_init+0x468 >> 5d17f7f8 6d2ba599 awt!Java_sun_awt_image_ImagingLib_transformBI+0x1e4
04-11-2009
WORK AROUND None
02-11-2009