JDK-6501871 : java.awt.LookupOp may crash on BufferedImage with alpha
  • Type: Bug
  • Component: client-libs
  • Sub-Component: 2d
  • Affected Version: 5.0,6u3,6u10,6u14-rev
  • Priority: P3
  • Status: Closed
  • Resolution: Fixed
  • OS:
    linux_suse_sles_10,windows_xp,windows_7 linux_suse_sles_10,windows_xp,windows_7
  • CPU: x86
  • Submitted: 2006-12-07
  • Updated: 2012-10-01
  • Resolved: 2011-03-07
The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.
Other JDK 6 JDK 7
5.0u23Fixed 6u18Fixed 7 b08Fixed
Related Reports
Duplicate :  
JDK-6652168 - sun/java2d/OpenGL/DrawBufImgOp.java crashes in the libmlib_image
Duplicate :  
JDK-6840207 - two java/class_2d tests failed just on Windows7 RC build 7100 build against 1.6.0_14_b06
Duplicate :  
JDK-6627423 - Call to LookupOp.filter after ColorConvertOp.filter crashes the VM
Description
FULL PRODUCT VERSION :
java version "1.5.0_01"
Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_01-b08)
Java HotSpot(TM) Client VM (build 1.5.0_01-b08, mixed mode, sharing)

ADDITIONAL OS VERSION INFORMATION :
Windows SP2

A DESCRIPTION OF THE PROBLEM :
LookupOp does not work with BufferedImage has alpha-value.

EXPECTED VERSUS ACTUAL BEHAVIOR :
EXPECTED -
LookupOp should return filtered Image.
ACTUAL -
JVM Crash.

ERROR MESSAGES/STACK TRACES THAT OCCUR :
#
# An unexpected error has been detected by HotSpot Virtual Machine:
#
#  EXCEPTION_ACCESS_VIOLATION (0xc0000005) at pc=0x6d0f59f8, pid=2128, tid=112
#
# Java VM: Java HotSpot(TM) Client VM (1.5.0_01-b08 mixed mode, sharing)
# Problematic frame:
# C  [awt.dll+0x859f8]
#

---------------  T H R E A D  ---------------

Current thread (0x00037cb0):  JavaThread "main" [_thread_in_native, id=112]

siginfo: ExceptionCode=0xc0000005, reading address 0x00000000

Registers:
EAX=0x22bdc23f, EBX=0x00000000, ECX=0x03039d34, EDX=0x00000001
ESP=0x0007f7b0, EBP=0x0007f7e8, ESI=0x00000004, EDI=0x00000000
EIP=0x6d0f59f8, EFLAGS=0x00010202

  Top of Stack: (sp=0x0007f7b0)
0x0007f7b0:   00000001 00000004 00000001 00000001
0x0007f7c0:   0307e300 0307ea14 fffffe00 6d0acb8e
0x0007f7d0:   00000001 00000001 00000004 00000001
0x0007f7e0:   00000001 6d0aceb3 0007f834 6d0aceee
0x0007f7f0:   22bdc03c 00000004 22bdc23c 00000004
0x0007f800:   00000001 22bdc23f 00000001 03039d28
0x0007f810:   00000003 00000001 00037d6c 00000001
0x0007f820:   00000004 00000001 22bdc03c 00000004

Instructions: (pc=0x6d0f59f8)
0x6d0f59e8:   8b 39 85 d2 7e 16 89 55 18 8b 5d e0 0f b6 1c 03
0x6d0f59f8:   8a 1c 3b 88 18 03 c6 ff 4d 18 75 ed 83 c1 04 ff


Stack: [0x00040000,0x00080000),  sp=0x0007f7b0,  free space=253k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
C  [awt.dll+0x859f8]
C  [awt.dll+0x3ceee]
C  [awt.dll+0x3920a]
j  sun.awt.image.ImagingLib.lookupByteBI(Ljava/awt/image/BufferedImage;Ljava/awt/image/BufferedImage;[[B)I+0
j  sun.awt.image.ImagingLib.filter(Ljava/awt/image/BufferedImageOp;Ljava/awt/image/BufferedImage;Ljava/awt/image/BufferedImage;)Ljava/awt/image/BufferedImage;+145
j  java.awt.image.LookupOp.filter(Ljava/awt/image/BufferedImage;Ljava/awt/image/BufferedImage;)Ljava/awt/image/BufferedImage;+283
j  LookupOpTest.filter(Ljava/awt/image/BufferedImageOp;ILjava/lang/String;)V+12
j  LookupOpTest.main([Ljava/lang/String;)V+55
v  ~StubRoutines::call_stub
V  [jvm.dll+0x8176e]
V  [jvm.dll+0xd481d]
V  [jvm.dll+0x8163f]
V  [jvm.dll+0x885cd]
C  [java.exe+0x14c0]
C  [java.exe+0x64cd]
C  [kernel32.dll+0x16fd7]

Java frames: (J=compiled Java code, j=interpreted, Vv=VM code)
j  sun.awt.image.ImagingLib.lookupByteBI(Ljava/awt/image/BufferedImage;Ljava/awt/image/BufferedImage;[[B)I+0
j  sun.awt.image.ImagingLib.filter(Ljava/awt/image/BufferedImageOp;Ljava/awt/image/BufferedImage;Ljava/awt/image/BufferedImage;)Ljava/awt/image/BufferedImage;+145
j  java.awt.image.LookupOp.filter(Ljava/awt/image/BufferedImage;Ljava/awt/image/BufferedImage;)Ljava/awt/image/BufferedImage;+283
j  LookupOpTest.filter(Ljava/awt/image/BufferedImageOp;ILjava/lang/String;)V+12
j  LookupOpTest.main([Ljava/lang/String;)V+55
v  ~StubRoutines::call_stub

---------------  P R O C E S S  ---------------

Java Threads: ( => current thread )
  0x03062d98 JavaThread "Java2D Disposer" daemon [_thread_blocked, id=1832]
  0x00a7fd70 JavaThread "Low Memory Detector" daemon [_thread_blocked, id=2648]
  0x00a7e948 JavaThread "CompilerThread0" daemon [_thread_blocked, id=1960]
  0x00a7dc90 JavaThread "Signal Dispatcher" daemon [_thread_blocked, id=1724]
  0x00a790b8 JavaThread "Finalizer" daemon [_thread_blocked, id=3388]
  0x00a53ba8 JavaThread "Reference Handler" daemon [_thread_blocked, id=3932]
=>0x00037cb0 JavaThread "main" [_thread_in_native, id=112]

Other Threads:
  0x00a772e8 VMThread [id=3204]
  0x00a80f80 WatcherThread [id=2384]

VM state:not at safepoint (normal execution)

VM Mutex/Monitor currently owned by a thread: None

Heap
 def new generation   total 576K, used 115K [0x22bd0000, 0x22c70000, 0x230b0000)
  eden space 512K,  10% used [0x22bd0000, 0x22bdcdc0, 0x22c50000)
  from space 64K, 100% used [0x22c50000, 0x22c60000, 0x22c60000)
  to   space 64K,   0% used [0x22c60000, 0x22c60000, 0x22c70000)
 tenured generation   total 1408K, used 273K [0x230b0000, 0x23210000, 0x26bd0000)
   the space 1408K,  19% used [0x230b0000, 0x230f4648, 0x230f4800, 0x23210000)
 compacting perm gen  total 8192K, used 362K [0x26bd0000, 0x273d0000, 0x2abd0000)
   the space 8192K,   4% used [0x26bd0000, 0x26c2abd0, 0x26c2ac00, 0x273d0000)
    ro space 8192K,  66% used [0x2abd0000, 0x2b128580, 0x2b128600, 0x2b3d0000)
    rw space 12288K,  46% used [0x2b3d0000, 0x2b964aa8, 0x2b964c00, 0x2bfd0000)

Dynamic libraries:
0x00400000 - 0x0040c000 	C:\jdk1.5.0_01\bin\java.exe
0x7c940000 - 0x7c9dd000 	C:\WINDOWS\system32\ntdll.dll
0x7c800000 - 0x7c931000 	C:\WINDOWS\system32\kernel32.dll
0x77d80000 - 0x77e29000 	C:\WINDOWS\system32\ADVAPI32.dll
0x77e30000 - 0x77ec1000 	C:\WINDOWS\system32\RPCRT4.dll
0x77bc0000 - 0x77c18000 	C:\WINDOWS\system32\MSVCRT.dll
0x6d6b0000 - 0x6d835000 	C:\jdk1.5.0_01\jre\bin\client\jvm.dll
0x77cf0000 - 0x77d7f000 	C:\WINDOWS\system32\USER32.dll
0x77ed0000 - 0x77f17000 	C:\WINDOWS\system32\GDI32.dll
0x76af0000 - 0x76b1b000 	C:\WINDOWS\system32\WINMM.dll
0x762e0000 - 0x762fd000 	C:\WINDOWS\system32\IMM32.DLL
0x60740000 - 0x60749000 	C:\WINDOWS\system32\LPK.DLL
0x73f80000 - 0x73feb000 	C:\WINDOWS\system32\USP10.dll
0x5a820000 - 0x5a827000 	C:\WINDOWS\system32\serwvdrv.dll
0x58a60000 - 0x58a67000 	C:\WINDOWS\system32\umdmxfrm.dll
0x6d2f0000 - 0x6d2f8000 	C:\jdk1.5.0_01\jre\bin\hpi.dll
0x76ba0000 - 0x76bab000 	C:\WINDOWS\system32\PSAPI.DLL
0x6d680000 - 0x6d68c000 	C:\jdk1.5.0_01\jre\bin\verify.dll
0x6d370000 - 0x6d38d000 	C:\jdk1.5.0_01\jre\bin\java.dll
0x6d6a0000 - 0x6d6af000 	C:\jdk1.5.0_01\jre\bin\zip.dll
0x6d070000 - 0x6d1d6000 	C:\jdk1.5.0_01\jre\bin\awt.dll
0x72f50000 - 0x72f76000 	C:\WINDOWS\system32\WINSPOOL.DRV
0x76970000 - 0x76aad000 	C:\WINDOWS\system32\ole32.dll
0x6d2b0000 - 0x6d2ed000 	C:\jdk1.5.0_01\jre\bin\fontmanager.dll

VM Arguments:
java_command: LookupOpTest

Environment Variables:
JAVA_HOME=C:\jdk1.5.0_01
CLASSPATH=;.;../../library/lib/fop\batik.jar;../../library/lib/fop\xalan-2.4.1.jar;../../library/lib/fop\xercesImpl-2.2.1.jar;../../library/lib/fop\xml-apis.jar;../../library/lib/fop\avalon-framework-cvs-20020806.jar;../../library/lib/fop\fop.jar;../../library/lib\jdbc\ojdbc14.jar;../../library/lib\jdbc\mysql-connector-java-3.1.12-bin.jar;../../library/lib\ju.jar;../../library/lib\mwizard.jar;../../library/lib\time.jar;../../library/lib\text.jar;../../library/lib\mutil1.jar;../../library/lib\mutil2.jar;../../library/lib\mutil3.jar;../../library/lib\mutil4.jar;../../library/lib\table.jar;../../library/lib\image-editor.jar;C:/iText\itext-1.4.jar;../../library/lib\stamp.jar;../../library/lib\draw.jar;../../library/lib\Capi.jar
PATH=c:\program files\imagemagick-6.2.7-q16;C:\jdk1.5.0_01\bin;C:\oracle\ora90\bin;C:\oracle\ora90\Apache\Perl\5.00503\bin\mswin32-x86;C:\Program Files\Oracle\jre\1.1.8\bin;C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\System32\Wbem;C:\Program Files\Common Files\Adaptec Shared\System;C:\DownLoaded\ANT\apache-ant-1.5.4\bin;C:\mysql\bin;.;C:\MySQL Server 5.0\bin;C:\jdk1.5.0_01\bin;C:\oracle\ora90\bin;C:\oracle\ora90\Apache\Perl\5.00503\bin\mswin32-x86;C:\Program Files\Oracle\jre\1.1.8\bin;C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\System32\Wbem;C:\Program Files\Common Files\Adaptec Shared\System;C:\DownLoaded\ANT\apache-ant-1.5.4\bin;C:\mysql\bin;.
USERNAME=itoh
OS=Windows_NT
PROCESSOR_IDENTIFIER=x86 Family 15 Model 2 Stepping 7, GenuineIntel


---------------  S Y S T E M  ---------------

OS: Windows XP Build 2600 Service Pack 2

CPU:total 1 family 15, cmov, cx8, fxsr, mmx, sse, sse2, ht

Memory: 4k page, physical 646644k(154640k free), swap 1583428k(998380k free)

vm_info: Java HotSpot(TM) Client VM (1.5.0_01-b08) for windows-x86, built on Dec  6 2004 19:51:00 by "java_re" with MS VC++ 6.0



REPRODUCIBILITY :
This bug can be reproduced always.

---------- BEGIN SOURCE ----------


import java.awt.*;

import java.awt.image.*;

import javax.imageio.ImageIO;

class LookupOpTest{
	
	public static LookupTable createInverseGrayTable(int offset, int brightness){
		byte[][] data = new byte[3][256];
		for(int i = 0; i < 256; i++){
			data[0][i] = (byte)(255 - i);
			data[1][i] = (byte)(255 - i);
			data[2][i] = (byte)(255 - i);
		}
		return new ByteLookupTable(offset, adjustBytes(data, brightness));
	}
	
	private static byte[][] adjustBytes(byte[][] data, int brightness){
		if(data.length != 3) throw new IllegalArgumentException("data.length != 3");
		if(data[0].length != data[1].length || data[1].length != data[2].length){
			throw new IllegalArgumentException("data[0].length != data[1].length || data[1].length != data[2].length");
		}
		if(brightness > 255) brightness = 255;
		else if(brightness < -255) brightness = -255;
		
		for(int i = 0; i < data[0].length; i++){
			int red   = (int)data[0][i]&0xFF;
			int green = (int)data[1][i]&0xFF;
			int blue  = (int)data[2][i]&0xFF;
			data[0][i] = adjustByte(red   + brightness);
			data[1][i] = adjustByte(green + brightness);
			data[2][i] = adjustByte(blue  + brightness);
		}
		return data;
	}
	
	private static byte adjustByte(int value){
		if(value > 255)		return (byte)255;
		else if(value < 0)	return (byte)0;
		return (byte)value;
	}
	
	private static void filter(BufferedImageOp imageOp, int type, String typeStr){
		imageOp.filter(new BufferedImage(1, 1, type), null);
		System.out.println(typeStr + " Done");
	}
	
	public static void main(String[] args){
		
		BufferedImageOp imageOp	= new LookupOp(createInverseGrayTable(0, 0), null);
		//These codes filter Image.
		filter(imageOp, BufferedImage.TYPE_3BYTE_BGR, "TYPE_3BYTE_BGR");
		filter(imageOp, BufferedImage.TYPE_INT_BGR, "TYPE_INT_BGR");
		filter(imageOp, BufferedImage.TYPE_INT_RGB, "TYPE_INT_RGB");
		filter(imageOp, BufferedImage.TYPE_USHORT_555_RGB, "TYPE_USHORT_555_RGB");
		filter(imageOp, BufferedImage.TYPE_USHORT_565_RGB, "TYPE_USHORT_565_RGB");
		
		
		/* These codes throw java.lang.IllegalArgumentException cause of Index Color Model.This is expected operation.
		filter(imageOp, BufferedImage.TYPE_BYTE_BINARY, "TYPE_BYTE_BINARY");
		filter(imageOp, BufferedImage.TYPE_BYTE_INDEXED, "TYPE_BYTE_INDEXED");
		*/
		
		/* These codes throw java.lang.IllegalArgumentException cause of argument miss.This is expected operation.
		filter(imageOp, BufferedImage.TYPE_BYTE_GRAY, "TYPE_BYTE_GRAY");
		filter(imageOp, BufferedImage.TYPE_USHORT_GRAY, "TYPE_USHORT_GRAY");
		*/
		
		
		//These BufferedImage has alpha-value make JVM down.
		//filter(imageOp, BufferedImage.TYPE_4BYTE_ABGR, "TYPE_4BYTE_ABGR");
		//filter(imageOp, BufferedImage.TYPE_4BYTE_ABGR_PRE, "TYPE_4BYTE_ABGR_PRE");
		//filter(imageOp, BufferedImage.TYPE_INT_ARGB_PRE, "TYPE_INT_ARGB_PRE");
		filter(imageOp, BufferedImage.TYPE_INT_ARGB, "TYPE_INT_ARGB");
	}
}

---------- END SOURCE ----------

CUSTOMER SUBMITTED WORKAROUND :
No use BufferedImage has alpha channel.
Comments
EVALUATION The LookupOp spec says that if number of lookup arrays in the lookup table is one or equal to number of color components, then lookup of the alpha component is not performed. Reported testcase has lookup table with 3 lookup arrays and 4-component (3 colors + alpha) image. So, according to the rule we should not perform lookip for alpha. To avoid this alpha lookup medialib glue code inserts identity lookup array for alpha component into the lookup table. However, due to mistake in the condition statement, this does not happen if lookup table has 3 lookup arrays (one per color component). As the result null pointer is dereferenced and this leads to crash. Fix is to make sure that lookup table is properly initialized with identity lookup array for alpha in cases like this.
08-12-2006