Java Version: 1.6.0_10-beta-b13, 1.6.0_04-beta-b12, 1.6.0-b105
Platforms: solaris-sparc10

Problems:
I have a sample code which use RescaleOp to constructs a new RescaleOp with the desired scale factor and offset, and filter a BufferedImage with type of TYPE_3BYTE_BGR, it crashed.
I only tried above jdk6 releases.

Please see attached code and core dump log as followings:

import java.awt.image.BufferedImage;
import java.awt.image.RescaleOp;
import java.awt.*;

public class RescaleOpTest
{
  public static void main ( String[] args )
  {
        BufferedImage srcImage = new BufferedImage( 1, 1,BufferedImage.TYPE_3BYTE_BGR);
        srcImage.setRGB( 0, 0, Color.WHITE.getRGB() );
        Color srcColor = new Color(srcImage.getRGB( 0, 0 ));

        RescaleOp rop = new RescaleOp(1F, 0F, null);
        BufferedImage destImage = rop.filter(srcImage, null );       
  }
}

log file:
#
# An unexpected error has been detected by Java Runtime Environment:
#
#  SIGSEGV (0xb) at pc=0xf366a9f8, pid=8767, tid=2
#
# Java VM: Java HotSpot(TM) Client VM (11.0-b11 mixed mode solaris-sparc)
# Problematic frame:
# C  [libmlib_image_v.so+0xea9f8]
#
# If you would like to submit a bug report, please visit:
#   http://java.sun.com/webapps/bugreport/crash.jsp
# The crash happened outside the Java Virtual Machine in native code.
# See problematic frame for where to report the bug.
#

---------------  T H R E A D  ---------------

Current thread (0x00030400):  JavaThread "main" [_thread_in_native, id=2, stack(0xfe700000,0xfe780000)]

siginfo:si_signo=SIGSEGV: si_errno=0, si_code=1 (SEGV_MAPERR), si_addr=0x00000000

Registers:
 O0=0x00000002 O1=0x000000ff O2=0x00000001 O3=0x000000ff
 O4=0x00000008 O5=0x00000000 O6=0xfe77f688 O7=0x00000002
 G1=0x00000001 G2=0xf8000830 G3=0x0000000c G4=0x00000006
 G5=0x00000000 G6=0x00000000 G7=0xff362400 Y=0x00000000
 PC=0xf366a9f8 nPC=0xf366a9fc


Top of Stack: (sp=0xfe77f688)
0xfe77f688:   00000003 fe77f7c0 f4039936 00000000
0xfe77f698:   f4039a54 00000000 00000000 f4039a54
0xfe77f6a8:   f4035d6e 00000003 f4039936 00000003
0xfe77f6b8:   00000000 f4035d70 fe77f6f0 f35809a8
0xfe77f6c8:   00000003 f4035d6e fe77f6f8 f39b9c4c
0xfe77f6d8:   00080000 00000080 0055b508 00000000
0xfe77f6e8:   00000000 00000000 fee32000 00000000
0xfe77f6f8:   00000003 00000001 00016cf8 0000d000 

Instructions: (pc=0xf366a9f8)
0xf366a9e8:   ca 0f 60 01 ba 07 60 02 d2 0d 00 0b d2 2c a0 00
0xf366a9f8:   c2 0d 40 05 aa 10 00 14 a8 10 00 0d c2 2c a0 01 

Stack: [0xfe700000,0xfe780000],  sp=0xfe77f688,  free space=509k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
C  [libmlib_image_v.so+0xea9f8]
C  [libmlib_image_v.so+0x9b0]  j2d_mlib_ImageLookUp+0xf0
C  [libawt.so+0xb8330]  Java_sun_awt_image_ImagingLib_lookupByteRaster+0x101c
j  sun.awt.image.ImagingLib.lookupByteRaster(Ljava/awt/image/Raster;Ljava/awt/image/Raster;[[B)I+0
j  sun.awt.image.ImagingLib.lookupByteRaster(Ljava/awt/image/Raster;Ljava/awt/image/Raster;[[B)I+0
j  sun.awt.image.ImagingLib.filter(Ljava/awt/image/RasterOp;Ljava/awt/image/Raster;Ljava/awt/image/WritableRaster;)Ljava/awt/image/WritableRaster;+97
j  java.awt.image.LookupOp.filter(Ljava/awt/image/Raster;Ljava/awt/image/WritableRaster;)Ljava/awt/image/WritableRaster;+197
j  java.awt.image.RescaleOp.filter(Ljava/awt/image/Raster;Ljava/awt/image/WritableRaster;)Ljava/awt/image/WritableRaster;+223
j  java.awt.image.RescaleOp.filter(Ljava/awt/image/BufferedImage;Ljava/awt/image/BufferedImage;)Ljava/awt/image/BufferedImage;+501
j  RescaleOpTest.main([Ljava/lang/String;)V+76
v  ~StubRoutines::call_stub
V  [libjvm.so+0xd698c]
V  [libjvm.so+0x154b3c]
C  [java+0x3a54]

Java frames: (J=compiled Java code, j=interpreted, Vv=VM code)
j  sun.awt.image.ImagingLib.lookupByteRaster(Ljava/awt/image/Raster;Ljava/awt/image/Raster;[[B)I+0
j  sun.awt.image.ImagingLib.filter(Ljava/awt/image/RasterOp;Ljava/awt/image/Raster;Ljava/awt/image/WritableRaster;)Ljava/awt/image/WritableRaster;+97
j  java.awt.image.LookupOp.filter(Ljava/awt/image/Raster;Ljava/awt/image/WritableRaster;)Ljava/awt/image/WritableRaster;+197
j  java.awt.image.RescaleOp.filter(Ljava/awt/image/Raster;Ljava/awt/image/WritableRaster;)Ljava/awt/image/WritableRaster;+223
j  java.awt.image.RescaleOp.filter(Ljava/awt/image/BufferedImage;Ljava/awt/image/BufferedImage;)Ljava/awt/image/BufferedImage;+501
j  RescaleOpTest.main([Ljava/lang/String;)V+76
v  ~StubRoutines::call_stub

---------------  P R O C E S S  ---------------

Java Threads: ( => current thread )
  0x000cdc00 JavaThread "Low Memory Detector" daemon [_thread_blocked, id=8, stack(0xf3c00000,0xf3c80000)]
  0x000cb000 JavaThread "CompilerThread0" daemon [_thread_blocked, id=7, stack(0xf3d00000,0xf3d80000)]
  0x000c9400 JavaThread "Signal Dispatcher" daemon [_thread_blocked, id=6, stack(0xf3e00000,0xf3e80000)]
  0x000b3c00 JavaThread "Finalizer" daemon [_thread_blocked, id=5, stack(0xf3f00000,0xf3f80000)]
  0x000b2400 JavaThread "Reference Handler" daemon [_thread_blocked, id=4, stack(0xfe500000,0xfe580000)]
=>0x00030400 JavaThread "main" [_thread_in_native, id=2, stack(0xfe700000,0xfe780000)]

Other Threads:
  0x000aec00 VMThread [stack: 0xfc080000,0xfc100000] [id=3]
  0x000cf800 WatcherThread [stack: 0xf3b00000,0xf3b80000] [id=9]

VM state:not at safepoint (normal execution)

VM Mutex/Monitor currently owned by a thread: None

Heap
 def new generation   total 1984K, used 263K [0xf4000000, 0xf4220000, 0xf4710000)
  eden space 1792K,  14% used [0xf4000000, 0xf4041e58, 0xf41c0000)
  from space 192K,   0% used [0xf41c0000, 0xf41c0000, 0xf41f0000)
  to   space 192K,   0% used [0xf41f0000, 0xf41f0000, 0xf4220000)
 tenured generation   total 4096K, used 0K [0xf4710000, 0xf4b10000, 0xf8000000)
   the space 4096K,   0% used [0xf4710000, 0xf4710000, 0xf4710200, 0xf4b10000)
 compacting perm gen  total 12288K, used 2194K [0xf8000000, 0xf8c00000, 0xfc000000)
   the space 12288K,  17% used [0xf8000000, 0xf8224860, 0xf8224a00, 0xf8c00000)
No shared spaces configured.

Dynamic libraries:
0x00010000 	/net/jre.sfbay/p/v06/jdk/6_10/beta/b13/binaries/solaris-sparc/bin/java
0xff3f8000 	/lib/libthread.so.1
0xff380000 	/net/jre.sfbay/p/v06/jdk/6_10/beta/b13/binaries/solaris-sparc/bin/../jre/lib/sparc/jli/libjli.so
0xff3a0000 	/lib/libdl.so.1
0xff200000 	/lib/libc.so.1
0xff358000 	/platform/SUNW,Ultra-5_10/lib/libc_psr.so.1
0xfe800000 	/net/jre.sfbay/p/v06/jdk/6_10/beta/b13/binaries/solaris-sparc/jre/lib/sparc/client/libjvm.so
0xff320000 	/lib/libsocket.so.1
0xff350000 	/usr/lib/libsched.so.1
0xff1e0000 	/lib/libm.so.1
0xff1b0000 	/usr/lib/libCrun.so.1
0xff190000 	/lib/libdoor.so.1
0xff080000 	/lib/libnsl.so.1
0xfef80000 	/lib/libm.so.2
0xff160000 	/lib/libscf.so.1
0xff140000 	/lib/libuutil.so.1
0xff050000 	/lib/libmd5.so.1
0xfef60000 	/platform/SUNW,Ultra-5_10/lib/libmd5_psr.so.1
0xfef40000 	/lib/libmp.so.2
0xfeed0000 	/net/jre.sfbay/p/v06/jdk/6_10/beta/b13/binaries/solaris-sparc/jre/lib/sparc/native_threads/libhpi.so
0xfee80000 	/net/jre.sfbay/p/v06/jdk/6_10/beta/b13/binaries/solaris-sparc/jre/lib/sparc/libverify.so
0xfe7c0000 	/net/jre.sfbay/p/v06/jdk/6_10/beta/b13/binaries/solaris-sparc/jre/lib/sparc/libjava.so
0xfe7a0000 	/net/jre.sfbay/p/v06/jdk/6_10/beta/b13/binaries/solaris-sparc/jre/lib/sparc/libzip.so
0xf3900000 	/net/jre.sfbay/p/v06/jdk/6_10/beta/b13/binaries/solaris-sparc/jre/lib/sparc/libawt.so
0xfe5a0000 	/net/jre.sfbay/p/v06/jdk/6_10/beta/b13/binaries/solaris-sparc/jre/lib/sparc/xawt/libmawt.so
0xfe6a0000 	/usr/openwin//lib/libXext.so.0
0xf3800000 	/usr/openwin//lib/libX11.so.4
0xfe4d0000 	/usr/openwin//lib/libXtst.so.1
0xfe4a0000 	/usr/openwin//lib/libXi.so.5
0xf3580000 	/net/jre.sfbay/p/v06/jdk/6_10/beta/b13/binaries/solaris-sparc/jre/lib/sparc/libmlib_image_v.so

VM Arguments:
java_command: RescaleOpTest
Launcher Type: SUN_STANDARD

Environment Variables:
PATH=.:/java/devtools/sparc/mercurial/0.9.3/bin:/usr/j2se/bin:/usr/bin:/sbin:/usr/sbin:/usr/X/bin:/usr/ccs:/usr/ccs/bin:/usr/lib:/usr/ucb:/usr/dist/local/exe:/usr/dist/exe:/usr/local/bin:/bin:/usr/dist/share/devpro,v4.2/5.x-sparc/bin:/usr/bin/X11:/usr/etc:/usr/etc/yp:/usr/openwin:/usr/lib/nis:/ws/on81-tools/SUNWspro/SC6.1/bin:/usr/lib/netsvc/yp:/usr/dist/pkgs/devpro,v4.2/5.x-sparc/bin
LD_LIBRARY_PATH=/net/jre.sfbay/p/v06/jdk/6_10/beta/b13/binaries/solaris-sparc/jre/lib/sparc/client:/net/jre.sfbay/p/v06/jdk/6_10/beta/b13/binaries/solaris-sparc/jre/lib/sparc:/net/jre.sfbay/p/v06/jdk/6_10/beta/b13/binaries/solaris-sparc/jre/../lib/sparc
SHELL=/bin/csh
DISPLAY=:0.0

Signal Handlers:
SIGSEGV: [libjvm.so+0x4fd4d0], sa_mask[0]=0xffbffeff, sa_flags=0x0000000c
SIGBUS: [libjvm.so+0x4fd4d0], sa_mask[0]=0xffbffeff, sa_flags=0x0000000c
SIGFPE: [libjvm.so+0x18a790], sa_mask[0]=0xffbffeff, sa_flags=0x0000000c
SIGPIPE: [libjvm.so+0x18a790], sa_mask[0]=0xffbffeff, sa_flags=0x0000000c
SIGXFSZ: [libjvm.so+0x18a790], sa_mask[0]=0xffbffeff, sa_flags=0x0000000c
SIGILL: [libjvm.so+0x18a790], sa_mask[0]=0xffbffeff, sa_flags=0x0000000c
SIGUSR1: SIG_DFL, sa_mask[0]=0x00000000, sa_flags=0x00000000
SIGUSR2: SIG_DFL, sa_mask[0]=0x00000000, sa_flags=0x00000000
SIGQUIT: [libjvm.so+0x41d7c8], sa_mask[0]=0xffbffeff, sa_flags=0x00000004
SIGHUP: [libjvm.so+0x41d7c8], sa_mask[0]=0xffbffeff, sa_flags=0x00000004
SIGINT: [libjvm.so+0x41d7c8], sa_mask[0]=0xffbffeff, sa_flags=0x00000004
SIGTERM: [libjvm.so+0x41d7c8], sa_mask[0]=0xffbffeff, sa_flags=0x00000004
SIG39: [libjvm.so+0x420990], sa_mask[0]=0x00000000, sa_flags=0x00000008
SIG40: [libjvm.so+0x18a790], sa_mask[0]=0xffbffeff, sa_flags=0x0000000c


---------------  S Y S T E M  ---------------

OS:                         Solaris 10 3/05 s10_74L2a SPARC
           Copyright 2005 Sun Microsystems, Inc.  All Rights Reserved.
                        Use is subject to license terms.
                            Assembled 22 January 2005

uname:SunOS 5.10 Generic_118833-36 sun4u  (T2 libthread)
rlimit: STACK 8192k, CORE infinity, NOFILE 65536, AS infinity
load average:0.30 0.26 0.16

CPU:total 1 has_v8, has_v9, has_vis1

Memory: 8k page, physical 524288k(51704k free)

vm_info: Java HotSpot(TM) Client VM (11.0-b11) for solaris-sparc JRE (1.6.0_10-beta-b13), built on Mar  6 2008 02:08:19 by "" with Workshop 5.8

time: Mon Mar 10 19:11:48 2008
elapsed time: 2 seconds