United StatesChange Country, Oracle Worldwide Web Sites Communities I am a... I want to...
JDK-4950176 : drawImage bad interpolation for non-opaque INT_ARGB pixels

Details
Type:
Bug
Submit Date:
2003-11-06
Status:
Resolved
Updated Date:
2005-01-20
Project Name:
JDK
Resolved Date:
2005-01-20
Component:
client-libs
OS:
windows_2000
Sub-Component:
2d
CPU:
x86
Priority:
P3
Resolution:
Fixed
Affected Versions:
1.4.2
Fixed Versions:

Related Reports
Relates:

Sub Tasks

Description

Name: rmT116609			Date: 11/06/2003


FULL PRODUCT VERSION :
java version "1.4.2_02"
Java(TM) 2 Runtime Environment, Standard Edition (build 1.4.2
Java HotSpot(TM) Client VM (build 1.4.2_02-b03, mixed mode)

ADDITIONAL OS VERSION INFORMATION :
Microsoft Windows 2000 [Versione 5.00.2195]

A DESCRIPTION OF THE PROBLEM :
I'm making deep use of java2d (I simply love it) for the creation of animated desktop video graphics and video titling.
 
During the development of my applications I have discovered a bug in drawImage() http://developer.java.sun.com/developer/bugParade/bugs/4916948.html that causes incorrect results when drawing images with a translation-only AffineTransform. I also found a workaround for the problem, but I still hope that SUN fixes it asap.

drawImage() bilinear interpolation gives ugly results on ARGB images.
 
consider the 2x2 bitmap pixels :
 
  X0
  00
 
where X is completely transparent (let's say a transparent RED -> 0x00FF0000) and 0 is opaque (let's say an opaque WHITE -> 0xFFFFFFFF).
 
The pixel bilinear interpolation takes the color of the X pixel into consideration. That's completely wrong because the pixel is transparent. When drawing the bitmap at 0.5,0.5 over a clean graphics (0x00000000) the pixel interpolator of java2d gives ugly red artifacts.
 

STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
consider the 2x2 bitmap pixels :
 
  X0
  00
 
where X is completely transparent (let's say a transparent RED -> 0x00FF0000) and 0 is opaque (let's say an opaque WHITE -> 0xFFFFFFFF).
 
The pixel bilinear interpolation takes the color of the X pixel into consideration. That's completely wrong because the pixel is transparent. When drawing the bitmap at 0.5,0.5 over a clean graphics (0x00000000) the pixel interpolator of java2d gives ugly red artifacts.

EXPECTED VERSUS ACTUAL BEHAVIOR :
EXPECTED -
Red artifacts should not appear.

Take a look at
http://www.classx.it/public/bilerp_test.png
http://www.classx.it/public/bilerp_test1.png
http://www.classx.it/public/bilerp_test.jar

Note that MacOSX Java 1.4.1 works correctly.
ACTUAL -
Ugly red artifacts.

REPRODUCIBILITY :
This bug can be reproduced always.

---------- BEGIN SOURCE ----------
package test.bug;

import java.awt.*;
import java.awt.geom.*;
import java.awt.image.*;

public class ImageInterpolationBug extends Panel implements java.awt.event.MouseMotionListener
{
	// storage
	private final static int WIDTH = 250;
	private final static int HEIGHT = 90;
	private static int[] pix = new int[WIDTH * HEIGHT];
	private static BufferedImage src = createBufferedImage(pix,WIDTH, HEIGHT, WIDTH,0);
	private float mx = 0, my = 0;

	// create some test gfx
	static
	{
		// the rendering hints
		java.util.HashMap map = new java.util.HashMap();
		map.put(RenderingHints.KEY_ALPHA_INTERPOLATION,	RenderingHints.VALUE_ALPHA_INTERPOLATION_QUALITY);
		map.put(RenderingHints.KEY_TEXT_ANTIALIASING,  	RenderingHints.VALUE_TEXT_ANTIALIAS_ON);
		map.put(RenderingHints.KEY_INTERPOLATION,    	RenderingHints.VALUE_INTERPOLATION_BILINEAR);
		map.put(RenderingHints.KEY_DITHERING,	    	RenderingHints.VALUE_DITHER_ENABLE);
		map.put(RenderingHints.KEY_FRACTIONALMETRICS,	RenderingHints.VALUE_FRACTIONALMETRICS_ON);
		map.put(RenderingHints.KEY_ANTIALIASING,		RenderingHints.VALUE_ANTIALIAS_ON);
		map.put(RenderingHints.KEY_RENDERING,			RenderingHints.VALUE_RENDER_QUALITY);
		map.put(RenderingHints.KEY_COLOR_RENDERING,		RenderingHints.VALUE_COLOR_RENDER_QUALITY);

		// set bitmap to all transparent red
		for(int i=0; i<WIDTH*HEIGHT; i++)
		{
			pix[i]=0x00FF0000;
		}

		// the graphics
		Graphics2D g2 = (Graphics2D) src.getGraphics();
		g2.setRenderingHints(map);
		g2.setComposite(AlphaComposite.Src);

		// draw a glyph (this creates a correct test image)
		g2.setFont(new Font("dialog",-1,80));
		java.awt.font.GlyphVector gv = g2.getFont().createGlyphVector(g2.getFontRenderContext(),"J2D");
		g2.translate(10,70);
		g2.setColor(Color.gray);
		g2.fill(gv.getOutline());
		g2.translate(2,2);
		g2.setColor(Color.black);
		g2.fill(gv.getOutline());
	}

	// create the dest raster
	BufferedImage raster = createBufferedImage(new int[640*400],640,400, 640,0);
	public ImageInterpolationBug()
	{
		super();
		setBackground(Color.gray);
	}
	// utility: create a new INT_ARGB bufferedimage with rowints support
	private static final BufferedImage createBufferedImage(int[] pixels, int width, int height, int rowints, int pixoffset)
	{
		// the RGBA color masks
		int[] bm = new int[]{0x00FF0000,0x0000FF00,0x000000FF,0xFF000000};
		// the colormodel
		ColorModel cm = new DirectColorModel(32,bm[0],bm[1],bm[2],bm[3]);
		// allocate bitmap data
		DataBufferInt db = new DataBufferInt(pixels,pixels.length,pixoffset);
		WritableRaster wr = WritableRaster.createPackedRaster(db,width,height,rowints,bm,null);
		// the image
		return new BufferedImage(cm,wr,false,null);
	}
	public static void main(String args[])
	{
		System.out.println("Starting test");

		ImageInterpolationBug canvas = new ImageInterpolationBug();
		Frame frame = new Frame(canvas.getClass().getName());

		frame.addWindowListener(new java.awt.event.WindowAdapter()
		{
			public void windowClosing(java.awt.event.WindowEvent e)
			{
				System.exit(0);
			};
		});

		
		frame.setSize(640, 480);

		canvas.addMouseMotionListener(canvas);

		frame.add("Center", canvas);
		frame.requestFocus();
		frame.setEnabled(true);

		frame.show();
	}
	//
	public void mouseDragged(java.awt.event.MouseEvent e)
	{
	}
	//
	public void mouseMoved(java.awt.event.MouseEvent e)
	{
		mx = e.getX();
		my = e.getY();
		repaint();
	}
	//
	public void paint(Graphics g)
	{
		// get graphics from the raster
		Graphics2D g2d = (Graphics2D) raster.getGraphics();

		// set hints
		g2d.setColor(Color.darkGray);
		g2d.fillRect(0, 0, raster.getWidth(), raster.getHeight());
		
		// draw
		g2d.setRenderingHint(RenderingHints.KEY_RENDERING, RenderingHints.VALUE_RENDER_SPEED);
		g2d.setRenderingHint(RenderingHints.KEY_INTERPOLATION, RenderingHints.VALUE_INTERPOLATION_NEAREST_NEIGHBOR);

		AffineTransform at0 = AffineTransform.getRotateInstance(mx/50f,src.getWidth()/2,src.getHeight()/2);
		g2d.translate(320-WIDTH/2,200-HEIGHT/2);
		g2d.drawImage(src, at0, null);

		g2d.setRenderingHint(RenderingHints.KEY_RENDERING, RenderingHints.VALUE_RENDER_QUALITY);
		g2d.setRenderingHint(RenderingHints.KEY_INTERPOLATION, RenderingHints.VALUE_INTERPOLATION_BILINEAR);
	

		AffineTransform at1 = AffineTransform.getRotateInstance(mx/50f,src.getWidth()/2,src.getHeight()/2);
		g2d.translate(-100,0);
		g2d.drawImage(src,at1, this);
		
		// paint raster on canvas
		g.drawImage(raster, 0, 0, null);
		//g.drawImage(raster, 0, 0, getWidth(), getHeight(), 0,0,raster.getWidth()-200,raster.getHeight()-200, this);

	}
	// avoid flickering, please
	public void update(Graphics g)
	{
		paint(g);
	}
}

---------- END SOURCE ----------
(Incident Review ID: 223638) 
======================================================================

                                    

Comments
WORK AROUND

Using premultiplied image sources (such as INT_ARGB_PRE) should behave
more as the submitter expects the operation to behave.
                                     
2004-09-30
CONVERTED DATA

BugTraq+ Release Management Values

COMMIT TO FIX:
dragon


                                     
2004-09-30
EVALUATION

One could argue that since an attempt was made to specify that the
pixels were specifically filled with a "red" transparent pixel that
somehow that "red" information was important to the application in
some manner even thought it was hidden behind a transparent alpha.

In particular, if the background were filled with 0x01ff0000 instead
of 0x00ff0000 then wouldn't one expect some red to appear in the
resulting transformed image?

In particular, we have demonstrated that some high-end commercial
image rendering programs also leak red into the image under
similar circumstances - again indicating that if the user went
to the trouble to put red into those tranparent pixels, then there
must have been some reason - some desire for redness to be
indicated there.

A workaround that should work here is to copy the image to a
BufferedImage of type INT_ARGB_PRE which will have the effect
of turning all transparent pixels into transparent black pixels
before the operation (note that this doesn't remove color from
the pixel - it simply makes it black instead of red which would
be less noticeable).  The src image could also be created as
a BufferedImage of type INT_ARGB_PRE as well, but then the
values "0x00ff0000" would not be legal for such an image and
should be changed to "0x00000000" to enforce the premultiplied
constraint that the magnitude of the color components be less
or equal to the alpha component.

Hopefully that workaround will produce results more in line with
what the submitter expects while we look for more input on the
philosophy of how to handle color data for transparent pixels in
transforming images...

###@###.### 2003-11-13

The operations behave as the submitter requests if they
are performed on premultiplied images (type INT_ARGB_PRE)
instead of the default non-premultiplied images (type INT_ARGB).
Ideally both operations should agree on their answers since
the two formats really should only represent different ways of
storing the same information (non-premult having greater precision
on the color values but premult having color values that are
ready to plug into Porter and Duff compositing equations).

Currently the operations simply interpolate each channel of
the image directly.  If one pixel has alpha and color components
A1 & C1 and another has components A2 & C2 then the current
equations give the following formula for the interpolated
pixel that is K (fraction) distance from pixel 1 to pixel 2:

	Aresult = A1 + (A2 - A1) * K
	Cresult = C1 + (C2 - C1) * K

Thus, both color values contribute to the result regardless
of their corresponding alphas.  If the color values above
are non-premultiplied then they may be non-zero even if
the resulting alpha is zero and thus affect the result.

If we perform this math on a premultiplied image then its
color components are related to the color components of
the non-premultiplied image above in the following way:

	C1premult = A1 * C1
	C2premult = A2 * C2
	Cresultpre = C1premult + (C2premult - C1premult) * K
		   = A1 * C1 + (A2 * C2 - A1 * C1) * K

This equation does indeed drop out any color value for any
pixel that has an alpha of zero - the effect requested
by the submitter.

If we wanted to store this premultiplied result back into
a non-premultiplied image, we would normally divide out
the corresponding alpha before we store the color components
giving us the following amended results:

	Cresult2 = (A1 * C1 + (A2 * C2 - A1 * C1) * K)
		  / (A1 + (A2 - A1) * K)

Trying to pull everything together, if we want to look at the
original operation and try to look at it in the terms that
Porter and Duff use when deriving their math, we would look
at the alpha values as "fractional pixel coverage" measurements
instead of "amount of transparency".  This would suggest a
weighted sum with weights considering both the interpolation
fraction (K) and the alphas themselves which represent how
much of each color is there in the first place.

The interpolation fraction suggests that we need to take
(1-K) of the first pixel and (K) of the second and the alpha
values further suggest that we only have A1 amount of the
first pixel and A2 amount of the second.  When we then
combine the two, we have to consider how "much" we have
of the result pixel.  We are no longer accumulating "1
whole pixel" but pieces of two fractional pixels so our
resulting answer is relative to a denominator that is no
longer 1.  This "amount" of the result is simply the
interpolated size between the two source pixel sizes (alphas):

	total = A1 * (1-K) + A2 * K
		= A1 + (A2 - A1) * K
		= Aresult

Thus, the "doubly weighted" equations for the color components
might look something like this:

	Cresult = (A1 * (1-K) * C1 + A2 * K * C2) / total
		= (A1 * C1 - A1 * C1 * K + A2 * C2 * K) / total
		= (A1 * C1 - K * (A2 * C2 - A1 * C1)) / total
		= (A1 * C1 - K * (A2 * C2 - A1 * C1)) / Aresult
		= Cresultpre / Aresult
		= Cresult2

So, operations on the premultiplied images agree exactly with
considering the operations on non-premultiplied images as
combining pixels with "fractional coverage".

Thus, the solution here appears to be to do the pixel 
interpolation always in the premultiplied color format, just
as regular image compositing is done, and convert the colors
back to the appropriate non-premultiplied space for storage
if needed.

We then get similar answers for both premultiplied and
non-premultiplied images and we don't get the confusing
red values leaking into the image from otherwise transparent
pixels.

Note that these extra calculations when working with
non-premultiplied images would seem to recommend once again
that developers use the premultiplied image formats in
preference to the non-premultiplied.

###@###.### 2004-02-17

This bug was fixed as a by-product of creating the new image transformation
loops.  The new loops perform all interpolation calculations on premultiplied
color data.

I will mark this bug fixed as soon as I develop an automated test case that
verifies it in our builds.

###@###.### 2005-1-19 01:48:08 GMT
                                     
2005-01-19



Hardware and Software, Engineered to Work Together