JDK-4950176 : drawImage bad interpolation for non-opaque INT_ARGB pixels
  • Type: Bug
  • Status: Resolved
  • Resolution: Fixed
  • Component: client-libs
  • Sub-Component: 2d
  • Priority: P3
  • Affected Version: 1.4.2
  • OS: windows_2000
  • CPU: x86
  • Submit Date: 2003-11-06
  • Updated Date: 2005-01-20
  • Resolved Date: 2005-01-20
The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availabitlity Release.

To download the current JDK release, click here.
JDK 6
6 betaResolved
Related Reports
Relates :  
Description

Name: rmT116609			Date: 11/06/2003


FULL PRODUCT VERSION :
java version "1.4.2_02"
Java(TM) 2 Runtime Environment, Standard Edition (build 1.4.2
Java HotSpot(TM) Client VM (build 1.4.2_02-b03, mixed mode)

ADDITIONAL OS VERSION INFORMATION :
Microsoft Windows 2000 [Versione 5.00.2195]

A DESCRIPTION OF THE PROBLEM :
I'm making deep use of java2d (I simply love it) for the creation of animated desktop video graphics and video titling.
 
During the development of my applications I have discovered a bug in drawImage() http://developer.java.sun.com/developer/bugParade/bugs/4916948.html that causes incorrect results when drawing images with a translation-only AffineTransform. I also found a workaround for the problem, but I still hope that SUN fixes it asap.

drawImage() bilinear interpolation gives ugly results on ARGB images.
 
consider the 2x2 bitmap pixels :
 
  X0
  00
 
where X is completely transparent (let's say a transparent RED -> 0x00FF0000) and 0 is opaque (let's say an opaque WHITE -> 0xFFFFFFFF).
 
The pixel bilinear interpolation takes the color of the X pixel into consideration. That's completely wrong because the pixel is transparent. When drawing the bitmap at 0.5,0.5 over a clean graphics (0x00000000) the pixel interpolator of java2d gives ugly red artifacts.
 

STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
consider the 2x2 bitmap pixels :
 
  X0
  00
 
where X is completely transparent (let's say a transparent RED -> 0x00FF0000) and 0 is opaque (let's say an opaque WHITE -> 0xFFFFFFFF).
 
The pixel bilinear interpolation takes the color of the X pixel into consideration. That's completely wrong because the pixel is transparent. When drawing the bitmap at 0.5,0.5 over a clean graphics (0x00000000) the pixel interpolator of java2d gives ugly red artifacts.

EXPECTED VERSUS ACTUAL BEHAVIOR :
EXPECTED -
Red artifacts should not appear.

Take a look at
http://www.classx.it/public/bilerp_test.png
http://www.classx.it/public/bilerp_test1.png
http://www.classx.it/public/bilerp_test.jar

Note that MacOSX Java 1.4.1 works correctly.
ACTUAL -
Ugly red artifacts.

REPRODUCIBILITY :
This bug can be reproduced always.

---------- BEGIN SOURCE ----------
package test.bug;

import java.awt.*;
import java.awt.geom.*;
import java.awt.image.*;

public class ImageInterpolationBug extends Panel implements java.awt.event.MouseMotionListener
{
	// storage
	private final static int WIDTH = 250;
	private final static int HEIGHT = 90;
	private static int[] pix = new int[WIDTH * HEIGHT];
	private static BufferedImage src = createBufferedImage(pix,WIDTH, HEIGHT, WIDTH,0);
	private float mx = 0, my = 0;

	// create some test gfx
	static
	{
		// the rendering hints
		java.util.HashMap map = new java.util.HashMap();
		map.put(RenderingHints.KEY_ALPHA_INTERPOLATION,	RenderingHints.VALUE_ALPHA_INTERPOLATION_QUALITY);
		map.put(RenderingHints.KEY_TEXT_ANTIALIASING,  	RenderingHints.VALUE_TEXT_ANTIALIAS_ON);
		map.put(RenderingHints.KEY_INTERPOLATION,    	RenderingHints.VALUE_INTERPOLATION_BILINEAR);
		map.put(RenderingHints.KEY_DITHERING,	    	RenderingHints.VALUE_DITHER_ENABLE);
		map.put(RenderingHints.KEY_FRACTIONALMETRICS,	RenderingHints.VALUE_FRACTIONALMETRICS_ON);
		map.put(RenderingHints.KEY_ANTIALIASING,		RenderingHints.VALUE_ANTIALIAS_ON);
		map.put(RenderingHints.KEY_RENDERING,			RenderingHints.VALUE_RENDER_QUALITY);
		map.put(RenderingHints.KEY_COLOR_RENDERING,		RenderingHints.VALUE_COLOR_RENDER_QUALITY);

		// set bitmap to all transparent red
		for(int i=0; i<WIDTH*HEIGHT; i++)
		{
			pix[i]=0x00FF0000;
		}

		// the graphics
		Graphics2D g2 = (Graphics2D) src.getGraphics();
		g2.setRenderingHints(map);
		g2.setComposite(AlphaComposite.Src);

		// draw a glyph (this creates a correct test image)
		g2.setFont(new Font("dialog",-1,80));
		java.awt.font.GlyphVector gv = g2.getFont().createGlyphVector(g2.getFontRenderContext(),"J2D");
		g2.translate(10,70);
		g2.setColor(Color.gray);
		g2.fill(gv.getOutline());
		g2.translate(2,2);
		g2.setColor(Color.black);
		g2.fill(gv.getOutline());
	}

	// create the dest raster
	BufferedImage raster = createBufferedImage(new int[640*400],640,400, 640,0);
	public ImageInterpolationBug()
	{
		super();
		setBackground(Color.gray);
	}
	// utility: create a new INT_ARGB bufferedimage with rowints support
	private static final BufferedImage createBufferedImage(int[] pixels, int width, int height, int rowints, int pixoffset)
	{
		// the RGBA color masks
		int[] bm = new int[]{0x00FF0000,0x0000FF00,0x000000FF,0xFF000000};
		// the colormodel
		ColorModel cm = new DirectColorModel(32,bm[0],bm[1],bm[2],bm[3]);
		// allocate bitmap data
		DataBufferInt db = new DataBufferInt(pixels,pixels.length,pixoffset);
		WritableRaster wr = WritableRaster.createPackedRaster(db,width,height,rowints,bm,null);
		// the image
		return new BufferedImage(cm,wr,false,null);
	}
	public static void main(String args[])
	{
		System.out.println("Starting test");

		ImageInterpolationBug canvas = new ImageInterpolationBug();
		Frame frame = new Frame(canvas.getClass().getName());

		frame.addWindowListener(new java.awt.event.WindowAdapter()
		{
			public void windowClosing(java.awt.event.WindowEvent e)
			{
				System.exit(0);
			};
		});

		
		frame.setSize(640, 480);

		canvas.addMouseMotionListener(canvas);

		frame.add("Center", canvas);
		frame.requestFocus();
		frame.setEnabled(true);

		frame.show();
	}
	//
	public void mouseDragged(java.awt.event.MouseEvent e)
	{
	}
	//
	public void mouseMoved(java.awt.event.MouseEvent e)
	{
		mx = e.getX();
		my = e.getY();
		repaint();
	}
	//
	public void paint(Graphics g)
	{
		// get graphics from the raster
		Graphics2D g2d = (Graphics2D) raster.getGraphics();

		// set hints
		g2d.setColor(Color.darkGray);
		g2d.fillRect(0, 0, raster.getWidth(), raster.getHeight());
		
		// draw
		g2d.setRenderingHint(RenderingHints.KEY_RENDERING, RenderingHints.VALUE_RENDER_SPEED);
		g2d.setRenderingHint(RenderingHints.KEY_INTERPOLATION, RenderingHints.VALUE_INTERPOLATION_NEAREST_NEIGHBOR);

		AffineTransform at0 = AffineTransform.getRotateInstance(mx/50f,src.getWidth()/2,src.getHeight()/2);
		g2d.translate(320-WIDTH/2,200-HEIGHT/2);
		g2d.drawImage(src, at0, null);

		g2d.setRenderingHint(RenderingHints.KEY_RENDERING, RenderingHints.VALUE_RENDER_QUALITY);
		g2d.setRenderingHint(RenderingHints.KEY_INTERPOLATION, RenderingHints.VALUE_INTERPOLATION_BILINEAR);
	

		AffineTransform at1 = AffineTransform.getRotateInstance(mx/50f,src.getWidth()/2,src.getHeight()/2);
		g2d.translate(-100,0);
		g2d.drawImage(src,at1, this);
		
		// paint raster on canvas
		g.drawImage(raster, 0, 0, null);
		//g.drawImage(raster, 0, 0, getWidth(), getHeight(), 0,0,raster.getWidth()-200,raster.getHeight()-200, this);

	}
	// avoid flickering, please
	public void update(Graphics g)
	{
		paint(g);
	}
}

---------- END SOURCE ----------
(Incident Review ID: 223638) 
======================================================================

Comments
EVALUATION One could argue that since an attempt was made to specify that the pixels were specifically filled with a "red" transparent pixel that somehow that "red" information was important to the application in some manner even thought it was hidden behind a transparent alpha. In particular, if the background were filled with 0x01ff0000 instead of 0x00ff0000 then wouldn't one expect some red to appear in the resulting transformed image? In particular, we have demonstrated that some high-end commercial image rendering programs also leak red into the image under similar circumstances - again indicating that if the user went to the trouble to put red into those tranparent pixels, then there must have been some reason - some desire for redness to be indicated there. A workaround that should work here is to copy the image to a BufferedImage of type INT_ARGB_PRE which will have the effect of turning all transparent pixels into transparent black pixels before the operation (note that this doesn't remove color from the pixel - it simply makes it black instead of red which would be less noticeable). The src image could also be created as a BufferedImage of type INT_ARGB_PRE as well, but then the values "0x00ff0000" would not be legal for such an image and should be changed to "0x00000000" to enforce the premultiplied constraint that the magnitude of the color components be less or equal to the alpha component. Hopefully that workaround will produce results more in line with what the submitter expects while we look for more input on the philosophy of how to handle color data for transparent pixels in transforming images... ###@###.### 2003-11-13 The operations behave as the submitter requests if they are performed on premultiplied images (type INT_ARGB_PRE) instead of the default non-premultiplied images (type INT_ARGB). Ideally both operations should agree on their answers since the two formats really should only represent different ways of storing the same information (non-premult having greater precision on the color values but premult having color values that are ready to plug into Porter and Duff compositing equations). Currently the operations simply interpolate each channel of the image directly. If one pixel has alpha and color components A1 & C1 and another has components A2 & C2 then the current equations give the following formula for the interpolated pixel that is K (fraction) distance from pixel 1 to pixel 2: Aresult = A1 + (A2 - A1) * K Cresult = C1 + (C2 - C1) * K Thus, both color values contribute to the result regardless of their corresponding alphas. If the color values above are non-premultiplied then they may be non-zero even if the resulting alpha is zero and thus affect the result. If we perform this math on a premultiplied image then its color components are related to the color components of the non-premultiplied image above in the following way: C1premult = A1 * C1 C2premult = A2 * C2 Cresultpre = C1premult + (C2premult - C1premult) * K = A1 * C1 + (A2 * C2 - A1 * C1) * K This equation does indeed drop out any color value for any pixel that has an alpha of zero - the effect requested by the submitter. If we wanted to store this premultiplied result back into a non-premultiplied image, we would normally divide out the corresponding alpha before we store the color components giving us the following amended results: Cresult2 = (A1 * C1 + (A2 * C2 - A1 * C1) * K) / (A1 + (A2 - A1) * K) Trying to pull everything together, if we want to look at the original operation and try to look at it in the terms that Porter and Duff use when deriving their math, we would look at the alpha values as "fractional pixel coverage" measurements instead of "amount of transparency". This would suggest a weighted sum with weights considering both the interpolation fraction (K) and the alphas themselves which represent how much of each color is there in the first place. The interpolation fraction suggests that we need to take (1-K) of the first pixel and (K) of the second and the alpha values further suggest that we only have A1 amount of the first pixel and A2 amount of the second. When we then combine the two, we have to consider how "much" we have of the result pixel. We are no longer accumulating "1 whole pixel" but pieces of two fractional pixels so our resulting answer is relative to a denominator that is no longer 1. This "amount" of the result is simply the interpolated size between the two source pixel sizes (alphas): total = A1 * (1-K) + A2 * K = A1 + (A2 - A1) * K = Aresult Thus, the "doubly weighted" equations for the color components might look something like this: Cresult = (A1 * (1-K) * C1 + A2 * K * C2) / total = (A1 * C1 - A1 * C1 * K + A2 * C2 * K) / total = (A1 * C1 - K * (A2 * C2 - A1 * C1)) / total = (A1 * C1 - K * (A2 * C2 - A1 * C1)) / Aresult = Cresultpre / Aresult = Cresult2 So, operations on the premultiplied images agree exactly with considering the operations on non-premultiplied images as combining pixels with "fractional coverage". Thus, the solution here appears to be to do the pixel interpolation always in the premultiplied color format, just as regular image compositing is done, and convert the colors back to the appropriate non-premultiplied space for storage if needed. We then get similar answers for both premultiplied and non-premultiplied images and we don't get the confusing red values leaking into the image from otherwise transparent pixels. Note that these extra calculations when working with non-premultiplied images would seem to recommend once again that developers use the premultiplied image formats in preference to the non-premultiplied. ###@###.### 2004-02-17 This bug was fixed as a by-product of creating the new image transformation loops. The new loops perform all interpolation calculations on premultiplied color data. I will mark this bug fixed as soon as I develop an automated test case that verifies it in our builds. ###@###.### 2005-1-19 01:48:08 GMT
2005-01-19

CONVERTED DATA BugTraq+ Release Management Values COMMIT TO FIX: dragon
2004-09-30

WORK AROUND Using premultiplied image sources (such as INT_ARGB_PRE) should behave more as the submitter expects the operation to behave.
2004-09-30