JDK-8090846 : High bit-depth Grayscale WritableImage support
  • Type: Enhancement
  • Component: javafx
  • Sub-Component: graphics
  • Priority: P4
  • Status: Open
  • Resolution: Unresolved
  • Submitted: 2012-09-01
  • Updated: 2018-09-05
The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.
Other
tbdUnresolved
Related Reports
Relates :  
JDK-8091279 - Support Writable Gray Image
Description
Our app works heavily with grayscale radiographic images. 

In the absence of a grayscale WritableImage, we are using a tedious four-tiered approach with our own MyGrayscaleImage/MyGrayscaleRaster for image filtering with fork/join, inserting our pixels into BufferedImages, converting to and from JavaFX Images using SwingFXUtils, and finally into ImageViews for the scene graph. 

It would be great to try forking and joining with WritableImages, which would cut two tiers from our current approach. This would mean a new PixelFormat.Type.INT_GRAY which would percolate up with new methods like PixelFormat.getIntGrayInstance() etc.
Comments
Ah, there's the min. Having never peered inside a processor, I will take your word for separate integer/fp units and parallelism. Next steps over here are to assess this approach on a wide variety of grayscale images and sample depths. Many thanks.
05-12-2013
I think you missed the part where I combined the min subtraction into the offset. Note "float or double offset = scale * 0.5 - min". The offset already contains the subtracted min so that only 1 add/sub operation was required in the inner loop. With 16-bit values there may be some scaled integer methods that can be applied to the math, but often the fp versions are faster because the float instructions can be executed alongside the integer instructions that manage the loop and array indexing so you get better parallel use of the various units inside the processor...
05-12-2013
To clarify, range = max - min. In your method A wherein scale = 255.0d / range, I think you left out a 'min' in the calculation. I think you meant to say: Then in the inner loop you'd have: value = (int) ((sourcePixel - min + offset) * scale); With precomputed constants up top, inlined getPixel(), use of doubles instead of floats, and casting to an int (trunc rather than round), performance is definitely way snappier. I am leaning towards your method B wherein scale = 256.0d / (max - min + 1) which seems to offer a qualitative edge to converted images. Fortunately, medical devices currently produce grayscale sample depth of 16 bits per pixel or less, which means that (max - min + 1) is presently at most 65536 and not at risk of overflow. We are unlikely to see 4 billion+ (2^32) shades of gray for a few decades.... Jim, you have been a great help to us. Cheers.
05-12-2013
On that last range suggestion, note that if min = 0 and max = 0x7fffffff then the denominator will overflow so care must be taken to calculate it without overflow (left as an exercise for the reader ;). Also, if min and max are far enough apart, then I would recommend using double precision as floats only have 24 bits of mantissa. If you need more bits than the float has mantissa then some of your integer values will be collapsed to the same mantissa and be indistinguishable in the rest of the calculations. doubles have more than enough bits of mantissa to represent every 32-bit integer exactly and precisely...
05-12-2013
Thanks - this is terrific! - we'll go to work on these suggestions.
05-12-2013
Also, if you know that the smallest value in the dataset is min and the largest is max, then I think a "fairer" conversion that evenly divides the data set into all possible values from 0 to 255 would be: float or double scale = 256.0 / (max - min + 1); value = (int) ((sourcePixel - min) * scale); Note that computing with sourcePixel=min gives you 0 and computing with sourcePixel=max gives you ((int) (255.999999)) which casts to 255 (and is the last possible integer that will cast to 255, the next highest int will calculate to 256 and cast to that value). This calculation will attempt to ensure that the same number of integer pixel values end up being converted to 0 as are converted to 255 or any other number in between (with +/- 1 value per slot if the range isn't a perfect multiple of 256).
05-12-2013
We'd likely end up with a very similar loop in our own code. The suggestions that I would recommend here would be: - if you can inline srce.getPixel(x,y), I'm not sure how complicated that method is, but it is likely having to recalculate y*scan+x frequently. - double precision tends to be as fast or faster than float on many modern processors (many do every operation in double anyway and then cast back to float for the result). It's enough to try both float and double and see if there is any difference in performance (and I'm betting double may be faster). - Precalculate some values and then rely on integer casting (which does a truncate, not a floor or round, but is faster). Note that: Math.round(((float)(sourcePixel - min) / range) * 255.0f) == // combine divide and multiply Math.round(((float)(sourcePixel - min) * (255.0f / range)) == // Cast to float/double is implicit with multiply by float value Math.round((sourcePixel - min) * (255.0f / range)) == // round(x) == floor(x + 0.5) Math.floor((sourcePixel - min) * (255.0f / range) + 0.5f) == // propagate +0.5 backwards Math.floor((sourcePixel - min + (0.5f * 255.0f / range)) * (255.0f / range)) == // cast to int == floor for non-negative numbers (int) ((sourcePixel - min + (0.5f * 255.0f / range)) * (255.0f / range)) At the top, precalculate: double or float scale = 255.0 / range; double or float offset = scale * 0.5 - min; Then in the inner loop you'd have: value = (int) ((sourcePixel + offset) * scale); Does that help performance?
05-12-2013
First, please note that we are extremely grateful to you for giving consideration to this JIRA posting. We have a commercial app with real customers and guidance related to this issue will be very helpful. We will be perfectly content if your position is "we're not going to do getIntGrayInstance(), but there's nothing special going on behind the scenes and you can do the conversion just as easily as us and let me show you how you might speed it up a bit". Most of the grayscale images we deal with do not use the full 16 bits per pixel sample depth - some are 10 bpp, some are 12, some are 14. So we figure out the extrema - min, max and range - and use these to convert each pixel value into the range 0..255 and thence into an ARGB format The inner loop of our conversion code, which resides in a RecursiveAction, looks like this (not sure how to format code in JIRA). Any guidance regarding optimization would be appreciated: int curY; int sourcePixel; int value; // traverse the range of rows for this pass for (int y = start; y < start+count; y++) { // compute the current row curY = y*width; // traverse the row for (int x = 0; x < width; x++) { // srce contains an array of ints sourcePixel = srce.getPixel(x,y); // convert the source pixel to a value between 0.0f and 255.0f // then round it to an int value = Math.round(((float)(sourcePixel-min) / range) * 255.0f); // dest is an array of ints to be passed to PixelWriter.setPixels() dest[curY+x] = (255 << 24) | (value << 16) | (value << 8) | value; } }
04-12-2013
Just to be clear - the mechanism is currently backed by 8-bit-per-component ARGB textures at the lowest level and the converters are all Java based so the only real performance advantage without a significant upgrade of the entire architecture would be to move the loops from your code into our code and to possibly save an extra arraycopy(). Given the math involved in pixel conversion (unless all you want is "pixel >> 24" as a conversion function), the memory movement may only be a small part of that equation. Have you done some profiling to see where the time is going? With respect to getIntGrayInstance(), if the conversion function is going to be anything more sophisticated than "component8bit = intpixel >> 24", what would the parameters be? Would you want some form of "(intpixel - minval) / scale" conversion (noting that the divide should be optimized into some other operation, but conceptually is that the right math)? What "monkeying around" is currently done to the 32-bit values to get them into an ARGB format?
04-12-2013
Ours is a JavaFX medical imaging application. A user adjusts a Slider control and the appearance of a high sample-depth grayscale image changes on the screen. We attach a listener to the Slider's value property and filter the image (or a subregion) on every change to the value. We neither need nor want an external Java package for high level image operations. Ours is supposed to be a lively responsive JavaFX app running on contemporary multicore processors. We can't be converting to and from BufferedImages and waiting around for some method from some third-party package to return. We have our own body of filtering code. Our code filters the integer pixel values inside a subclass of RecursiveAction and invokes on a ForkJoinPoin to get all available cores in on the action as a user slides a slider. If we had PixelFormat.Type.INT_GRAY and PixelFormat.getIntGrayInstance(), we could filter using fork/join then directly set a WritableImage's PixelWriter's pixels so that the ImageView in the scene graph might be refreshed at a halfways decent rate. But there isn't, so we filter using fork/join then monkey around to get the pixels into an ARGB format so that we can set the pixels of a vanilla PixelWriter. And refresh rate is lousy. We discovered long ago that there are no range-clipping or range-scaling, or mathematical operations supported here, which is the reason for this JIRA feature request. We respectfully request that you apply your finely honed skills of "targeting ways of getting displayable pixel data into a form that we can optimize loading it into hardware" to do precisely this with grayscale integer pixels. We are requesting that you create a new kind of PixelFormat and perform grayscale pixel range-scaling (and/or other mathematical) operations behind the scenes in an optimal way, and optimize loading it into hardware, and we are happy to precompute extrema.
04-12-2013
WritableImage has so far been mainly targeting ways of getting displayable pixel data into a form that we can optimize loading it into hardware. As such, it is not targeted at all to high end image manipulation of the sort one might use for arbitrary-range grayscale data. There are no range-clipping or range-scaling, or mathematical operations supported here, there is "what color do you want this pixel to be" types of interfaces. Given the number of packages already available for Java to perform such high level image operations, it is not clear what value the simplistic color-based API of WritableImage can provide there...?
04-12-2013
Just to throw a spanner into the works. I know of many scientists who use ImageJ to collect 8-bit and 16-bit grayscale images, particularly for microscopy. Indeed, multi-channel images are also prevalent. Frequently you will find stacked / multipage images which are difficult to work with in JFX2.0+ too. I personally have a hack for 8 and 16 bit TIFFs, building on work done for ImageJ and it works great for my needs. I would love to see what Rick suggests implemented. My TIFF methods are here (in Java and Scala): https://gist.github.com/3344090 https://gist.github.com/3638888
05-09-2012
We would greatly prefer integer grayscale INT_GRAY support. This means one 32-bit signed int per pixel. We want to get an array of ints from a grayscale WritableImage using a PixelReader, filter at will, then put the array back with a PixelWriter. With current x-ray and ct devices, images are generally acquired at 16 bits per pixel. So as of today we work with pixel values in the range of 0-65535 but we this range may grow. X-ray equipment manufacturers are always pushing for acquisition at higher sample depths. We understand that in order to get 32-bit grayscale pixel values into a form that is viewable, each pixel must be downsampled such that its value lies within the range 0-255. We are presently doing this but it's a tedious workaround and it would be nice if you were able to optimally massage single channel 32-bit ints into argb bytes (or whatever) for display purposes behind the scenes. If you can do this, as a suggestion perhaps consider an Extrema class to keep track of min and max grayscale values in an image and a PixelReader.getExtrema() method such that it is our responsibility to keep filtered pixel values within the established range of the native image before calling PixelWriter.setPixels().
04-09-2012
We support byte-indexed formats for injecting data into a WritableImage, but that only provides byte precision. It would be fairly easy to add BYTE_GRAY to the supported formats (as a readable format only), but it looks like the submitter is asking for integer grayscale support. To be clear, is 32-bit precision required for these images? (As opposed to 16 or 8 bit grayscale?) Providing more than 8 bits of precision for displaying grayscale images would require much larger internal changes to our hardware pipelines...
04-09-2012