JDK-8099657 : Provide (official) support for bitmapped rendering
  • Type: Enhancement
  • Component: javafx
  • Sub-Component: graphics
  • Priority: P3
  • Status: Resolved
  • Resolution: Duplicate
  • Submitted: 2009-08-28
  • Updated: 2015-06-16
  • Resolved: 2012-05-16
Related Reports
Duplicate :  
Duplicate :  
Duplicate :  
Description
I'm porting the StrangeAttractor demo to JavaFX; current code attached (needs javafx-anim.jar in a classpath library to compile). It's derived from the Silverlight version (http://blog.joa-ebert.com/2009/08/10/flirting-with-silverlight), there are others for JavaScript/canvas and AS3/Flash. It's a little demo and benchmark of particle graphics.

The only viable implementation for this kind of gfx is direct manipulation of a bitmapped image. The number of particles is too high (300K), so good luck trying to manage such number of scenegraph nodes.

My approach was simple: I use the ImageView.image.platformImage, which (on JavaSE) is a Java2D BufferedImage, so I can obtain access to the backing int[] and read/write individual pixels without any overhead. As you can see by running the program, this works with good performance (up to 100fps here, although this relies on pulse=1000 and at(0ms) which have known bugs/tradeoffs). Except for the following issues:

0) The ImageView's Image creation requires a small hack, I must have a dummy image to load.

1) I need a much worse hack to invalidate the ImageView so it updates at every frame; I'm calling impl_transformsChanged().

Now, I know that my approach is not Pure JavaFX, I'm resorting to Java2D APIs which is not portable across all JavaFX profiles, and perhaps not version-compatible even in the desktop profile because platformImage's type is not documented. But that's OK for this kind of program. And yes it's worth the trouble using JavaFX for this (instead of just writing pure AWT/Swing code); I'm benefiting from the JavaFX Script language, the animation engine, sequences... so I really want to be able to program such stuff in JavaFX. Related bug RT-2383 (Providing pixel support in JavaFX) is being considered, but that bug is probably a harder call (*). If you just make JavaFX/Java2D integration easy and reliable (and JavaFX/LCDUI for mobile), developers can reach to the full power of the underlying 2D API, and the single disadvantage is loss of profile portability. But that's a pretty small tradeoff for advanced effects, high-end games, custom controls and other advanced effects.

Suggestions:

0) Make possible to declare Image { width: W height: H } to easily allocate a blank Image.

1) Make possible to create a ImageView { platformImage: ... } so I can provide my own Image, if I have any reason to need this. I know this an be tricky if the graphics engine relies on a specific image type, color model etc., so that's just in the nice-to-have category.

2) Conversely, make the platformImage documented (for each profile), even if that's subject to change in major releases. Right now my code works reliably but I'm just trusting that this object is always a BufferedImage, AND it always contains a DataBufferInt, AND this buffer has a single data band, AND you don't play tricks like deallocating/reallocating the buffer so I can fetch its int[] and play with it. Keep this behavior stable at least within major versions.

3)  Add a public and official ImageView.invalidate() method, so I don't need to call internal methods that may disappear and may also have other overheads.

Minor collateral issue: If I change the code to not invoke impl_transformsChanged() or any similar invalidation hack, the ImageView will still update the image, but very rarely, producing poor perceived FPS. It's worth investigating why the ImageView object is refreshing at all, gratuitously. If it's just because some volatile VRAM buffer was lost and had to be recreated, fine, but I wouldn't bet on this since I have a 512Mb dedicated video card and no GPU-demanding activity other than StrangeAttractor (well, this and Vista Aero) so I should have plenty free VRAM.

(*) A good solution for RT-2383 is just providing a Canvas node with a paint(g:Object) function, and let the programmer cast "g" to Graphics2D (or to LCDUI's Graphics) and go ahead. This should be easy to implement, the Canvas would have a platformImage:BufferedImage, it's even simpler than my proposal as I don't have to create any Image or worry about invalidation. High performance rendering like in StrangeAttractor would still have to dig inside the platformImage to hack individual pixels fast, so item 2 above would still be relevant to this solution: the platform buffer should be reasonably documented and stable, at least for the Canvas object if not for Image. This can be a good scope limitation because Canvas would be used in a more narrow context than Image, so you could be free to play evil tricks with Image.platformImage. OTOH, if I want fast animation, I guess I should call a Canvas.invalidate() method from a keyframe, to force its paint() to be eventually invoked... but that doesn't seem to fit in the JavaFX programming model, and could imply in extra context switches, although this could be irrelevant for "normal" animation rates (pulse, keyframe delays). Perhaps there's no cost if, after all queue of keyframes is processed, the runtime immediately invokes paint() on just-invalidated Canvas nodes, without waiting for some future event.

P.S.: I noticed also that BufferedImage.setRGB(startX, startY, width, height, rgbArray, offset, scansize) does not optimize the extremely common case of scansize==width and single-band DataBuffer; that case could be implemented by a fast path that just goes to the DataBuffer and calls a single arraycopy() on the backing array. (Doesn't even require other conditions like zero values for startX/startY/offset, or any bounds checking; setRGB()'s spec allows any bounds exception to happen, with no guarantee of partial update, for out-of-bound parameters.) But that would be a JRE RFE, and I'm not using this API anymore. The same optimization could benefit the bulk getRGB().
Comments
In JavaFX 2.2 we provide all the functionality that is needed to implement this with Image Ops (writable images and public to/from BufferedImage methods), a Canvas node, and snapshot (render to image) methods on Node and Scene.
16-05-2012

Dupe of RT-14038?
08-06-2011

This is still a valid request. We are providing a "non-public" way of doing this in JavaFX 2.0 and could consider making that public at some point.
15-04-2011

Old JavaFX issue.
15-04-2011

OK, could you at least give some position on the approach I'm following? For one thing, I only tested this code in the Windows platform; I don't know if my assumptions about platformImage will be valid on MacOSX, Linux, Solaris... I don't have all those platforms for testing. Well I could just install all these in testing VMs, but this will only get worse when JavaFX Mobile and JavaFX TV start rolling and there are many more implementations. Right now JavaFX is pretty hard for third-party extension developers. Suppose I want to create a new Control that really demands custom (non scenegraph-based) rendering, what should I do? Full source code is not available so I can't consult it; in-depth technical documentation does not exist at all; the platform still misses important functionality for some people. If the team at least provides some guideline about JavaFX-to-native-2D integration, at least we can work around these limitations while they exist.
29-08-2009

Consider for a future release.
28-08-2009