JDK-8094046 : Poor DisplacementMap effect performance on Mac
  • Type: Bug
  • Component: javafx
  • Sub-Component: graphics
  • Affected Version: 8
  • Priority: P2
  • Status: Closed
  • Resolution: Fixed
  • Submitted: 2012-12-04
  • Updated: 2015-06-12
  • Resolved: 2013-07-24
The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.
JDK 8
8Fixed
Related Reports
Blocks :  
JDK-8095336 - checkState in BaseShaderContext changes the texture unit currently attached to the context
Duplicate :  
JDK-8120606 - Prism leaks OpenGL resources on Mac
Relates :  
JDK-8097430 - [Glass, Mac] Extremely poor performance with ColorfulCirclesApp on Early 2013 retina MacBook Pro
Relates :  
JDK-8124072 - Mac: 8.0-graphics-scrum-792: up to 30% performance regression on MacOS
Description
*NOTE:  This JIRA is also tracking the performance issue RT-28752.  Before this is marked fixed, it is important to run the tests from this JIRA and generally make sure it is also fixed.*


Steps to reproduce:
1. Open BookPanel project
2. Run bookpanel.BendingPagesTest from Test Packages
3. Drag the right bottom edge of the page.
Observe poor performance (~5-6 fps) with Renderer: NVIDIA GeForce GT 650M OpenGL Engine. 
At the same time performance tracker reports ~60 fps.

You could also run the automated version (booktest.BeagleboardTest) which does the same using Timeline. The effect is very smooth and actually appearing at 60 fps.


On Windows with the following driver performace is at ~45 fps
D3D Driver Information:
	NVIDIA NVS 4200M
	\\.\DISPLAY2
	Driver nvd3dumx.dll, version 8.17.12.9679
	Pixel Shader version 3.0
	Device : ven_10DE, dev_1056, subsys_04931028
Comments
Hard to verify without specific test. Hard to create test. I will create issue for test creation. Closing as not verified as for now.
11-02-2014
Yes, the initial bug description implies a significant visible difference in performance. But taking into account the reason (skipping of frames as described by Oleg M.) I would prefer to rely on automated test to check if the frames are actually skipped or not.
06-01-2014
Oleg: you may be running into a different problem. The way to see whether this bug is fixed or still a problem is to see whether there is a significant difference in performance when you rapidly move the mouse as compared to the performance when the mouse is still.
06-01-2014
For my config (GeForce 320M) the performance is still insufficient both for manual and automated version of the test. An automated test is needed.
06-01-2014
http://jfxsrc.us.oracle.com/javafx/8.0/scrum/graphics/rt/rev/dd30604ab7d0
24-07-2013
I've been busy on different tasks. Now I've made a final sanity check, ensured all involved tests show a stable improvement in performance and will send this for review tomorrow morning.
22-07-2013
Is there a reason why this issue is still unresolved? My concern is that the patch might get stale if this work is stalled for too long.
22-07-2013
The updated patch looks good to me. I see that the gl context gets restored and all known problems were addressed including RT-31186. I have no more concern about the patch. I will let Alex or Oleg do the performance improvement evaluation, though the visual feel of BendingPagesTest feels good to me. Thanks Petr!
12-07-2013
I have updated the double buffering again and tested all the known failures. I could not find any new troubles with it.
12-07-2013
Note that the current behavior (performance) is not the same as the original because of my tentative fix (see my comment of Dec, 7 2012). When we have a patch that everybody is satisfied with in terms of proposed solution, implementation (code review), functionality (no regressions, crashes, exceptions, responsiveness problems) then we can have another round of thorough performance testing. For that we won't be able to rely on FPS alone as we'll need to count not just frames prepared by FX but those actually passed to the layer (and make sure we don't double count the same frame passed twice). I think an ad hoc solution (glass level instrumentation?) will be required. Since the current state is prone to excessive delays in the FX thread we'll also need a way to account for that.
12-07-2013
I just compared both builds on Chien's laptop and I can see the difference, i. e. the rendering is improved by patch. Unfortunately, there are regressions that are necessary to address. Seeing this performance difference is tricky. Oleg, do we have any better and easy ways to check if the problem persist?
11-07-2013
I'm catching up on the discussion on this thread. I have a question and a concern about the proposed fix. Maybe someone (Petr?) can address it. I tried the BendingPagesTest project (It was moved and thanks to Alex for pointing me to the place). I'm unable to observe any performance difference between with and wthout the proposed patch applied. Do we have a good performance to qualify the value of this patch? My concern with this patch is that it didn't restore or mess up the context use by Prism. Besides the issue reported in HelloComboBox (see RT-31186) it also breaks helloworld.HelloRenderSnapshot and helloworld.HelloRenderSnapshot3. Here is the exceptions throw by HelloRenderSnapshot3 after applying the proposed patch: ckyang-mac:HelloWorld ckyang$ jjfx helloworld.HelloRenderSnapshot3 java.lang.reflect.InvocationTargetException at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:491) at com.sun.prism.es2.ES2ResourceFactory.createStockShader(ES2ResourceFactory.java:260) at com.sun.prism.impl.ps.BaseShaderContext.getPaintShader(BaseShaderContext.java:223) at com.sun.prism.impl.ps.BaseShaderContext.validatePaintOp(BaseShaderContext.java:455) at com.sun.prism.impl.ps.BaseShaderContext.validatePaintOp(BaseShaderContext.java:341) at com.sun.prism.impl.ps.BaseShaderGraphics.renderGeneralRoundedPgram(BaseShaderGraphics.java:836) at com.sun.prism.impl.ps.BaseShaderGraphics.renderGeneralRoundedRect(BaseShaderGraphics.java:539) at com.sun.prism.impl.ps.BaseShaderGraphics.fillEllipse(BaseShaderGraphics.java:1463) at com.sun.prism.impl.shape.BasicEllipseRep.fill(BasicEllipseRep.java:41) at com.sun.javafx.sg.prism.NGShape.renderContent(NGShape.java:219) at com.sun.javafx.sg.prism.NGNode.doRender(NGNode.java:412) at com.sun.javafx.sg.prism.NGNode.doRender(NGNode.java:69) at com.sun.javafx.sg.BaseNode.render(BaseNode.java:1283) at com.sun.javafx.sg.prism.NGGroup.renderChildren(NGGroup.java:237) at com.sun.javafx.sg.prism.NGGroup.renderContent(NGGroup.java:203) at com.sun.javafx.sg.prism.NGNode.doRender(NGNode.java:412) at com.sun.javafx.sg.prism.NGNode.doRender(NGNode.java:69) at com.sun.javafx.sg.BaseNode.render(BaseNode.java:1283) at com.sun.javafx.tk.quantum.QuantumToolkit$21.draw(QuantumToolkit.java:1524) at com.sun.javafx.tk.quantum.QuantumToolkit$21.run(QuantumToolkit.java:1559) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304) at com.sun.javafx.tk.RenderJob.run(RenderJob.java:58) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at com.sun.javafx.tk.quantum.QuantumRenderer$PipelineRunnable.run(QuantumRenderer.java:129) at java.lang.Thread.run(Thread.java:724) Caused by: java.lang.RuntimeException: Error creating vertex shader at com.sun.prism.es2.ES2Shader.createFromSource(ES2Shader.java:132) at com.sun.prism.es2.ES2Shader.createFromSource(ES2Shader.java:173) at com.sun.prism.es2.ES2ResourceFactory.createShader(ES2ResourceFactory.java:175) at com.sun.prism.shader.FillCircle_Color_Loader.loadShader(FillCircle_Color_Loader.java:45) ... 30 more java.lang.InternalError: Error loading stock shader FillCircle_Color at com.sun.prism.es2.ES2ResourceFactory.createStockShader(ES2ResourceFactory.java:263) at com.sun.prism.impl.ps.BaseShaderContext.getPaintShader(BaseShaderContext.java:223) at com.sun.prism.impl.ps.BaseShaderContext.validatePaintOp(BaseShaderContext.java:455) at com.sun.prism.impl.ps.BaseShaderContext.validatePaintOp(BaseShaderContext.java:341) at com.sun.prism.impl.ps.BaseShaderGraphics.renderGeneralRoundedPgram(BaseShaderGraphics.java:836) at com.sun.prism.impl.ps.BaseShaderGraphics.renderGeneralRoundedRect(BaseShaderGraphics.java:539) at com.sun.prism.impl.ps.BaseShaderGraphics.fillEllipse(BaseShaderGraphics.java:1463) at com.sun.prism.impl.shape.BasicEllipseRep.fill(BasicEllipseRep.java:41) at com.sun.javafx.sg.prism.NGShape.renderContent(NGShape.java:219) at com.sun.javafx.sg.prism.NGNode.doRender(NGNode.java:412) at com.sun.javafx.sg.prism.NGNode.doRender(NGNode.java:69) at com.sun.javafx.sg.BaseNode.render(BaseNode.java:1283) at com.sun.javafx.sg.prism.NGGroup.renderChildren(NGGroup.java:237) at com.sun.javafx.sg.prism.NGGroup.renderContent(NGGroup.java:203) at com.sun.javafx.sg.prism.NGNode.doRender(NGNode.java:412) at com.sun.javafx.sg.prism.NGNode.doRender(NGNode.java:69) at com.sun.javafx.sg.BaseNode.render(BaseNode.java:1283) at com.sun.javafx.tk.quantum.QuantumToolkit$21.draw(QuantumToolkit.java:1524) at com.sun.javafx.tk.quantum.QuantumToolkit$21.run(QuantumToolkit.java:1559) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304) at com.sun.javafx.tk.RenderJob.run(RenderJob.java:58) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at com.sun.javafx.tk.quantum.QuantumRenderer$PipelineRunnable.run(QuantumRenderer.java:129) at java.lang.Thread.run(Thread.java:724) Exception in thread "JavaFX Application Thread" java.lang.IllegalArgumentException: Unrecognized image loader: null at javafx.scene.image.WritableImage.loadTkImage(WritableImage.java:230) at javafx.scene.image.WritableImage.access$000(WritableImage.java:44) at javafx.scene.image.WritableImage$1.loadTkImage(WritableImage.java:49) at javafx.scene.Scene.doSnapshot(Scene.java:1268) at javafx.scene.Scene.doSnapshot(Scene.java:1300) at javafx.scene.Scene.snapshot(Scene.java:1387) at helloworld.HelloRenderSnapshot3$1.handle(HelloRenderSnapshot3.java:40) at helloworld.HelloRenderSnapshot3$1.handle(HelloRenderSnapshot3.java:38) at com.sun.javafx.event.CompositeEventHandler.dispatchBubblingEvent(CompositeEventHandler.java:86) at com.sun.javafx.event.EventHandlerManager.dispatchBubblingEvent(EventHandlerManager.java:238) at com.sun.javafx.event.EventHandlerManager.dispatchBubblingEvent(EventHandlerManager.java:191) at com.sun.javafx.event.CompositeEventDispatcher.dispatchBubblingEvent(CompositeEventDispatcher.java:59) at com.sun.javafx.event.BasicEventDispatcher.dispatchEvent(BasicEventDispatcher.java:58) at com.sun.javafx.event.EventDispatchChainImpl.dispatchEvent(EventDispatchChainImpl.java:114) at com.sun.javafx.event.BasicEventDispatcher.dispatchEvent(BasicEventDispatcher.java:56) at com.sun.javafx.event.EventDispatchChainImpl.dispatchEvent(EventDispatchChainImpl.java:114) at com.sun.javafx.event.BasicEventDispatcher.dispatchEvent(BasicEventDispatcher.java:56) at com.sun.javafx.event.EventDispatchChainImpl.dispatchEvent(EventDispatchChainImpl.java:114) at com.sun.javafx.event.EventUtil.fireEventImpl(EventUtil.java:74) at com.sun.javafx.event.EventUtil.fireEvent(EventUtil.java:49) at javafx.event.Event.fireEvent(Event.java:203) at javafx.scene.Node.fireEvent(Node.java:8002) at javafx.scene.control.Button.fire(Button.java:184) at com.sun.javafx.scene.control.behavior.ButtonBehavior.mouseReleased(ButtonBehavior.java:183) at com.sun.javafx.scene.control.skin.BehaviorSkinBase$1.handle(BehaviorSkinBase.java:91) at com.sun.javafx.scene.control.skin.BehaviorSkinBase$1.handle(BehaviorSkinBase.java:84) at com.sun.javafx.event.CompositeEventHandler$NormalEventHandlerRecord.handleBubblingEvent(CompositeEventHandler.java:218) at com.sun.javafx.event.CompositeEventHandler.dispatchBubblingEvent(CompositeEventHandler.java:80) at com.sun.javafx.event.EventHandlerManager.dispatchBubblingEvent(EventHandlerManager.java:238) at com.sun.javafx.event.EventHandlerManager.dispatchBubblingEvent(EventHandlerManager.java:191) at com.sun.javafx.event.CompositeEventDispatcher.dispatchBubblingEvent(CompositeEventDispatcher.java:59) at com.sun.javafx.event.BasicEventDispatcher.dispatchEvent(BasicEventDispatcher.java:58) at com.sun.javafx.event.EventDispatchChainImpl.dispatchEvent(EventDispatchChainImpl.java:114) at com.sun.javafx.event.BasicEventDispatcher.dispatchEvent(BasicEventDispatcher.java:56) at com.sun.javafx.event.EventDispatchChainImpl.dispatchEvent(EventDispatchChainImpl.java:114) at com.sun.javafx.event.BasicEventDispatcher.dispatchEvent(BasicEventDispatcher.java:56) at com.sun.javafx.event.EventDispatchChainImpl.dispatchEvent(EventDispatchChainImpl.java:114) at com.sun.javafx.event.EventUtil.fireEventImpl(EventUtil.java:74) at com.sun.javafx.event.EventUtil.fireEvent(EventUtil.java:54) at javafx.event.Event.fireEvent(Event.java:203) at javafx.scene.Scene$MouseHandler.process(Scene.java:3622) at javafx.scene.Scene$MouseHandler.process(Scene.java:3453) at javafx.scene.Scene$MouseHandler.access$1900(Scene.java:3405) at javafx.scene.Scene.impl_processMouseEvent(Scene.java:1697) at javafx.scene.Scene$ScenePeerListener.mouseEvent(Scene.java:2473) at com.sun.javafx.tk.quantum.GlassViewEventHandler$MouseEventNotification.run(GlassViewEventHandler.java:312) at com.sun.javafx.tk.quantum.GlassViewEventHandler$MouseEventNotification.run(GlassViewEventHandler.java:237) at java.security.AccessController.doPrivileged(Native Method) at com.sun.javafx.tk.quantum.GlassViewEventHandler.handleMouseEvent(GlassViewEventHandler.java:354) at com.sun.glass.ui.View.handleMouseEvent(View.java:514) at com.sun.glass.ui.View.notifyMouse(View.java:877) ES2 Vram Pool: 1,607,108 used (0.6%), 1,607,108 managed (0.6%), 268,435,456 total 11 total resources being managed 4 permanent resources (36.4%) 1 resources locked (9.1%) 7 resources contain interesting data (63.6%) 0 resources disappeared (0.0%)
11-07-2013
Sure, I'm attaching a new patch which should be warning-free. These warnings are not even generated with my version of the compiler.
09-07-2013
We'd like to keep our code warnings-free. So if there's a way to eliminate them, we should do it.
09-07-2013
These warnings do not really mean anything. OBJC is not smart enough to understand forward declarations. Moving the declaration of the setContext/unsetContext to the beginning of the source file resolves this warnings. They do not affect the execution.
09-07-2013
Much of the file structures have changed since we moved to use gradle for build. Attached is double-bufferNew.patch which is a line by line port of double-buffer.patch. I noticed the following warning were generated when compiling the patch: :graphics:javahMacGlass :graphics:ccMacGlass /Users/ckyang/java/javafx/800-exp/jfx/rt/modules/graphics/src/main/native-glass/mac/GlassOffscreen.m: In function ���-[GlassOffscreen initWithContext:]���: /Users/ckyang/java/javafx/800-exp/jfx/rt/modules/graphics/src/main/native-glass/mac/GlassOffscreen.m:52: warning: ���GlassOffscreen��� may not respond to ���-setContext��� /Users/ckyang/java/javafx/800-exp/jfx/rt/modules/graphics/src/main/native-glass/mac/GlassOffscreen.m:52: warning: (Messages without a matching method signature /Users/ckyang/java/javafx/800-exp/jfx/rt/modules/graphics/src/main/native-glass/mac/GlassOffscreen.m:52: warning: will be assumed to return ���id��� and accept /Users/ckyang/java/javafx/800-exp/jfx/rt/modules/graphics/src/main/native-glass/mac/GlassOffscreen.m:52: warning: ���...��� as arguments.) /Users/ckyang/java/javafx/800-exp/jfx/rt/modules/graphics/src/main/native-glass/mac/GlassOffscreen.m:61: warning: ���GlassOffscreen��� may not respond to ���-unsetContext��� /Users/ckyang/java/javafx/800-exp/jfx/rt/modules/graphics/src/main/native-glass/mac/GlassOffscreen.m: In function ���-[GlassOffscreen dealloc]���: /Users/ckyang/java/javafx/800-exp/jfx/rt/modules/graphics/src/main/native-glass/mac/GlassOffscreen.m:73: warning: ���GlassOffscreen��� may not respond to ���-setContext��� /Users/ckyang/java/javafx/800-exp/jfx/rt/modules/graphics/src/main/native-glass/mac/GlassOffscreen.m:78: warning: ���GlassOffscreen��� may not respond to ���-unsetContext��� :graphics:linkMacGlass :graphics:nativeGlass
09-07-2013
Regarding your double_buffering.patch, looks like you are adding a blit method. I am working on MSAA, I have add blit method to GLContext. It is required to resolve MSAA render buffer. Thus ES2SwapChain will then blit the stableBackbuffer directly.
19-06-2013
Attached the current patch with the double-buffering approach implementation.
19-06-2013
I was looking for a summary of the current patch and why it can drop frames (to decide whether we care). The collapsing happens in the patch because the notification is posted to the UI thread. It's not a problem as long as we understand why it can Direct draw does not work in Applets either. I added flags to turn it off. Debugging in Applets was too hostile. I imagine that remote layers expect automatic updating rather than setNeedsDisplay. The remote layer stuff is not documented but comes from Apple and Apple supports it for Java.
30-05-2013
The current implementation drop frames for the reasons Oleg described in the beginning. Double buffering could drop one or 2 frames, because sometimes redraw notifies could get collapsed, but in quite rare cases. Direct draw never drop frames. I have found a problem with double buffering: it does not work in applets. I am trying to figure out why. Does direct draw work in applets? So, as I understand, the only issue with direct draw is that it does not follow apple's guidelines?
30-05-2013
I have reproduced and fixed the problem with mouse moving which Steve has reported. At the moment when we post a setNeedsDisplay selector from render thread to the main thread it is queued in the main queue, which is filled by mouse events. Now the repaint selector is dispatched with the highest possible priority, so we do not drop frames. Changing it's priority has almost no effect on event processing performance, because setNeedsDisplay is a very fast non-blocking call. Also I can confirm that the garbage on the first frame has gone with Steve's changes in Quantum. I have hacked a code a bit to measure how many times we actually draw on screen (-drawInGLContext method is called) and made some performance measurements: 1. Steves test: no fix: no mouse wiggling: Prism: 24.5 real: 18.5 mouse wiggling: Prism: 21 real: 18 with the fix: no mouse wiggling: Prism: 29 real: 28 mouse wiggling: Prism: 27 real: 27 2. bitmap-canvas-18000 no fix: no mouse wiggling: Prism: 44 real: 38 mouse wiggling: Prism: 44 real: 38 with the fix: no mouse wiggling: Prism: 44 real: 44 mouse wiggling: Prism: 42 real: 42 3. vector-fullspeed no fix: no mouse wiggling: Prism: 49 real: 45 mouse wiggling: Prism: 48 real: 40 with the fix: no mouse wiggling: Prism: 50 real: 49 mouse wiggling: Prism: 47 real: 47 The numbers are quite approximate, but it illustrates that with the double buffering approach we get increase in performance reported by prism, but not all the time. However, the real FPS we draw on screen increases all the time and we do not drop frames.
30-05-2013
Anthony had a slow machine that we were using to figure out why using the native timer on my Mac increased fps. If this is easy for you, could you try your code on that and get some performance numbers? The "direct draw" approach should never drop a frame (Prism and real should be the same) and seemed to be faster on slow machines. Are we ok with dropping frames and do we know exactly why this is happening (ie. is it something like "Prism draws are too fast and redraw notifies get collapsed") so that we are ok with it?
30-05-2013
Thank you Steve. I will check the double buffering with new changes tomorrow and report the results. As long as the results on dropped frames.
29-05-2013
Quantum Java changes have been released.
29-05-2013
@Petr I believe the Quantum changes in the patch will fix the draw garbage problem. There is an extra lock/unlock in validateStageGraphics() that might be causing you trouble. @Kevin I believe the Quantum changes are good and should just go in. Do you agree? It gets rid of a lock/unlock and moves the creation of the default pipeline into the runnable. Really validateStageGraphics should just go in the garbage and it should be inlined in the two different places it is used. This would move the code that is doing the drawing closer together and would help with debugging/dangling locks etc. This can be done as part of Quantum clean up.
29-05-2013
Here is what I am running (the same as in the previous comment). I build the shared library and sent it to Richard and he sees the same thing. Main: guimark2.BitmapTest Program args: -monsters 18000 VM args: -Djavafx.animation.fullspeed=true
29-05-2013
Thank you for testing, Steve. Could you please specify which of the bitmap tests did you run, because I did not see any problems there. I will test it more extensively. I think I understand the possible reason of the behavior you see, we dispatch a call to the main thread from the render thread when we finished rendering, so if the queue is filled with mouse events out rendering calls could get delayed. I will check this idea tomorrow.
29-05-2013
After actually applying the patch (I did a --dry-run by mistake), I am still seeing the dropped frames when I move the mouse like mad inside guimark2.BitmapTest. FPS goes up or stays the same, but visually frames are not drawn. Resizing EnsembleApp like mad by grabbing the right bottom corner no longer crashes.
29-05-2013
After some investigations I have found a problem with double buffering approach and fixed it. To summarize, the idea of the double buffering is the following: We have 2 frame buffer objects, FBO1 used by Prism and FBO2 used by glass. Each FBO has an associated context. To write into the FBO we need to set current context to the FBO's context, reading from FBO is ok under any context. Prism renders to FBO1 on render thread, than we flush it to FBO2 on render thread and then post a selector to the main thread to do [layer setNeedsDisplay]. After that main thread calls canDrawInGLContext, we return if the FBO2 is dirty, than drawInGLContext is called and we render FBO2 to screen. Locking is required only between FBO1->FBO2 and FBO2->Screen procedures. Prism->FBO1 and FBO2->Screen can go concurrently. The crash occurred because we took a lock at the start of drawInGLContex method, which apparently is too late. It looks like Quartz does something with the context before calling drawInGLContext method. I have made a new version of the fix where mutex was replaced by CGLLockContext to lock over a context for FBO2, not FBO2 itself and now everything works fine without crashes. Additionally, I have found an interesting problem: the first frame Prism render contains garbage. We did not see it without double buffering because we are always skipping the first frame currently. Now we use async layers, so the codepath on startup is the following: Prism starts to draw the first frame. When it finishes we return YES from the canDrawInGLContext method. But, before drawInGLContext is called, prism starts to render the next frame and locks the framebuffer. So when drawInGLContext is called it stops on a lock. When Prism finishes it unlocks the frame buffer and drawInGLContext proceeds. But it already renders the second frame. With the new approach every rendered frame gets to screen, so we now see the garbage in the first frame. If I make it skip the first frame with some static flag I do not see garbage any more and everything works as expected. So, I suppose garbage in first frame is a bug in Prism? Another question to the Graphics team: you are creating to contexts on startup: shared and client. You use client context to render, what is the reason for you to have a second context? To summarize: the main advantage of the double buffering approach over what we have now is that we need to lock for a very fast procedure (copying FBO1->FBO2). It does not crash now. At least a lot of testing shows it does not, but I cannot be sure as always with threading issues. The only remaining problem is the garbage on the first frame, but it is a problem of this fix.
29-05-2013
Just tried with the latest repository and the attached patch applied, and it builds fine. So no problem.
17-05-2013
Does this still fail after a clean Glass build?
08-05-2013
Note: I get the following error after applying the patch attached to this issue to the latest Graphics scrum: [exec] g++ -framework AppKit -framework ApplicationServices -framework JavaVM -framework JavaRuntimeSupport -framework OpenGL -framework QuartzCore -dynamiclib -o ./dist/libglass.dylib -lobjc -mmacosx-version-min=10.7 -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.7.sdk -arch i386 -arch x86_64 -F/System/Library/Frameworks/JavaVM.framework/Frameworks build/GlassAccessibleBaseProvider.obj build/GlassAccessibleRoot.obj build/GlassAccessibleSelectionItemProvider.obj build/GlassAccessibleSelectionProvider.obj build/GlassAccessibleToggleProvider.obj build/GlassApplication.obj build/GlassCursor.obj build/GlassDialogs.obj build/GlassDragSource.obj build/GlassEmbeddedWindow+Npapi.obj build/GlassEmbeddedWindow+Overrides.obj build/GlassFrameBufferObject.obj build/GlassFullscreenWindow.obj build/GlassGestureSupport.obj build/GlassHelper.obj build/GlassHostView.obj build/GlassKey.obj build/GlassLayer3D.obj build/GlassMacros.obj build/GlassMenu.obj build/GlassNSEvent.obj build/GlassOffscreen.obj build/GlassPasteboard.obj build/GlassPixels.obj build/GlassRobot.obj build/GlassScreen.obj build/GlassStatics.obj build/GlassSystemClipboard.obj build/GlassTimer.obj build/GlassTouches.obj build/GlassView.obj build/GlassView2D.obj build/GlassView3D+Remote.obj build/GlassView3D.obj build/GlassViewDelegate.obj build/GlassWindow+Java.obj build/GlassWindow+Overrides.obj build/GlassWindow.obj build/ProcessInfo.obj build/RemoteLayerSupport.obj [exec] Undefined symbols for architecture x86_64: [exec] "_OBJC_IVAR_$_GlassWindow.isEmbedded", referenced from: [exec] _Java_com_sun_glass_ui_mac_MacView__1end in GlassView.obj [exec] -[GlassWindow(Java) _initWithContentRect:styleMask:screen:jwindow:jIsChild:] in GlassWindow+Java.obj [exec] ld: symbol(s) not found for architecture x86_64 [exec] collect2: ld returned 1 exit status [exec] lipo: can't open input file: /var/folders/3j/7gjfpfgn1k15d733tsm6y3wc00677c/T//ccHivFB4.out (No such file or directory) [exec] make: *** [dist/libglass.dylib] Error 1
07-05-2013
Of course, the image must remain as a texture for performance. Dho! @Artem - Multiple threads do not necessarily mean multiple GL contexts. Typically, a single GL context is still used and a software command buffer per thread is used to gather up draw commands and then serialize the result. There are other strategies.
09-04-2013
Multiple render thread and multiple OGL contexts are two different things, I didn't think we had yet considered multiple OGL contexts (I know Chris Campbell went down that road with J2D and it was much slower than single threaded rendering): http://weblogs.java.net/blog/campbell/archive/2005/03/strcrazy_improv_1.html Maybe having multiple rendering contexts and a single thread per context and compositing results at the end however will give a much better improvement (I assume that is what you are suggesting), although I don't know when that is going to happen (probably not in the near future).
09-04-2013
Do you mean NSImage, when say "OS X image"? Can NSImages reside in video memory? If not, we'll get huge performance and CPU regression, when copying data from video memory (texture) to shared (NSImage) and back.
09-04-2013
Eventually we'll have multiple render threads, otherwise we won't be able to utilize all power of modern GPUs. Multiple render threads mean multiple contexts anyway, so we'll have to solve this problem anyway, sooner of later.
09-04-2013
It is safer from the standpoint that it is drawing how Apple recomends. I suppose that texture "L" in Artem's description could just be an OS X image and that step 3) could blt to an image rather than a texture (or some combination of this where we convert from texture to OS X image and Glass deals in OS X images all the time).
09-04-2013
If we are talking about a single OpenGL context for glass and a separate, single OpenGL context for Prism then yes, I think this is a safe approach -- likely safer than what we are doing today. If you mean two glass contexts or two Prism contexts, then probably not.
09-04-2013
Is managing multiple OpenGL contexts really safer?
09-04-2013
There are two approaches that can work: 1) "direct draw" (AKA Steve's patch) 2) "double buffer" (AKA Petr's patch) Petr's patch does not have performance problems. It currently crashes but may not crash if we create another open GL context. I have experienced lost frames guimark2.BitmapTest when wiggling the mouse over the monsters window. Petr doesn't see this. To move forward with Petr's patch, we need to investigate creating another OpenGL context. Petr's patch is safer, but more complicated.
09-04-2013
I suggest backing up -- anything complicated in the OpenGL space is likely to lead to crashes -- we're seeing it here and we've seen it before over the years. From the thread it sounds like: 1) Steve's patch. Works. Uses the API in a way Apple doesn't recommend, and so on the next OS upgrade we might start seeing problems (and won't get any prior notification from Apple). Requires a different (slower) code path for the Applet (not a big deal). 2) Oleg's Patch (or related). Works. Doesn't get the full benefit of render/ui threads because there is some time spent locking, waiting for UI thread to paint. 3) As yet unknown double-buffer style patch. Doesn't work (yet). Performance lost with blitting, or loss of dirty regions. Is that right? What is the *simplest* possible fix we can use for now, that works?
09-04-2013
@Steve I do not see any problems with guimark2, however I suppose we have a bigger issue with the patch, which was reported by Thor here. I have double-checked the synchronization, but it still crashes sometimes. The crash appears deep in the GPU driver. So, I suppose that it is exactly the problem Thor specified: we cannot render concurrently with a shared context, however could we make 2 contexts? Is it possible? However, it is probably way out of my OpenGL expertise.
09-04-2013
Hi Petr, have you been able to confirm the dropped frames when wiggling the mouse during guimark2.BitmapTest?
08-04-2013
I have made this point before; I don't think you can do what Petr is doing (simply and safely) with two FBO���s and a single context. Where two threads are accessing a context simultaneously. I expect the result would be unpredictable failures. Typically passing a context between threads is fine, (we have been doing that for a while) so long there is no simultaneous access (use of locks). Granted the above is an over simplification. There are a few finer points: 1. Use of two contexts with shared resources is possible. But that is complicated and I don���t think you will get any more performance than Steve���s patch. 2. Some implementations of OGL have a setting to support multi-threaded access to a single context. Think this is worse than (1). I suspect it is essentially single threaded context with some access control by OS/impl.
03-04-2013
HelloWindowAbuse is a good thing to run as well. It will catch leaks and crashes.
02-04-2013
The crash is new and should be easy for you to fix along with the garbage. Please investigate the dropped frames first though. In the meantime, I don't seem to have BendingPagesTest app. What is the exact name and where is the code? I'll test that too.
02-04-2013
Sounds good. FYI, when I run EnsembleApp and resize the bottom right corner like crazy for around 30 seconds, it crashes. Let me investigate this a bit more.
02-04-2013
The first frame is a known problem. The patch is a prototype, so I did not try to fix the first frame issue before we could confirm that the approach is good and it worths spending time on it. I did not see the problem with frame drops, probably because I've tested it mostly on BendingPagesTest app. Thank you for testing, I will check on this.
02-04-2013
When running guimark2.BitmapTest, using Petr's patch ("double buffer"), if I wiggle the mouse inside the window while it is running, I drop frames. FX prints good FPS numbers, but visually, frames are missing. Further, the first frame that is drawn is garbage. This does not happen with my patch ("direct draw"). Here is what I am running: Main: guimark2.BitmapTest Program args: -monsters 18000 VM args: -Djavafx.animation.fullspeed=true Oleg/Petr you confirm this behaviour?
02-04-2013
Ok, I think we should go ahead with Artem/Petr's approach. It uses more memory, but it is a safer fix.
02-04-2013
Performance comparison (as reported by JavaFX): Controls.CheckBox benchmark orig: 86.7 fps petr: 108.5 fps steve: 107.8 fps GUIMark2.Bitmap, 18000 monsters orig: 23.4 fps petr: 31.5 fps steve: 31.4 fps I used patches from Steve and Petr applied to the workspace as of this morning (03/28/2013).
28-03-2013
According to OpenGL Driver Monitor, Free Video Memory for the CheckBox benchmark in fullscreen mode changes as follows: Petr's implementation: from 670M to 483M = 187M current FX: from 670M to 497M..511M = 159M..173M (monitor's reading switches between the two values while the app is running) It's difficult to measure impact of a particular application as the monitor reports system wide values. The numbers above were reproduced several times on an otherwise quiescent system.
28-03-2013
3. What are the performace characteristics? I like the double buffering code from the stand point that it behaves nicely with respect to the OS X layer painting model. The code is a bit more complex and needs to be checked for leaks etc. Oleg, can you compare the FPS of both patches on your machine?
27-03-2013
Steve, yes, I asked Petr to prototype double buffering. The build was sent to Oleg for initial investigation, and based on his comments above, nothing critical has been found so far. However, a few questions are still open: 1. Does it work in Java Plugin? 2. What is memory (shared, video) footprint impact?
27-03-2013
Petr Pchelko provided a tentative double buffering implementation in http://bigmac.ru.oracle.com/~pchelko/26702/ I looked at the CheckBox benchmark and can confirm that the implementation provides the desired behavior: maximum parallelism with no delays in the FX thread and higher performance (improved from 64 to 80 fps as reported by JavaFX). It would be very useful to know how many frames are actually delivered to CALayer and eventually to the screen (or OpenGL at least). The OpenGL profiler does not seem reliable in that regard as it reports 140 fps (probably double counting CGLFlushDrawable calls from two separate contexts, I'm not sure why we have that).
26-03-2013
Why don't we just go with the simplest possible fix. If it turns out apple hoses us down the line, then we have a more complicated fix waiting in the wings that we can switch to.
26-03-2013
@Artem - So are we going to try out the double buffering you describe? It is possible that we will end up moving the timing issue somewhere else or dropping frames, however, we won't really know until we implement something. In step 4), I believe that setNeedsDisplay() needs to be called from the UI thread and hopefully this causes the callbacks (5 and 6) to happen right away. So, we either need an implementation of the new double buffering or we need to go with the patch in this bug report. We need to move on this to allow a lot of time for testing and debugging. Kevin? Richard?
26-03-2013
Thor, I can't say for sure whether we should have different GL contexts for textures P and L. I'm not that strong in OpenGL, unfortunately... Kevin said we currently use a single GL context to draw into all Prism textures, so the answer is probably "no".
22-03-2013
@Kevin - I believe the Quantum changes fix the "draw garbage" problem but I have not tested them indepenedently. We could consider checking into that. Is there a JIRA around for this? Do you have a repeatable case? @Kevin - You are correct. We are accessing flag from the render thread. This should not be a problem because we lock in Prism around create/destroy. I'm not a fan of adding another state variable in Glass and a lock. The only time the flag is changed dynamically is during full screen and it is possible that we could lock around that, but I wouldn't want to. There would be a lock in Java that was needed for some reason in Glass. We have is in the case of create/destroy, but it's pretty clear why.
21-03-2013
Artem is the plan to have two GL contexts. One for P the other L. Where L is a shared resource?
21-03-2013
Yes, I didn't think about it, but it indeed should make things even faster.
21-03-2013
But if you use dirty opts to render to P, you really don't want to blit all of P onto L -- that's a lot of pixels and would hurt performance, wouldn't it?
21-03-2013
Thor, you're right, if we swap buffers (textures), we'll have problems with dirty opts. However, if we use blitting instead of swapping, we'll be fine. So here are more details about double buffering: 1. There are two textures, let's say texture L (for drawing to CALayer) and texture P (for drawing from prism). 2. Prism always draws into the same texture P. 3. At the end of each rendering cycle, texture P is blitted to texture L. 4. After that, we explicitly ask the layer to repaint ([layer setNeedsDisplay]). 5. -canDrawInCGLContext:pixelFormat:forLayerTime:displayTime: always returns YES. 6. -drawInCGLContext:pixelFormat:forLayerTime:displayTime: is just blitting texture L to the passed GL context of the layer. 7. The layer operates in synchronous mode ([layer setAsynchronous:NO]). Texture P is only written to and read from on the render thread. Texture L is updated on the render thread and read on the event thread. That's why we need synchronization, which is pretty simple: take a lock at step #3 and at step #6. Both are very simple operations that are supposed to be very fast because are executed on GPU. The drawback of this approach is memory footprint (two textures instead of one).
21-03-2013
The Quantum changes look fine. As for the glass changes, are you sure that switching between async and sync (e.g., for enter / exit full screen) is recommended and robust? Since you are setting the flag from the app thread and checking the flag from the render thread (if I understand correctly...hard to see from just the diffs), can this cause problems? It seems like this could lead to hard-to-diagnose timing-related issues.
20-03-2013
The latest code works in Applet mode and is triggered by testing the asnchronous flag of CAOpenGLLayer. If drawing is asynchronous, then we won't flush from the render thread. This allow the code to be turned off easily should we encounter a problem. Let's go forward with this and get the performance back. Later we can revisit/rewrite should we decide not to use layers on the desktop. Thors discussion of layers is more around GL layers which might apply in future in either Glass or Prism.
20-03-2013
Oleg, I do remember the discussion but I don't remember the exact details about how a second texture in Glass would help. I also remember we discussed forcing a draw in the UI-thread after the Prism thread had finished.
20-03-2013
It has been a while since I looked at this problem. But here are a few thoughts: If you are starting rendering from anything besides a clear background it might be difficult to use conventional double buffer technique. And doesn���t dirtyopts present extra challenges when double buffered? In theory we could create two FBOs, one which glass lock and blits, while the other FBO can be freed for prism rendering. Assuming this is possible, my guess is we would see a performance gain when there is vsync and/or on Mac (Since CALayer blits when ever it feels like it). After all the same hardware must do the work. Off the top of my head, rendering from two different threads with different shared contexts using two shared resources FBOs is probably very difficult. I think Steve���s solution of avoid vsync and Mac rendering peculiarities is a good approach.
20-03-2013
We use only one texture (buffer) in GlassLayer3d/GlassOffscreen/GlassFrameBufferObject. We discussed with Steve that a possible solution would be a second texture (as Artem is suggesting) or a better synchronization scheme around the one we have, or whatever else.
20-03-2013
We are double buffering right now. Oleg has a pretty clear idea about how/why this is failing. There is something wrong with code that causes FPS to get faster when we add a sleep (see RT-28752 for details). I am not happy about two code paths either however, if performance is much faster on the desktop, then we should consider it. In the mean time, I think we should make the change to speed up the desktop using the existing code base earlier rather than later. If we decide to get rid of layers for the desktop, then the code can easily come out again and/or be disabled.
19-03-2013
What about double buffering? One texture is to draw in to the layer (on the event thread), another one is what Prism renders to (on the render thread). Locking will only be required to switch buffers. As for using layers in general, I think we'll eventually have no choice. Apple is pushing developer really hard to use layers, because layers leave much more room for optimizations at the platform level. Supporting two code paths, with and without layers, is also not that exciting.
19-03-2013
It is not recommended but we are in a difficult spot. The Mac has a "paint" model where the operating system wants to call you back when it is time to draw (from the UI-thread). We have a "render thread" model in Prism and graphics are expected to happen outside the UI-thread. Tjhis is not really supported with OS X layers. The code works by forcing a "paint" to be delivered from the render thread. The patch works on the desktop but not in the browser. I have code that works in both places, but it needs a bit more testing (I can finally test the applet case). I am pretty much ready to go forward if we decide to keep this approach. Since drawing from the render thread does not work in the browser, a better solution would be run the layer code when running in the browser and go back to drawing directly with no layers on the desktop. The layer code can run in the browser where live resize is not supported anyway (live resize caused all this work to happen). Unfortunately, this coder was too hard for Gerard to write and I understand the burden of having two code paths to test. In my mind, the improved performance would be worth it.
19-03-2013
Looking at Steve's patch... I am not sure I clearly understand why it works. What about [CALayer display] being explicitly not recommended to call directly? https://developer.apple.com/library/mac/documentation/graphicsimaging/reference/CALayer_class/Introduction/Introduction.html#//apple_ref/occ/instm/CALayer/display
19-03-2013
- The Quantum changes look fine to me. - I don't see anything obviously wrong with the Glass changes (although one of the glass folks may want to chime in). As to the question of whether there are any issues drawing the CALayer in the render thread: In standalone mode I don't see any issues, but it doesn't work in applet mode -- nothing is rendered at all. If I use the libglass.dylib without your changes, then it renders fine.
11-03-2013
I suggest that we release the patch and leave all the buffering code in place as it is now. If there is some problem, we can back go back easily to use asynchronous drawing. In a week or so, we can delete all the unnecessary code and replace it with something else (perhaps a callback from the operating system that Prism can use to draw and present a texture).
06-03-2013
I can confirm that Steve's patch eliminates waiting in the FX thread and boosts performance. The JPA profile shows that the two threads run in parallel on every frame.
06-03-2013
The latest version of the patch fixes the inital garbage problem. Since some changes are in code that is shared by Mac and Windows, I tried the new code on Windows too. The garbage was caused by the lock/unlock during validateStageGraphics().
06-03-2013
The problem is that we are not drawing in the render thread. Instead we are creating a backing image that eventually gets displayed by the UI thread at the best time for the operating system. Here is a patch that draws in the render thread (ignore the garbage that is drawn on the first frame). It is a proof of concept. If we decide to go with it, lots of code can be deleted.
04-03-2013
Given new data on the effect of the partial fix, more analysis and a more complete solution are required.
01-03-2013
I'm observing a behavior with the current fix which is likely to lead to more work on this issue. My original description contains at least one mistake: asynchronous calls to canDrawInCGLContext and drawInCGLContext by CAOpenGLLayer are done on the FX thread, not a separate thread. That means that delays in drawInCGLContext, while waiting for unbind to release the lock, effectively stop the FX thread from processing input or new pulse events. The mechanics are as follows: 1. the FX thread prepares frame n and passes it to the render thread 2. the FX thread responds positively to canDrawInCGLContext as frame n-1 is ready 3. the render thread starts the render job and acquires the lock in bindForWidth 4. drawInCGLContext is called on the FX thread which gets blocked on the lock until it's released by the render thread in unbind. When unlocked, it draws frame n. Frame n-1 is dropped. If 4 occurs before 3, as it does in most cases, everything is fine. If 3 occurs before 2, frame n-1 is also dropped.
16-01-2013
I'm not an expert in layers and OS X. As long as the fix eliminates jerkiness and performance regression is acceptable, I'm fine with it.
21-12-2012
Artem, are you happy with the fix?
12-12-2012
Some performance regression should be expected as now the render thread may wait for drawInCGLContext to finish its job before starting a new frame. I've been unable to reproduce the reported regression on my iMac though.
10-12-2012
I believe this is caused by fake fps measurements we're currently having. This is not 100% related to actual user experience as per this issue. However, this requires further investigation.
10-12-2012
The jerkiness is fixed. I'm wondering whether the performance regresssion is because we are now drawing correctly. Just a thought.
10-12-2012
graphics-scrum build #419 has the fix. check it out
07-12-2012
I took the liberty of implementing the simplest fix: change setting _dirty in bindForWidth from FALSE to TRUE. I'm pretty confident that's all that's needed. The logic is as follows. We notify the CA layer that we can draw whenever we start preparing a new frame (in bindForWidth). The layer issues drawInCGLContext immediately but it will not proceed until we release the lock in unbind. After that drawInCGLContext does what it's supposed to do with the freshly prepared frame and resets the flag to FALSE. Any call to canDrawInCGLContext will return FALSE until we start a new frame. Here is a diagram: time action _dirty comment --------------------------------------------- t bindForWidth _dirty=TRUE -- start frame n t1 canDraw _dirty==TRUE -- we can draw! t2 drawInCGLContex _dirty==TRUE -- hangs on the lock t3 unbind _dirty==TRUE -- frame n is ready, releases the lock t4 drawInCGLContex _dirty=FALSE -- pushes frame n, resets _dirty t5 canDraw _dirty==FALSE -- no frame to be rendered t6 canDraw _dirty==FALSE -- still no frame to be rendered ... The only scenario when we can drop a frame is when no canDraw call occurs between two consecutive bindForWidth but that means that two consecutive calls to canDraw are at least 16 ms apart. That apparently can happen but that represents a deficiency in the lower CA layer (under usual circumstances those calls are coming every 10 ms), which we can only do so much about. I marked my change as PARTIAL FIX, of course. If it is accepted the name of the property should probably change and setting it in unbind becomes redundant. Anyway, I wanted other engineers to try it out and see if there is any performance impact.
07-12-2012
Yup.
07-12-2012
Spoke to Kevin. We agree that we should disable vsync on the Mac until we sort this out. The current behavior is making us appear totally broken on the Mac.
07-12-2012
Oleg, thank you for very detailed description of what's going. I don't see what can be done in Glass to help with this. All the rendering is controlled in Quantum: where the next frame rendering is started, when it's finished, when a request to flush it on screen is sent, etc. In this particular case, the problem is that Quantum doesn't wait until the frame is blitted to CALayer. On Windows, the request (be it present() or uploadPixels()) is handled synchronously, so we can start the next frame immediately, but this is not true on Mac. We should not only notify Glass that the previous frame is ready, but also wait until drawInCGLContext: is finished.
07-12-2012
I believe I have enough understanding of the root cause if not to answer all questions at least to set the right direction. The problem is that although we render 60 frames per second only few of them reach the screen. Responsibility for that falls on GlassLayer3D.m which implements asynchronous updates using CAOpenGLLayer. Basically, we have an external thread that asynchronously calls our canDrawInCGLContext and when we return TRUE then our drawInCGLContext is invoked where we provide the actual frame for rendering. When I trace all drawInCGLContext calls I see that their number and timestamps correspond to what I see on the screen. Our response to the canDrawInCGLContext request is based on two conditions: [self needsDisplay] == YES, which is never true, or [self->_offscreen isDirty] == YES, which I consider in detail below. The dirty flag, _dirty, belongs to GlassOffscreen. It is set to FALSE when we lock GlassOffscreen in bindForWidth in the beginning of our frame processing, to TRUE when we unlock it in unbind signaling that the frame is ready, and to FALSE again in blitForWidth, part of drawInCGLContext - actual rendering. Although formally read/write accesses to that flag constitute a data race, that's not a problem by itself. The problem is that we prepare frames and respond to queries about their availability asynchronously without ensuring that every prepared frame is rendered. That is to say that we may start a new frame (and set the dirty flag to FALSE) before we answer canDrawInCGLContext positively, which effectively means that we dropp the frame. Under some circumstances, the correlation between starting a new frame and inquires about whether we canDrawInCGLContext becomes so severe that we are lucky if 10% of all prepared frames get rendered. One such case is vsync synchronization via CVOutputCallback, another - timer synchronization at 16 ms. This is why Steve was able to reproduce the problem after commenting out all vsync related code: if vsync is not available we use the timer with the default 16 ms interval. I went a step further to try 15ms and 17ms and observed that I regained most (though not all) of the smoothness we believe we deserve. Here are two diagrams taken from actual trace output that show how we can drop frames due to the asynchronous check/update mechanism described above: time action _dirty comment --------------------------------------------- t bindForWidth FALSE start frame n t+1 canDraw FALSE t+2 canDraw FALSE t+3 unbind TRUE frame n is ready t+4 bindForWidth FALSE start frame n+1 t+5 canDraw FALSE t+6 canDraw FALSE t+7 unbind TRUE frame n+1 is ready t+8 bindForWidth FALSE start frame n+2 ��� No frame reaches the screen. Another diagram: time action _dirty comment --------------------------------------------- t bindForWidth FALSE start frame n t+1 unbind TRUE frame n is ready t+2 canDraw TRUE we can draw! t+3 bindForWidth FALSE start frame n+1 t+4 unbind TRUE frame n+1 is ready t+5 drawInCGLContex N/A frame n+1 is rendered Here we respond about availability of frame n but when the actual draw call is made we render frame n+1. Frame n is lost. It appears that canDraw calls are scheduled at ~10ms on my iMac/Lion, though in many cases a particular call may be delayed so that two consecutive calls may be less than 1ms apart (that's actually what happens in the first diagram). I don't know what causes it. I also can't answer the question what role the mouse plays or may play in the picture. I think that to fix this problem we need to ensure that once a frame is prepared it is rendered. Dropping frames is not a solution for this stage in the pipeline (even if the mechanism was designed to allow for that).
07-12-2012
Oleg has a bunch more information here so he needs to comment before we go farther. The Mac my be doing something magic around the native vsync callback which is provided by Glass and used by Quantum (probably incorrectly). I am interested in seeing a case where the vsync support on Mac helps, if not, we can take it out or disable it for the time being ... which of course would be a Quantum change!
07-12-2012
Nope, I have done research. The problem is in Glass. I have commented out all code dealing with vsync (vsyncHint() etc.) and the problem remains. I could be wrong here but I have done the initial investigation.
06-12-2012
Vsync is handled by Quantum, so assigning to Thor.
06-12-2012
It's related to vsync. Running wth -Djavafx.animation.fullspeed=true fixes the jerkiness. I also commented out the place where vsync is queried and was able to get rid of the jerkiness without going full speed.
05-12-2012
Talked to Jasper about this yesterday. This looks like a glass issue on Mac where mouse events are interfering with rendering. When mouse events are processed, FX appears to drop frames such that the frames are rendered by prism but never show up in the window.
05-12-2012
Mouse events seem to kill visible updates on Mac. So everything looks jerky when dragging etc.
04-12-2012