JDK-8115085 : Gtk: Glass should honor the contract of only use the VisualID, passed in from Prism, to create window
  • Type: Bug
  • Component: javafx
  • Sub-Component: window-toolkit
  • Affected Version: 7u6
  • Priority: P2
  • Status: Closed
  • Resolution: Fixed
  • Submitted: 2012-05-09
  • Updated: 2015-06-17
  • Resolved: 2012-05-12
The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.
JDK 7
7u6Fixed
Related Reports
Blocks :  
JDK-8101235 - Rewrite prism-es2 pipeline to not use OpenGL bindings
Description
*** This is critical bug that need to be fixed before the new prism-es2 (es2n) is fully functional ***

Glass currently creates transparent window without using the VisualID passed in by Prism. This broke the contract between Prism and Glass, and causes user program with transparent Stage to crash. Since the VisualID use in Prism (es2n) is alpha channel capable, Glass should simply use it. The fix in Glass maybe as simple as the change:


--- a/glass/glass-mat-lib-gtk/src/com/sun/glass/ui/glass_gtkcompat.c	Tue May 08 14:55:17 2012 -0700
+++ b/glass/glass-mat-lib-gtk/src/com/sun/glass/ui/glass_gtkcompat.c	Wed May 09 15:20:49 2012 -0700
@@ -310,6 +310,7 @@
 
 static gboolean
 configure_window_transparency(GtkWidget *window, gboolean transparent) {
+/*
     if (transparent) {
         if (configure_transparent_window(window)) {
             return TRUE;
@@ -319,8 +320,13 @@
                " support alpha channel."
                " You need to enable XComposite extension.\n");
     }
+*/   
+    configure_opaque_window(window);
 
-    configure_opaque_window(window);
-    return FALSE;
+    if (transparent) {
+        fprintf(stderr, "Need to clear the alpha channel of transparent window\n");
+    }
+    
+    return TRUE;
 }
Comments
verified in 2.2.0b18
26-07-2012
Fixed as suggested in the previous comment.
12-05-2012
This patch looks fine to me.
12-05-2012
OK. Thanks for the information. I did a quick test of option #2, by overriding shouldUpdateWindows() in GtkApplication seems to work well on transparent Stage. I tested with HelloComboBox, HelloTooltip and Ensemble2 and work without problem. This is the only change I did in Glass without making any change to Prism. I agree with Kevin that this is the safest approach for 2.2 release. We can look into option #1 if performance is an issue: cyang@cyang-Latitude-E6510:~/java/javafx/220/jfx/rt-closed$ hg diff diff -r 1721de2065dd glass/glass-mat/src/com/sun/glass/ui/gtk/GtkApplication.java --- a/glass/glass-mat/src/com/sun/glass/ui/gtk/GtkApplication.java Fri May 11 14:41:15 2012 +0200 +++ b/glass/glass-mat/src/com/sun/glass/ui/gtk/GtkApplication.java Fri May 11 13:59:19 2012 -0700 @@ -91,7 +91,11 @@ } super.finishTerminating(); } - + + @Override public boolean shouldUpdateWindow() { + return true; + } + private native void _terminateLoop(); private native void _init();
11-05-2012
1. 0x74 vs. 0x28. It's not only the alpha channel that matters for transparent windows on X11. The visual must be "special" in some way. This is determined with the XRenderFindVisualFormat() function. See src/solaris/native/sun/awt/awt_GraphicsEnv.c in JDK sources for details. Just being a 32bit visual is not enough. 2. Try setting a non-opaque fill for the rectangle (e.g. (255, 0, 0, 100) instead of (255, 0, 0, 255)) and see if the rectangle is still opaque. For both OPAQUE and UNDECORATED stages it should be opaque, and only if the stage is TRANSPARENT should the rectangle become visually semi-transparent. Even if #2 passes well on your system with the 0x74 visual for all kinds of stages, I doubt it will be the case on all other systems. So I'm really reluctant to using a transparency-capable visual for all windows. I'd prefer to use it for TRANSPARENT stages only.
11-05-2012
I did the following experiment to see is sharing a single visualID workable. 1) Instrumented Prism to send an argb-capable visualID to Glass (in my case it is 0x74). 2) Modified HelloRectangle in 2 tests as suggested by Kevin: Test A: Change stageStyle = StageStyle.TRANSPARENT and Scene fill = Color.TRANSPARENT Result: I see a red rectangle in the middle of a transparent window Test B: Contine with Test A change stageStyle = StageStyle.UNDECORATED Result: I see a WHITE background around the red rectangle even though the scene fill is TRANSPARENT A question for Anthony: Whether this just works on certain configurations / systems, or whether this really is a robust solution.
11-05-2012
Qt. for Anthony: Just for my curiosity, referring to the partial table I included earlier, do you know the different between visualId 0x28 and visual 0x74 to glass? It seems like glass treated them differently though they both have a alpha channel.
11-05-2012
Yes, #2 is a slow path (and Kevin did point out that Windows is using it). I hate to give up what we have now to go with #1 or #2 if we can avoid. Kevin and I have chatted. I will do some testing on my side. I will get back to you once I have the result to decide which direction we will take. Thanks!
11-05-2012
#2 isn't Linux-specific. It's used on MS Windows, too. The real issue here is that if you use a transparent visual and draw something with alpha == 0, the area will be transparent. If this is happening to a window which should be OPAQUE from FX perspective, this is a bug. And we can't prevent user code from painting something using a transparent color even when they paint into an opaque window. Thus, opaque windows should use the default (or, perhaps, some special, GL-enabled) visual, and transparent windows must use an argb-capable visual.
11-05-2012
Yes, I understand visualID vary from system to system that is why I mentioned on my system as an example. It is important for us to understand the real issue before we go for option 1 or 2, as non is ideal to me. Either one is now an Linux specific implementation which I prefer not to have if possible.
11-05-2012
The ID numbers vary from system to system, so we can't just hardcode it. Note that it may happen that Glass will use the same visual ID for all windows if X11 (and thus GTK) report an argb-capable Visual as the default one. I'd consider this strange anyway, since it may lead to visual artifacts for transparency-unaware applications. In general case, Glass will use the default Visual for opaque windows, and an argb-capable one for transparent windows.
11-05-2012
Anthony has already indicated that we cannot use the same visual ID for both opaque and transparent windows on GTK.
11-05-2012
To elaborate my point, I have included a partial table please see below. Currently, on my system, prism-es2 passes to glass an opaque visualID (0x27) and prism-es2n passes a transparent visualID (0x28). In our instrumented test, we found that glass uses a transparent visualID (0x74) for transparent window, we then hardcoded 0x74 as the pass in VisualID to glass. Glass uses it for all opaque and transparent windows. This approach seems to work fine in our testing of a number of JavaFX programs. Do you see problem with this approach? The trick seems to be finding the right visualID for both opaque and transparent. 84 GLX Visuals visual x bf lv rg d st colorbuffer sr ax dp st accumbuffer ms cav id dep cl sp sz l ci b ro r g b a F gb bf th cl r g b a ns b eat ---------------------------------------------------------------------------- .... 0x027 24 tc 0 32 0 r y . 8 8 8 0 . s 4 24 0 16 16 16 16 0 0 None 0x028 24 tc 0 32 0 r y . 8 8 8 8 . s 4 24 0 16 16 16 16 0 0 None .... 0x074 32 tc 0 32 0 r . . 8 8 8 8 . s 4 24 8 16 16 16 16 4 1 Ncon
11-05-2012
I don't quite understand what you mean by "to hardcode" in this context. Could you elaborate please?
11-05-2012
In our instrumented test, we were able to hardcode a transparent visual (by using xwininfo to readback the visualid set by glass) for both opaque or transparent window. Is this a possible approach? This will be the simplest design if there isn't any drawback.
11-05-2012
We initially were thinking #2, and in fact we instrumented shouldUpdateWindow to force it down that path, but had problems. I think that was before Chien fixed the offscreen rendering, though, so I would expect it might just work now. Chien: what do you think?
11-05-2012
So, what option do we choose? 1. Prsim passes in two visual ids. or 2. We use uploadPixels() for transparent windows on GTK.
11-05-2012
Yes, we can do that.
11-05-2012
Btw, the plan of passing in two visual IDs in the device details map seems good. I presume glass will check for the existence of the key and create its own visual if not set?
11-05-2012
Actually, we might still use a single context in the ES2N, but use the uploadPixels() for transparent windows on GTK. This is how they work on MS Windows (see Application.shouldUpdateWindow() in Glass and the corresponding logic in Quantum).
11-05-2012
The black border is the result of using a NOT argb-capable visual, but still considering the window transparent and painting pixels with ARGB values == (0, 0, 0, 0), which would be transparent if the visual is argb-capable, but get painted black with a transparency-unaware visual.
11-05-2012
This is how transparency works in X11. It's only controlled by the visual a window is created with. If the visual is argb-capable the window is transparent, if not - it will be opaque. Creating an opaque window with a transparent visual won't work correctly - transparent areas will shine through.
11-05-2012
The current design of the ES2N pipeline only uses a single OpenGL context for all windows. X11 will not allow a context to be used on a drawable whose visual is different from that used to create the context, so this will need to be addressed if there really is no way to use the same XVisual for all windows.
11-05-2012
Chien: I'm attaching a more appropriate patch that does what you've suggested. However, there are issues with it: 1. I'm unable to enable the es2n pipeline. QuantumRenderer throws out an exception at startup: "no suitable pipeline found" at QuantumRenderer.java:247. Could you apply the patch I've attached and see if this proof-of-concept solution works for you? 2. The suggested fix is not quite good. On X11 opaque windows must use an opaque visual, and only transparent windows must use an ARGB-enabled visual. If we create an opaque window with a transparent visual, the window will actually be transparent - i.e. non-opaque areas will shine through which is incorrect. The current implementation of Glass GTK does the following: 1. It always applies the XVisualID passed through the DeviceDetails map (see glass_window.cpp:386). 2. Later, at line 398 of the same file we call glass_gtk_configure_transparency_and_realize(), which overrides the visual for any window. If the window should be opaque or transparency is unsupported, it installs the default visual (see glass_gtkcompat.c:308), otherwise an argb-capable visual is installed (see line 120). So it looks like we have to pass two different visual ids via the DeviceDetails map. I propose to pass an opaque visual via the old "XVisualID" key in the device details map to preserve compatibility with other Glass implementations (embedded in the first place), and also add an "XTransparentVisualID" key and pass an argb-capable visual id through it. Does this sound like a plan? If you could verify the proof-of-concept solution, and apply the proposed changes today, and also provide me with instructions on how to build and run the es2n pipeline, I could experiment with it tomorrow since it's going to be a work day for me.
11-05-2012
HelloComboBox and HelloTooltip are 2 good programs for testing this fix. Please use -Dprism.order=es2n when verifying the fix.
11-05-2012
This fix is in the right direction, but it still missing the clear operation to the background I think something like the following: if (isTransparent) { GdkRGBA rgba = { 1.0, 1.0, 1.0, 0.0 }; gdk_window_set_background_rgba(gtk_widget_get_window(window), &rgba); } (Please see glass_gtk_configure_transparency_and_realize for the case where GTK_CHECK_VERSION(3, 0, 0) is true) My test on HelloComboBox the pulldown menu has black border instead of transparent,
11-05-2012
Up priority to critical since the switch to the new prism-es2 (es2n) pipe is dependence on this fix.
09-05-2012