JDK-6275887 : PIT:D3D: VM Crashes when using D3D with FullScreen Frame and ALT+TABing a couple of times, Win32
  • Type: Bug
  • Component: client-libs
  • Sub-Component: 2d
  • Affected Version: 6
  • Priority: P3
  • Status: Closed
  • Resolution: Fixed
  • OS: windows_xp
  • CPU: x86
  • Submitted: 2005-05-25
  • Updated: 2008-02-05
  • Resolved: 2005-12-05
The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.
JDK 6
6 b63Fixed
Related Reports
Duplicate :  
JDK-6275905 - PIT: D3D: VM crash when running a buffer strategy - FullScreen app and ALT + TABing, Win32
Relates :  
JDK-6429971 - D3D: ALT + TABing a FS application crashes the VM when AA is enabled, WinXP
Description
I am running an application that creates many animating balls on the screen when clicking inside a fullscreen frame. I am running this with '-Dsun.java2d.d3d=true' flag. FullScreen frame appears on the screen and I am creating as many balls as I could by clicking on the frame at different locations. When the app is running, I am pressing ALT + TAB 3-4 times and switching back and forth between application and desktop. I am getting a VM crash at the end. Sometimes it happens second time, some other time it happens 4th time. 

This is noticed only on the following PIT build on WinXP (config details given under comments):

java version "1.6.0-2d.pit-jcg-win-03"
Java(TM) 2 Runtime Environment, Standard Edition (build 1.6.0-2d.pit-jcg-win-03-java2d_24_may_2005_00_34-b00)
Java HotSpot(TM) Client VM (build 1.6.0-ea-b37, mixed mode)

I have attached a sample test. Execute the sample test with D3D flag. You would see a FullScreen frame. Create as many balls as you like by clicking on the frame. Press ALT+TAB 3-4 times and you would see a VM crash on the console. Here is the crash information:

#
# An unexpected error has been detected by HotSpot Virtual Machine:
#
#  EXCEPTION_ACCESS_VIOLATION (0xc0000005) at pc=0x73956201, pid=3344, tid=2788
#
# Java VM: Java HotSpot(TM) Client VM (1.6.0-ea-b37 mixed mode)
# Problematic frame:
# C  [D3DIM700.DLL+0x16201]
#

---------------  T H R E A D  ---------------

Current thread (0x0aec3918):  JavaThread "Thread-2" [_thread_in_native, id=2788]

siginfo: ExceptionCode=0xc0000005, writing address 0x0b5c0700

Registers:
EAX=0x0b5c0700, EBX=0x00000870, ECX=0x0000021c, EDX=0x000a0454
ESP=0x0b25f71c, EBP=0x0b25f728, ESI=0x0ae30030, EDI=0x0b5c0700
EIP=0x73956201, EFLAGS=0x00010202

Top of Stack: (sp=0x0b25f71c)
0x0b25f71c:   00000142 0009e9e0 0009e9e0 0b25f780
0x0b25f72c:   73957450 0ae30030 0000005a 0009e9e0
0x0b25f73c:   0000000f 00000022 0b25fbe4 77c35c94
0x0b25f74c:   ffffffff 0af1cfe4 0af1cfc8 77c2c3e7
0x0b25f75c:   0aebaee0 0ad267cd 0af1cfc8 00000008
0x0b25f76c:   00000004 0ad26638 0009e9e0 0b25f7ac
0x0b25f77c:   0b25f7ac 0b25f7ac 73958ee0 00000001
0x0b25f78c:   0ae9c700 0ae308a0 0000000f 0000005a 

Instructions: (pc=0x73956201)
0x739561f1:   00 8b d9 8d 96 74 1a 00 00 8b 32 c1 e9 02 8b f8
0x73956201:   f3 a5 8b cb 83 e1 03 f3 a4 89 02 e9 9d 00 00 00 


Stack: [0x0b220000,0x0b260000),  sp=0x0b25f71c,  free space=253k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
C  [D3DIM700.DLL+0x16201]
C  [D3DIM700.DLL+0x17450]
C  [D3DIM700.DLL+0x18ee0]

Java frames: (J=compiled Java code, j=interpreted, Vv=VM code)
J  sun.java2d.d3d.D3DRenderer.devFillSpans(JJLsun/java2d/pipe/SpanIterator;JII)V
J  sun.java2d.d3d.D3DRenderer.fill(Lsun/java2d/SunGraphics2D;Ljava/awt/Shape;)V
J  sun.java2d.d3d.D3DRenderer.fillOval(Lsun/java2d/SunGraphics2D;IIII)V
J  sun.java2d.SunGraphics2D.fillOval(IIII)V
J  Ball.paint(Ljava/awt/Graphics;Ljava/awt/Color;)V
J  BouncingBalls.renderOffscreen()V
J  BouncingBalls.render(Ljava/awt/Graphics;)V
j  BouncingBalls$BallThread.run()V+13
v  ~StubRoutines::call_stub

---------------  P R O C E S S  ---------------

Java Threads: ( => current thread )
  0x000360f8 JavaThread "DestroyJavaVM" [_thread_blocked, id=2684]
=>0x0aec3918 JavaThread "Thread-2" [_thread_in_native, id=2788]
  0x0aeb9120 JavaThread "AWT-EventQueue-0" [_thread_blocked, id=2764]
  0x0aeb44d8 JavaThread "AWT-Windows" daemon [_thread_in_native, id=2872]
  0x0aeb38f8 JavaThread "AWT-Shutdown" [_thread_blocked, id=3488]
  0x0aeabd18 JavaThread "Java2D Disposer" daemon [_thread_blocked, id=3476]
  0x00a66198 JavaThread "Low Memory Detector" daemon [_thread_blocked, id=2316]
  0x00036958 JavaThread "CompilerThread0" daemon [_thread_blocked, id=2568]
  0x00a63200 JavaThread "Signal Dispatcher" daemon [_thread_blocked, id=2732]
  0x00a627d8 JavaThread "Attach Listener" daemon [_thread_blocked, id=2728]
  0x00a580d0 JavaThread "Finalizer" daemon [_thread_blocked, id=2700]
  0x00a56c58 JavaThread "Reference Handler" daemon [_thread_blocked, id=2696]

Other Threads:
  0x00a0c478 VMThread [id=2692]
  0x00a536c0 WatcherThread [id=2140]

VM state:not at safepoint (normal execution)

VM Mutex/Monitor currently owned by a thread: None

Heap
 def new generation   total 576K, used 219K [0x02ae0000, 0x02b80000, 0x02fc0000)
  eden space 512K,  42% used [0x02ae0000, 0x02b16bf0, 0x02b60000)
  from space 64K,   1% used [0x02b60000, 0x02b602c8, 0x02b70000)
  to   space 64K,   0% used [0x02b70000, 0x02b70000, 0x02b80000)
 tenured generation   total 4180K, used 3398K [0x02fc0000, 0x033d5000, 0x06ae0000)
   the space 4180K,  81% used [0x02fc0000, 0x03311bc0, 0x03311c00, 0x033d5000)
 compacting perm gen  total 8192K, used 4191K [0x06ae0000, 0x072e0000, 0x0aae0000)
   the space 8192K,  51% used [0x06ae0000, 0x06ef7fa8, 0x06ef8000, 0x072e0000)
No shared spaces configured.

Dynamic libraries:
0x00400000 - 0x0040c000 	y:\disk05\2d-SQE\PIT\2005-05-24.mustang\windows-i586\bin\java.exe
0x7c900000 - 0x7c9b0000 	C:\WINDOWS\system32\ntdll.dll
0x7c800000 - 0x7c8f4000 	C:\WINDOWS\system32\kernel32.dll
0x77dd0000 - 0x77e6b000 	C:\WINDOWS\system32\ADVAPI32.dll
0x77e70000 - 0x77f01000 	C:\WINDOWS\system32\RPCRT4.dll
0x77c10000 - 0x77c68000 	C:\WINDOWS\system32\MSVCRT.dll
0x6d710000 - 0x6d8a0000 	y:\disk05\2d-SQE\PIT\2005-05-24.mustang\windows-i586\jre\bin\client\jvm.dll
0x77d40000 - 0x77dd0000 	C:\WINDOWS\system32\USER32.dll
0x77f10000 - 0x77f56000 	C:\WINDOWS\system32\GDI32.dll
0x76b40000 - 0x76b6d000 	C:\WINDOWS\system32\WINMM.dll
0x10000000 - 0x10008000 	y:\disk05\2d-SQE\PIT\2005-05-24.mustang\windows-i586\jre\bin\hpi.dll
0x76bf0000 - 0x76bfb000 	C:\WINDOWS\system32\PSAPI.DLL
0x003c0000 - 0x003cc000 	y:\disk05\2d-SQE\PIT\2005-05-24.mustang\windows-i586\jre\bin\verify.dll
0x003d0000 - 0x003ed000 	y:\disk05\2d-SQE\PIT\2005-05-24.mustang\windows-i586\jre\bin\java.dll
0x003f0000 - 0x003ff000 	y:\disk05\2d-SQE\PIT\2005-05-24.mustang\windows-i586\jre\bin\zip.dll
0x0ace0000 - 0x0ae6e000 	Y:\disk05\2d-SQE\PIT\2005-05-24.mustang\windows-i586\jre\bin\awt.dll
0x73000000 - 0x73026000 	C:\WINDOWS\system32\WINSPOOL.DRV
0x76390000 - 0x763ad000 	C:\WINDOWS\system32\IMM32.dll
0x774e0000 - 0x7761d000 	C:\WINDOWS\system32\ole32.dll
0x73760000 - 0x737a9000 	C:\WINDOWS\system32\ddraw.dll
0x73bc0000 - 0x73bc6000 	C:\WINDOWS\system32\DCIMAN32.dll
0x73940000 - 0x73a10000 	C:\WINDOWS\system32\D3DIM700.DLL
0x0b0b0000 - 0x0b100000 	Y:\disk05\2d-SQE\PIT\2005-05-24.mustang\windows-i586\jre\bin\fontmanager.dll

VM Arguments:
jvm_args: -Dsun.java2d.d3d=True
java_command: BouncingBalls
Launcher Type: generic

Environment Variables:
PATH=y:\disk05\2d-SQE\PIT\2005-05-24.mustang\windows-i586\bin;C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\system32\WBEM;C:\PROGRA~1\MKSTOO~1\bin;C:\PROGRA~1\MKSTOO~1\bin\x11;C:\PROGRA~1\MKSTOO~1\mksnt;C:\PROGRA~1\ULTRAE~1
USERNAME=praveen
SHELL=C:/PROGRA~1/MKSTOO~1/mksnt/sh.exe
DISPLAY=:0.0
OS=Windows_NT
PROCESSOR_IDENTIFIER=x86 Family 15 Model 2 Stepping 9, GenuineIntel



---------------  S Y S T E M  ---------------

OS: Windows XP Build 2600 Service Pack 2

CPU:total 1 family 15, cmov, cx8, fxsr, mmx, ht

Memory: 4k page, physical 522992k(235992k free), swap 1278288k(971316k free)

vm_info: Java HotSpot(TM) Client VM (1.6.0-ea-b37) for windows-x86, built on May 19 2005 01:42:56 by "java_re" with MS VC++ 6.0


###@###.### 2005-05-25 13:26:21 GMT
Comments
EVALUATION I've rebooted the test system and run the tests again and I can still reproduce the crash, but in a different place this time. So it looks like the fix was incomplete. It still intermittently crashes during a random DrawPrimitive call when a surface loss happens. Our code relies on DrawPrimitive to report a surface loss. Well, guess what - it doesn't do it. So we continue to render to the lost surface because no error is given. And at some point the d3d runtime dies (so far I've only seen this on older/cheaper nvidia boards). I still believe that this is a driver bug - it shouldn't crash. Even if it doesn't return a surface loss error code, it should live until an EndScene which does return a error in this case. But unfortunately we try to optimize the nbumber of BeginScene/EndScene calls, so these two can be very far apart timewise, with a lot of d3d rendering in between, and the chances of running into the bug are high. If we survive until a ddraw blit from the backbuffer to the screen (or any other situation which causes an EndScene, like a clip change or any other rendering mode change), we correctly notice the surface loss and react appropriately. So it appears that the only way is to explicitly check if the target surface was lost before any DrawPrimitive operation to avoid running into driver bugs like this. Unfortunately, this approach is pretty expensive: on 1x1 primitives the performance penalty is up to 20%. But at this point I don't see any other way. The good news is tha with this fix the crashes disappear completely.
11-11-2005
SUGGESTED FIX Minimal fix: *** /tmp/geta8904 Mon Nov 7 17:24:33 2005 --- D3DRuntimeTest.cpp Mon Nov 7 16:45:21 2005 *************** *** 329,335 **** } } res = d3dContext->ForceEndScene(); ! d3dDevice->SetTexture(0, NULL); } // REMIND: at this point we ignore the results of // the test. --- 329,335 ---- } } res = d3dContext->ForceEndScene(); ! d3dContext->SetTexture(NULL); } // REMIND: at this point we ignore the results of // the test. More involved fix: *** /tmp/geta8987 Mon Nov 7 17:39:49 2005 --- D3DContext.cpp Mon Nov 7 17:29:26 2005 *************** *** 622,628 **** BOOL bResetTexture = ((opState & STATE_TEXTURE) && lastTexture[0] != NULL); if (bResetTexture) { ! d3dDevice->SetTexture(0, NULL); } D3DMATRIX tx, idTx; --- 622,628 ---- BOOL bResetTexture = ((opState & STATE_TEXTURE) && lastTexture[0] != NULL); if (bResetTexture) { ! SetTexture(0, NULL); } D3DMATRIX tx, idTx; *************** *** 691,701 **** // reset the transform d3dDevice->SetTransform(D3DTRANSFORMSTATE_WORLD, &tx); - // reset the texture if needed - if (bResetTexture) { - d3dDevice->SetTexture(0, lastTexture[0]); - } - // reset the alpha compositing d3dDevice->SetRenderState(D3DRENDERSTATE_ALPHABLENDENABLE, dwAlphaSt); d3dDevice->SetRenderState(D3DRENDERSTATE_SRCBLEND, dwSrcBlendSt); --- 691,696 ---- *************** *** 1117,1123 **** --- 1112,1121 ---- res = d3dDevice->SetTexture(dwStage, newTexture); if (SUCCEEDED(res)) { lastTexture[dwStage] = newTexture; + } else { + lastTexture[dwStage] = NULL; } + } return res; }
08-11-2005
EVALUATION OK, it looks like I've located the problem. When we test the newly created device for rendering quality and capabilities, we create a temp texture which we use for rendering the patterns. When we're done with the texture, it's removed from the device using SetTexture(NULL) and is disposed of. Unfortunately instead of using D3DContext::SetTexture() to null out the current texture I used IDirect3DDevice7::SetTexture instead. It does remove the texture from the device, but doesn't invalidate D3DContext's current set texture cache. So we continue to think that we still have a texture set to the device. Then, later we attempt to re-install this cached (and disposed of) texture to the device, so at some point the Direct3D runtime dies when trying to use freed pointer. We reset the texture back after setting a new clip. But it's not really necessary to do it at that point - it'll be attempted to be set later anyway (when doing a blit or text rendering). A minimal fix is to use proper SetTexture call (D3DContext::SetTexture). A bit more elaborate fix would make sure we never try to reset cached textures to the device (which we do in SetClip) as we may not be sure that it's still valid.
08-11-2005
EVALUATION This is a known problem. On some video boards (mostly on nvidias) the applicaion sometimes crashes in the DirectX7 dll during the alt+tab: # Problematic frame: # C [D3DIM700.DLL+0xfb7a] Specifically, it dies in a DrawPrimitive call, the same instant the display mode changes (presumably, that's when the target surface is lost). I found that it crashes more readily on MX-based Nvidia line. It never crashes if a reference rasterizer is used instead of T&L or HAL. At this point it's not clear why does it die there. ###@###.### 2005-05-25 14:52:56 GMT
25-05-2005