The OOME comes from the fact that the testcase creates a new Frame
(--> a new BufferStrategy --> a new VolatileImage --> a new "backup"
BufImageSurfaceData) for each iteration. For each iteration we have:
Each "backup" surface created is about the size of the screen (in my case
that's 1600x1200) and is created in the Java heap, so if these are not
disposed quickly, they can easily exhaust the Java heap and cause the OOME.
I'm not sure what we can do to prevent this, but it's not a scenario that
most fullscreen apps are likely to run into. Transferring back to the
test's owner while we give this some more thought.
The test itself doesn't call setDisplayMode. So it appears that we
call it during the setFullScreenWindow() call (as per your
Looks like we do it when restoring the original display mode.
In this case the display mode doesn't change, so there is no need
for calling setDisplayMode.
We do the same thing on windows, except that in
Win32GD.setDisplayMode we check if the passed display mode is the
same as the current mode, and noop in this case.
We don't do this on Unix, though (see X11GD.setDisplayMode()) - not
sure why - so the display mode change event gets generated, and so
Anyway, what I'm saying is that this test shouldn't really be
causing all that carnage with display mode changes etc.
Also, on my solaris system I can't reproduce the failure because the
display mode changes aren't supported.
Reassigning back to unix FS engineer (aka the victim).
We will fix this particular problem by avoiding the actual setDisplayChange()
call when the given mode is the same as the current mode. This avoids the OOME
so that this test and other apps that use only setFullScreenWindow() (and do
not change the display mode) will now work properly. However, the underlying
cause of the OOME when calling setDisplayMode() repeatedly is still present
and will be handled under a separate bug (6459841). (That issue is less
important because it only affects testcases that really stress the calling
of setDisplayMode() frequently, which is not a common scenario in the real world.)