Writing a test for this issue is hard because we should read taskbar buttons and check JDialog's button presence here.
There is no Java way to do this except JNI interface with the native implementation of such reading.
Moreover this will require to support several window managers.
Checking the existence of _NET_WM_STATE_SKIP_TASKBAR Atom for JDialog native window is not enough
because it doesn't actually check that the taskbar button doesn't appear.
So, changing the state to Won't fix.