JDK-8277299 : STACK_OVERFLOW in Java_sun_awt_shell_Win32ShellFolder2_getIconBits
  • Type: Bug
  • Component: client-libs
  • Sub-Component: java.awt
  • Affected Version: 17
  • Priority: P3
  • Status: Resolved
  • Resolution: Fixed
  • OS: windows
  • CPU: x86
  • Submitted: 2021-11-16
  • Updated: 2022-06-16
  • Resolved: 2021-12-10
The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.
JDK 17 JDK 18 JDK 19
17.0.3Fixed 18 b28Fixed 19Fixed
Related Reports
Relates :  
Description
The fix for JDK-8182043 causes a regression for 32 bit builds. The change of MAX_ICON_SIZE from 128 to 256 increases stack usage by 370k, which causes an overflow because the default Windows 32 bit stack size is 320k. 

Workaround is to set stack size, e.g. -Xss1m

Reproduces with 

import javax.swing.UIManager;

public class Repro {
    public static void main(final String... args) throws Exception {
        UIManager.setLookAndFeel("com.sun.java.swing.plaf.windows.WindowsLookAndFeel");
        UIManager.getIcon("Tree.openIcon"); // Same with "Tree.closedIcon" and "Tree.leafIcon"
    }
}

Comments
Fix Request Makes colorBits and maskBits dynamic variables to force allocation on the heap instead of the stack and avoid a stackoverflow. Also adds a test to confirm this stackoverflow is no longer happening. Tier1 and the newly added test passes on Windows. Clean backport. ^ copied from PR
28-01-2022

[~macarte] When adding fix request labels, please also add a "Fix Request" comment to the bug. This helps evaluating the request. Thanks! See description here: https://openjdk.java.net/projects/jdk-updates/approval.html
28-01-2022

A pull request was submitted for review. URL: https://git.openjdk.java.net/jdk17u-dev/pull/98 Date: 2022-01-14 20:52:25 +0000
18-01-2022

Microsoft will pick up the backport. No problem. Thanks for the feedback!
06-01-2022

> Will this be backported to 17? Since Oracle is no longer contributing to the open JDK 17u updates, the backport will need to be initiated and done by an external contributor. Also since Oracle does not produce 32 bit binaries, and this is merely a nice-to-have for 64 bit binaries, since the larger stack size there can accommodate the larger icons, there's no visible benefit to an Oracle JDK backport.
06-01-2022

Will this be backported to 17?
16-12-2021

Changeset: 94127f43 Author: Alexander Zuev <kizune@openjdk.org> Date: 2021-12-10 20:27:07 +0000 URL: https://git.openjdk.java.net/jdk18/commit/94127f43a4a28a89094fa93cd1da49763134f9db
10-12-2021

ah .. so they are ! I don't know why my brain assumed bytes. These likely should always have been on heap as it could have been 128K even before the change.
16-11-2021

the arrays are longs so thats 16k*4 and 64k*4 respectively
16-11-2021

I see in Java_sun_awt_shell_Win32ShellFolder2_getIconBits that the pre-existing code : long colorBits[MAX_ICON_SIZE * MAX_ICON_SIZE]; previously would max out at 16K, now its 64K. For good measure there's also long maskBits[MAX_ICON_SIZE * MAX_ICON_SIZE]; So we could be up to 128K .. Hmm .. seems like an increase of up to 96K not 370K .. I must be missing something else. These allocations maybe should be on the heap not the stack.
16-11-2021