JDK-8261632 : Cached text AA rendering crashes with Metal API validation
  • Type: Bug
  • Component: client-libs
  • Sub-Component: 2d
  • Affected Version: internal
  • Priority: P2
  • Status: Resolved
  • Resolution: Fixed
  • OS: os_x
  • CPU: generic
  • Submitted: 2021-02-12
  • Updated: 2021-02-13
  • Resolved: 2021-02-12
The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.
Other
internalFixed
Related Reports
Relates :  
JDK-8261650 - Add comment with details for MTLVC_MAX_INDEX
Relates :  
JDK-8260931 - Implement JEP 382: New macOS Rendering Pipeline
Relates :  
JDK-8261620 - Running metal with API validation crashes immediately
Description
If we enable Metal API validation and run text related tabs in J2DDemo, SwingSet2 or Font2DTest. It crashes with following error :

Enable Metal API validation using :
export METAL_DEVICE_WRAPPER_TYPE=1 

Debug log:
I] MTLTR_DrawGlyphList Grayscale cache
[I] MTLVertexCache_AddGlyphQuad
[I] Entered for loop for glyph list
[I] Glyph width = 4 height = 4
[I] rowBytes = 4
[I] MTLTR_DrawGlyphList Grayscale cache
[I] MTLVertexCache_AddGlyphQuad
[I] Entered for loop for glyph list
[I] Glyph width = 4 height = 4
[I] rowBytes = 4
[I] MTLTR_DrawGlyphList Grayscale cache
[I] MTLVertexCache_AddGlyphQuad
[I] MTLTR_DisableGlyphVertexCache
[I] MTLVertexCache_FlushGlyphVertexCache
-[MTLDebugRenderCommandEncoder setVertexBytes:length:atIndex:]:1356: failed assertion `length(7296) must be <= 4096.'
Abort trap: 6
Comments
Sanity testing with J2DDemo, SwingSet2 and Font2DTest looks fine. Performance numbers are also fine: Swingmark : Default LAF : Before fix : Setting L&F to: javax.swing.plaf.metal.MetalLookAndFeel Startup Time: 1416 Sub-Menus = 280 (Paint = 0) TextArea = 1663 (Paint = 693) Sliders = 1981 (Paint = 502) Lists = 1007 (Paint = 493) Table Rows = 1450 (Paint = 213) Tree = 4009 (Paint = 814) Score: 11942 After fix : Setting L&F to: javax.swing.plaf.metal.MetalLookAndFeel Startup Time: 1498 Sub-Menus = 294 (Paint = 0) TextArea = 1621 (Paint = 693) Sliders = 1757 (Paint = 502) Lists = 877 (Paint = 493) Table Rows = 1342 (Paint = 213) Tree = 3751 (Paint = 814) Score: 11271 Aqua LAF : Before fix : Setting L&F to: com.apple.laf.AquaLookAndFeel Startup Time: 1435 Sub-Menus = 282 (Paint = 0) TextArea = 1468 (Paint = 693) Sliders = 1119 (Paint = 502) Lists = 1211 (Paint = 494) Table Rows = 993 (Paint = 214) Tree = 4033 (Paint = 815) Score: 10870 After fix : Setting L&F to: com.apple.laf.AquaLookAndFeel Startup Time: 1435 Sub-Menus = 284 (Paint = 0) TextArea = 1469 (Paint = 693) Sliders = 996 (Paint = 502) Lists = 1283 (Paint = 494) Table Rows = 1022 (Paint = 214) Tree = 3913 (Paint = 815) Score: 10728 Nimbus LAF : Before fix : Setting L&F to: javax.swing.plaf.nimbus.NimbusLookAndFeel Startup Time: 1528 Sub-Menus = 2653 (Paint = 0) TextArea = 1477 (Paint = 602) Sliders = 2020 (Paint = 502) Lists = 884 (Paint = 493) Table Rows = 1084 (Paint = 212) Tree = 3106 (Paint = 813) Score: 13028 After fix : Setting L&F to: javax.swing.plaf.nimbus.NimbusLookAndFeel Startup Time: 1476 Sub-Menus = 2606 (Paint = 0) TextArea = 1409 (Paint = 602) Sliders = 1967 (Paint = 502) Lists = 842 (Paint = 493) Table Rows = 1092 (Paint = 212) Tree = 3404 (Paint = 814) Score: 13022 Motif LAF : Before fix : Setting L&F to: com.sun.java.swing.plaf.motif.MotifLookAndFeel Startup Time: 1453 Sub-Menus = 279 (Paint = 0) TextArea = 1447 (Paint = 792) Sliders = 595 (Paint = 502) Lists = 715 (Paint = 493) Table Rows = 850 (Paint = 213) Tree = 2706 (Paint = 813) Score: 8143 After fix : Setting L&F to: com.sun.java.swing.plaf.motif.MotifLookAndFeel Startup Time: 1479 Sub-Menus = 265 (Paint = 0) TextArea = 1420 (Paint = 792) Sliders = 594 (Paint = 502) Lists = 761 (Paint = 493) Table Rows = 837 (Paint = 213) Tree = 2571 (Paint = 813) Score: 8034 RenderPerfTest : Before fix : TextNoAA : 31.38 FPS TextGray : 30.60 FPS WhiteTextNoAA : 37.46 FPS WhiteTextGray : 38.31 FPS After fix : TextNoAA : 31.59 FPS TextGray : 29.92 FPS WhiteTextNoAA : 37.28 FPS WhiteTextGray : 39.92 FPS J2DBench : Grayscale small: Before fix : All test results: Test(text.Rendering.tests.drawString) averaged 7932047.4823492905 chars/sec with 1, Serif, ident, Off, !tfm, font, 13.0, std, english, !gaa, to Default Frame, ident, 64, plain After fix : All test results: Test(text.Rendering.tests.drawString) averaged 8082858.26957962 chars/sec with !gaa, ident, ident, std, Serif, !tfm, 13.0, to Default Frame, 1, font, english, plain, 64, Off
12-02-2021
In setVertexBytes we can pass maximum of 4KB data. Looks like in some scenarios we are passing more than4KB vertex data. Clamping vertexcache to 4KB in MTLVertexCache fixes this issue. This will increase number of text encoders per frame(in case of large amount of text), but from earlier testing i have noticed that having more encoders per frame will have negligible impact on performance compared to having multiple commandbuffer commits.
12-02-2021