JDK-6360305 : Regression: java/awt/FontClass/NegativeAngleTest/NegativeAngleTest.java fails with SIGSEGV
  • Type: Bug
  • Component: client-libs
  • Sub-Component: 2d
  • Affected Version: 1.4.2_11
  • Priority: P2
  • Status: Resolved
  • Resolution: Fixed
  • OS: solaris_8
  • CPU: sparc
  • Submitted: 2005-12-07
  • Updated: 2010-05-11
  • Resolved: 2007-11-12
The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.
Other JDK 6
1.4.2_12Fixed 6 b73Fixed
Related Reports
Relates :  
Relates :  
Relates :  
Relates :  
Relates :  
Description
1.4.2_11b01 fails the regression test java/awt/FontClass/NegativeAngleTest/NegativeAngleTest.java fails
on Solaris 8. A core dump is produced.

Appears that regression was introduced from 1.4.2_08b03(fcs) to 1.4.2_09b01

To Reproduce:
Compile and run the attached file:

mack 5.8 $  /export/home/JDK/142_11b01/j2sdk1.4.2_11/bin/javac NegativeAngleTest.java 
mack 5.8 $  /export/home/JDK/142_11b01/j2sdk1.4.2_11/bin/java NegativeAngleTest      
#
# An unexpected error has been detected by HotSpot Virtual Machine:
#
#  SIGSEGV (0xb) at pc=0xff232974, pid=3914, tid=1
#
# Java VM: Java HotSpot(TM) Client VM (1.4.2_11-ea-b01 mixed mode)
# Problematic frame:
# C  [libc.so.1+0x32974]  longjmp+0x8
#
# An error report file with more information is saved as hs_err_pid3914.log
#
# If you would like to submit a bug report, please visit:
#   http://java.sun.com/webapps/bugreport/crash.jsp
#
Abort(coredump)
mack 5.8 $         

Please see error report file attached 
Note: Used default Solaris 8 threads for this, fail also observed with T2 threads:

I tested the following variations:

Java Platform    Solaris 8 (default)  Solaris 8 (T2)    Solaris 9 (default - T2)

1.4.2_FCS                               PASS                  FAIL

1.4.2_04b05                             PASS                   -

1.4.2_07b02                             PASS                   -

1.4.2_08b03(fcs)        PASS            PASS                  FAIL

1.4.2_09b01             FAIL            FAIL                  FAIL


CR 4908028 tracked this problem on 5.0
No currently open bug is tracking this problem on 1.4.2


mack 5.8 $  /export/home/JDK/142_11b01/j2sdk1.4.2_11/bin/java -version
java version "1.4.2_11-ea"
Java(TM) 2 Runtime Environment, Standard Edition (build 1.4.2_11-ea-b01)
Java HotSpot(TM) Client VM (build 1.4.2_11-ea-b01, mixed mode)

mack 5.8 $ cat /etc/release
                       Solaris 8 2/04 s28s_hw4wos_05a SPARC
           Copyright 2004 Sun Microsystems, Inc.  All Rights Reserved.
                            Assembled 08 January 2004
mack 5.8 $

SunOS mack 5.8 Generic_117350-29 sun4u sparc SUNW,Sun-Blade-2500
This test failing  in sol10-sparc & sol10-x86 for 142_xx :

ERROR :
=======
 Assertion failed: t->numPointers == 0, file ../../../src/share/native/sun/awt/font/t2k/tsimem.c, line 88
Abort (core dumped)

But for Sol10-AMD This test passing.

OUPUT:
=====
No Fonts with negative angles - test PASSED

Comments
EVALUATION While mustang does not crash it still worth to fix potential vulnerabitiy.
04-02-2006

SUGGESTED FIX ------- autogrid.c ------- *** /tmp/sccs.bBaOGw Thu Feb 2 19:21:02 2006 --- autogrid.c Thu Feb 2 18:54:46 2006 *************** *** 924,929 **** --- 924,933 ---- lastPoint = hData->endPoint[ ctr ]; firstPoint = hData->startPoint[ ctr ]; + /* For glyph with empty contours lastPoint might be -1. + We still want to continue topology analysis for other contours though. */ + if (lastPoint < firstPoint) continue; + if ( lastPoint != firstPoint ) { /* A normal multiple point contour */ A = lastPoint; B = firstPoint; C = B + 1;
03-02-2006

EVALUATION Crash happens due to memory leak tracking assertion. It is caused by another assertion that detects memory corruption during deallocation. The real reason is memory smash due to absense of boundary checks in the ag_AnalyzeChar. For empty countour end point is set to -1 but code in the ag_AnalyzeChar does not check pointer index for validity. Interestingly but mustang and tiger builds i have tried do not crash while code there seems still vulnerable.
03-02-2006