JDK-6650666 : Java_sun_awt_color_CMM_cmmCombineTransforms contains error in use of unions/casting
  • Type: Bug
  • Component: client-libs
  • Sub-Component: 2d
  • Affected Version: 6
  • Priority: P4
  • Status: Closed
  • Resolution: Cannot Reproduce
  • OS: generic
  • CPU: other
  • Submitted: 2008-01-15
  • Updated: 2022-11-20
  • Resolved: 2022-11-20
Related Reports
Relates :  
Description
The fix for Sunbug 62955235 contains a change to the way transform pointers are marshalled and unmarshalled into jlong fields. On some compilers (notably gcc for Linux PPC32, xlC_r on AIX and the zOS compiler) the way that data is packed into unions means that data is lost passing into Java_sun_awt_color_CMM_cmmCombineTransforms.

In CMM.c, when a transform is loaded in java_sun_awt_color_CMM_cmmGetTransform, the pointer to the xform structure is passed into the storeID_t union(theXform) to convert it to a jlong.

In Java_sun_awt_color_CMM_cmmCombineTransforms previously the code used a storeID_t union to convert the jlong data back into the transform pointer.

After the fix for 6295525 was applied, a new function getObjectID was created which unmarshalls the pointer from the jlong by casting it. This makes assumptions about the way the shorter fields are being encoded into the union data area - which is unspecified and compiler-specific.

The original code for getObjectID was:

static SpStatus_t getObjectID (JNIEnv *env, jobject theObject, SpXform_p theID)
{
jclass        cls;
jfieldID    fid;
    cls = (*env)->GetObjectClass (env, theObject);
    fid = (*env)->GetFieldID (env, cls, "ID", "J");

    if (fid == 0) {
        return SpStatFailure;
    }

    *theID = (const struct SpXform_t_tag*)(*env)->GetLongField (env, theObject, fid);
    return SpStatSuccess;
}

The fix we've applied is:

static SpStatus_t getObjectID (JNIEnv *env, jobject theObject, SpXform_p theID)
{
jclass        cls;
jfieldID    fid;
storeID_t   theXform;
    cls = (*env)->GetObjectClass (env, theObject);
    fid = (*env)->GetFieldID (env, cls, "ID", "J");

    if (fid == 0) {
        return SpStatFailure;
    }

    theXform.j = (*env)->GetLongField (env, theObject, fid);
    *theID = theXform.xf;

    return SpStatSuccess;
}

Comments
The affected code was deleted from JDK
20-11-2022