United StatesChange Country, Oracle Worldwide Web Sites Communities I am a... I want to...
JDK-6434840 : Memory Leak in XSL Transform leading to OutOfMemory Exception

Details
Type:
Bug
Submit Date:
2006-06-06
Status:
Resolved
Updated Date:
2012-04-25
Project Name:
JDK
Resolved Date:
2007-03-27
Component:
xml
OS:
solaris_9,windows_xp
Sub-Component:
javax.xml.transform
CPU:
x86,sparc
Priority:
P4
Resolution:
Fixed
Affected Versions:
orion3,5.0,5.0u6
Fixed Versions:
5.0u12 (b02)

Related Reports
Duplicate:
Duplicate:

Sub Tasks

Description
FULL PRODUCT VERSION :
java version "1.5.0_06"
Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_06-b05)
Java HotSpot(TM) Client VM (build 1.5.0_06-b05, mixed mode, sharing)


ADDITIONAL OS VERSION INFORMATION :
Microsoft Windows XP
Professional Version 2002
Service Pack 2


A DESCRIPTION OF THE PROBLEM :
The XSL Transformation code in the standard JDK has a severe memory leak, which leads to an OutOfMemory exception when it is executed multiple times in a program.  To replicate, run the test program.

When the start button is pressed in the test program, it creates and runs a new thread which performs an XSL transformation.  This transformation reads an input XML file, and then generates an output HTML file, based on the rules XSLT file which is also read in.  Looking at the test program, you will notice that there are no dangling references to any of the transformation objects.  Start the test program under a profiler, and then monitor the memory usage.  For each successive click on the start button, the memory profile of the program exhibits a class memory leak i.e. memory usage grows until the JVM runs out of memory, whereby an OutOfMemory Exception is thrown.

I have included a sample input XML file and rules XSLT file, however any valid XML and appropriate XSLT document will do.  You may wish to use a larger XML file, and run the test program with a small JVM memory size, to more easily show the memory leak.

I have looked at the JVM heap under a profiler, and I believe that the bug ultimately lies in com.sun.org.apache.xml.internal.utils.XMLReaderManager.  This class is a factory, which creates instances of XMLReader when getXMLReader() is called.  The class also caches created XMLReaders, on a per-thread basis.  When a thread is finished with an XMLReader, it calls releaseXMLReader() - XMLReaderManager marks its cached instance of XMLReader as being free, but hangs onto the instance.  This is the problem, as the XMLReader has a reference to the complete source XML document that it has just transformed, as well as some other data structures.

The problem gets worse when the thread that performed the XSL transformation dies.  The cached instance of XMLReader in XMLReaderManager is automatically freed, as per the semantics of the ThreadLocal var that is fine.  But the  Hashtable in XMLReaderManager still hangs onto a ref to the XMLReader instance, because the instance is a key to an entry in that table.  So, if you start up a background thread to perform the transformation, and you do this multiple times as in the test program, you have multiple instances of XMLReader and the complete XML document being hung onto, and never freed.  Hence the OutOfMemory exception.

As a comparison, you can comment out the creation of the background thread in the test program, and just do the transformation in the Swing thread, when the start button is pressed.  You will notice in the profiler that only one instance of XMLReader is being held onto by XMLReaderManager, and successive clicks on the start button does not lead to an OutOfMemory exception; the XMLReader though still holds onto the last XML document it transformed, when really that memory should be released post the transformation.

I could be wrong about the ultimate source of the bug being XMLReaderManager i.e. the bug instead could be that reset() in not being called appropriately on the cached XMLReaders, which is meant to empty its state.  I am not wrong though about a memory leak in the XSL Transformation.


STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
See above notes.


EXPECTED VERSUS ACTUAL BEHAVIOR :
EXPECTED -
See above notes.
ACTUAL -
See above notes.

REPRODUCIBILITY :
This bug can be reproduced always.

---------- BEGIN SOURCE ----------
The test program is:

import java.awt.*;
import java.awt.event.*;
import java.io.*;
import javax.swing.*;
import javax.xml.transform.Transformer;
import javax.xml.transform.TransformerFactory;
import javax.xml.transform.stream.StreamResult;
import javax.xml.transform.stream.StreamSource;

public class TestXSLTransform extends JFrame {

  public TestXSLTransform() {
    init();
    validate();
    setVisible(true);
  }

  public static void main(String[] args) {
    SwingUtilities.invokeLater(new Runnable() {
      public void run() {
        new TestXSLTransform();
      }
    });
  }

  private void init() {
    addWindowListener(new WindowAdapter() {
      public void windowClosing(WindowEvent evt) {
        System.exit(0);
      }
    });

    getContentPane().setLayout(new BorderLayout());

    JButton btn = new JButton("Start");
    btn.addActionListener(new ActionListener() {
      public void actionPerformed(ActionEvent e) {
        Thread t = new Thread(new Runnable() {
          public void run() {
            transform();
          }
        });

        t.start();
      }
    });


    getContentPane().add(btn, BorderLayout.CENTER);
    setSize(new Dimension(200, 100));
  }

  private void transform() {
    System.out.println("start transform");

    try {
      InputStream in = new BufferedInputStream(new FileInputStream("input.xml"));
      OutputStream out = new BufferedOutputStream(new FileOutputStream("output.html"));
      InputStream rulesIn = new BufferedInputStream(new FileInputStream("rules.xsl"));

      TransformerFactory tFactory = TransformerFactory.newInstance();
      Transformer transformer = tFactory.newTransformer(new StreamSource(rulesIn));

      transformer.transform(new StreamSource(in), new StreamResult(out));

      in.close();
      out.close();
      rulesIn.close();

    } catch (Exception e) {
      System.out.println("exception = " + e);
    }

    System.out.println("end transform");
  }

}


-----------------------------------

The input.xml file is:

<?xml version='1.0' encoding='utf-8'?>
<report>
<body>
<table>

<table_row>
<table_cell><cell_value><![CDATA[In progress]]></cell_value><cell_type>string</cell_type></table_cell>
<table_cell><cell_value><![CDATA[Unit Pricing]]></cell_value><cell_type>string</cell_type></table_cell>
<table_cell><cell_value><![CDATA[TEST_PAUL]]></cell_value><cell_type>string</cell_type></table_cell>
<table_cell><cell_value><![CDATA[30-Dec-2005]]></cell_value><cell_type>string</cell_type></table_cell>
<table_cell><cell_value><![CDATA[1]]></cell_value><cell_type>string</cell_type></table_cell>
<table_cell><cell_value><![CDATA[System]]></cell_value><cell_type>string</cell_type></table_cell>
<table_cell><cell_value><![CDATA[366162]]></cell_value><cell_type>string</cell_type></table_cell>
<table_cell><cell_value><![CDATA[05-May-2006 10:52:39]]></cell_value><cell_type>string</cell_type></table_cell>
<table_cell><cell_value><![CDATA[0]]></cell_value><cell_type>string</cell_type></table_cell>
<table_cell><cell_value><![CDATA[19305345]]></cell_value><cell_type>string</cell_type></table_cell>
<table_cell><cell_value><![CDATA[0]]></cell_value><cell_type>string</cell_type></table_cell>
<table_cell><cell_value><![CDATA[0]]></cell_value><cell_type>string</cell_type></table_cell>
<table_cell><cell_value><![CDATA[server]]></cell_value><cell_type>string</cell_type></table_cell>
</table_row>

</table>
</body>
</report>

-----------------------------------

The rules.xsl file is:

<?xml version="1.0"?>
<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" version="1.0">
  <xsl:output method="html" indent="yes"/>

  <xsl:template match="report">
    <html>
      <xsl:apply-templates/>
    </html>
  </xsl:template>

  <xsl:template match="body">
    <body>
      <xsl:apply-templates/>
    </body>
  </xsl:template>

  <xsl:template match="table">
    <table border="1">
      <xsl:apply-templates/>
    </table>
  </xsl:template>

  <xsl:template match="table_cell">
    <xsl:choose>
      <xsl:when test="cell_value != ''">
        <td>
          <xsl:value-of select="cell_value"/>
        </td>
      </xsl:when>
      <xsl:otherwise>
        <td>
          -
        </td>
      </xsl:otherwise>
    </xsl:choose>
  </xsl:template>

  <xsl:template match="table_row">
    <tr>
      <xsl:apply-templates/>
    </tr>
  </xsl:template>

</xsl:stylesheet>

---------- END SOURCE ----------

CUSTOMER SUBMITTED WORKAROUND :
I do not have a work-around.

                                    

Comments
EVALUATION

To respond to ashokM's comments:

This patch had been integrated into JDK5 since update 12. It's possible that you could have hit a different issue. Could you contact support with a testcase that shows the issue you're having?

Thanks
                                     
2008-06-18
EVALUATION

I'm able to reproduce this using JDK 5_06, jconsole, a max heap size of 5 M. However, this does not appear to be reproducible using JDK 6 (Mustang) beta (I started over 200 threads using the GUI button). If Xerces is the problem, we should look for any patches applied to Mustang that may be backported to JDK 5.
                                     
2006-06-08
SUGGESTED FIX

The method XMLReaderManager.releaseXMLReader() has been fixed in Mustang, but as the user indicates, it is buggy in Tiger. Here is the correct implementation:

    public synchronized void releaseXMLReader(XMLReader reader) {
        // If the reader that's being released is the cached reader
        // for this thread, remove it from the m_isUse list.
        if (m_readers.get() == reader && reader != null) {
            m_inUse.remove(reader);
        }
    }
                                     
2006-06-08



Hardware and Software, Engineered to Work Together