FULL PRODUCT VERSION :
java version "1.8.0_60"
Java(TM) SE Runtime Environment (build 1.8.0_60-b27)
Java HotSpot(TM) 64-Bit Server VM (build 25.60-b23, mixed mode)
ADDITIONAL OS VERSION INFORMATION :
Microsoft Windows 7 Enterprise Service Pack 1 [version: 6.1.7601]
A DESCRIPTION OF THE PROBLEM :
When validating XML files against XSD schemas using the Java XML Validation API (package javax.xml.validation) I experienced that if an exception occurs at the beginning of the process (i.e. the prolog in the xml is bad) then the implementation fails to close the input stream to the XML file being processed.
This is true for the parsing of XSD schemas (SchemaFactory.newSchema(Source[]) method) and the validation of XML files (Validator.validate(Source) method) as well. In both cases the Source object that is passed as parameter is a StreamSource constructed from a File, so the input stream to the file is opened by the JAXP implementation. The implementation should also close the file in all cases, even on error.
See the source code below for a detailed description of the problem.
STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
Put the attached source files in a directory in the exact hierarchy as described below. Go into the directory where the pom.xml is, and build the project with the following command (requires maven to be installed):
# mvn clean install
The test cases and their javadoc comments in the StreamBugReproTest class show and explain the problem in detail. The 2 test cases will fail. This is intentional, to indicate the deviation from the expected behaviour.
I can attach/send the source files in a zip archive, but I could not attach them using the http://bugreport.java.com/ form.
EXPECTED VERSUS ACTUAL BEHAVIOR :
EXPECTED -
The expected result is that after the newSchema() and validate() methods return or are interrupted by an exception, there are no open streams to the files that were passed to them as parameters (provided that the input Source was created from a File and not an InputStream). This would make it possible to delete the files afterwards, in which case the attached tests would pass.
ACTUAL -
The actual result is that the input streams remain open in some cases if an exception is thrown from the newSchema() or the validate() methods, which prevents deletion of the files that were passed in as parameters. This causes the attached tests to fail.
ERROR MESSAGES/STACK TRACES THAT OCCUR :
java.nio.file.FileSystemException: C:\Users\RETI~1.KOR\AppData\Local\Temp\todelete_7167110778758351288\not_well_formed_7716458762950649646.xml: The process cannot access the file because it is being used by another process.
at sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:86)
at sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:97)
at sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:102)
at sun.nio.fs.WindowsFileSystemProvider.implDelete(WindowsFileSystemProvider.java:269)
at sun.nio.fs.AbstractFileSystemProvider.delete(AbstractFileSystemProvider.java:103)
at java.nio.file.Files.delete(Files.java:1126)
at xml.validation.experiment.StreamBugReproTest.testXmlFileCannotBeDeletedAfterValidation(StreamBugReproTest.java:47)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:283)
at org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:173)
at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:153)
at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:128)
at org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:203)
at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:155)
at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:103)
REPRODUCIBILITY :
This bug can be reproduced always.
---------- BEGIN SOURCE ----------
pom.xml:
----------------------------------------
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>xml.validation.experiment</groupId>
<artifactId>bug_repro_tests</artifactId>
<version>1.0-SNAPSHOT</version>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.3</version>
<configuration>
<source>1.7</source>
<target>1.7</target>
<encoding>UTF-8</encoding>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.18.1</version>
</plugin>
</plugins>
</build>
<dependencies>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.12</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.hamcrest</groupId>
<artifactId>hamcrest-library</artifactId>
<version>1.3</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.jmock</groupId>
<artifactId>jmock-junit4</artifactId>
<version>2.6.0</version>
<scope>test</scope>
<exclusions>
<exclusion>
<groupId>org.hamcrest</groupId>
<artifactId>hamcrest-library</artifactId>
</exclusion>
<exclusion>
<groupId>junit</groupId>
<artifactId>junit-dep</artifactId>
</exclusion>
</exclusions>
</dependency>
</dependencies>
</project>
src/main/java/xml/validation/experiment/XmlValidator.java:
----------------------------------------
package xml.validation.experiment;
import org.w3c.dom.ls.LSResourceResolver;
import org.xml.sax.SAXException;
import javax.xml.XMLConstants;
import javax.xml.transform.Source;
import javax.xml.transform.stream.StreamSource;
import javax.xml.validation.Schema;
import javax.xml.validation.SchemaFactory;
import javax.xml.validation.Validator;
import java.io.File;
import java.io.IOException;
public class XmlValidator {
Validator validator;
public XmlValidator(String accessExternalSchema, LSResourceResolver resourceResolver, Source... xsdSources) throws SAXException {
final SchemaFactory factory = SchemaFactory.newInstance(XMLConstants.W3C_XML_SCHEMA_NS_URI);
factory.setFeature(XMLConstants.FEATURE_SECURE_PROCESSING, true); // try to be secure
factory.setProperty(XMLConstants.ACCESS_EXTERNAL_DTD, ""); // prevent XXE
if (accessExternalSchema != null) { // impose limitations on external schemas too
factory.setProperty(XMLConstants.ACCESS_EXTERNAL_SCHEMA, accessExternalSchema);
}
factory.setResourceResolver(resourceResolver);
Schema schema;
if (xsdSources != null && xsdSources.length > 0) {
schema = factory.newSchema(xsdSources);
} else {
schema = factory.newSchema();
}
validator = schema.newValidator();
validator.setFeature(XMLConstants.FEATURE_SECURE_PROCESSING, true); // try to be secure
validator.setProperty(XMLConstants.ACCESS_EXTERNAL_DTD, ""); // prevent XXE
if (accessExternalSchema != null) { // impose limitations on external schemas too
validator.setProperty(XMLConstants.ACCESS_EXTERNAL_SCHEMA, accessExternalSchema);
}
validator.setResourceResolver(resourceResolver);
}
public void validate(File xmlFile) throws IOException, SAXException {
validator.validate(new StreamSource(xmlFile));
}
}
src/test/java/xml/validation/experiment/StreamBugReproTest.java:
----------------------------------------
package xml.validation.experiment;
import org.junit.Test;
import org.xml.sax.SAXParseException;
import javax.xml.transform.stream.StreamSource;
import java.io.File;
import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Path;
import static org.junit.Assert.assertFalse;
public class StreamBugReproTest {
private static final File BASE_DIR = new File("target/test-classes/xml/validation/experiment");
private static final File SIMPLE_XSD_FILE = new File(BASE_DIR, "simple.xsd");
private static final File SIMPLE_XML_FILE = new File(BASE_DIR, "simple_with_local_schemalocation.xml");
/**
* On Windows, this test demonstrates that the implementation does not close the input stream to the XML file
* being validated if there is an exception at the beginning of the validation process (i.e. the prolog in
* the xml is bad).
* <p/>
* The Source object that is passed as parameter to the Validator.validate(Source) method is a
* StreamSource constructed from a File, so the input stream to the file is opened by the JAXP implementation.
* The implementation should also close the file in all cases, even on error. So this seems to be a bug.
* <p/>
* This test is designed to fail in order to indicate the bug. The stack trace of the IOException printed to
* stderr proves that the temp file could not be deleted because the stream is still open.
*/
@Test
public void testXmlFileCannotBeDeletedAfterValidation() throws Exception {
System.out.println("Running: testXmlFileCannotBeDeletedAfterValidation");
Path tempDir = null;
Path tempFile = null;
try {
try {
tempDir = Files.createTempDirectory("todelete_");
tempFile = Files.createTempFile(tempDir, "not_well_formed_", ".xml");
tempDir.toFile().deleteOnExit();
tempFile.toFile().deleteOnExit();
Files.write(tempFile, "In fact, this is not an xml file.".getBytes());
new XmlValidator(null, null, new StreamSource(SIMPLE_XSD_FILE)).validate(tempFile.toFile());
} finally {
if (tempFile != null) {
Files.delete(tempFile);
}
if (tempDir != null) {
Files.delete(tempDir);
}
}
} catch (SAXParseException | IOException ex) {
ex.printStackTrace();
assertFalse("Could not delete the xml file: " + tempFile.toString(), Files.exists(tempFile));
assertFalse("Could not delete the directory: " + tempDir.toString(), Files.exists(tempDir));
}
}
/**
* On Windows, this test demonstrates that the implementation does not close the input stream to the schema file
* being parsed if there is an exception at the beginning of the parsing process (i.e. the prolog in
* the xml is bad).
* <p/>
* The Source object that is passed as parameter to the SchemaFactory.newSchema(Source[]) method is a
* StreamSource constructed from a File, so the input stream to the file is opened by the JAXP implementation.
* The implementation should also close the file in all cases, even on error. So this seems to be a bug.
* <p/>
* This test is designed to fail in order to indicate the bug. The stack trace of the IOException printed to
* stderr proves that the temp file could not be deleted because the stream is still open.
*/
@Test
public void testXsdSchemaCannotBeDeletedAfterValidation() throws Exception {
System.out.println("Running: testXsdSchemaCannotBeDeletedAfterValidation");
Path tempDir = null;
Path tempFile = null;
try {
try {
tempDir = Files.createTempDirectory("todelete_");
tempFile = Files.createTempFile(tempDir, "not_a_schema_", ".xsd");
tempDir.toFile().deleteOnExit();
tempFile.toFile().deleteOnExit();
Files.write(tempFile, "In fact, this is not an xml file.".getBytes());
new XmlValidator(null, null, new StreamSource(tempFile.toFile())).validate(SIMPLE_XML_FILE);
} finally {
if (tempFile != null) {
Files.delete(tempFile);
}
if (tempDir != null) {
Files.delete(tempDir);
}
}
} catch (SAXParseException | IOException ex) {
ex.printStackTrace();
assertFalse("Could not delete the xsd file: " + tempFile.toString(), Files.exists(tempFile));
assertFalse("Could not delete the directory: " + tempDir.toString(), Files.exists(tempDir));
}
}
}
src/test/resources/xml/validation/experiment/simple_with_local_schemalocation.xml:
----------------------------------------
<?xml version="1.0" encoding="UTF-8"?>
<Root xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="simple.xsd">
<Elem>Some text content.</Elem>
</Root>
src/test/resources/xml/validation/experiment/simple.xsd:
----------------------------------------
<?xml version="1.0" encoding="UTF-8"?>
<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema">
<xs:element name="Root">
<xs:complexType>
<xs:sequence>
<xs:element name="Elem" type="xs:string"/>
</xs:sequence>
</xs:complexType>
</xs:element>
</xs:schema>
---------- END SOURCE ----------
CUSTOMER SUBMITTED WORKAROUND :
My workaround is to open and close the files myself, and construct the StreamSource objects from InputStream rather than File. Like this:
public void validate(final File... xmlFiles) throws IllegalArgumentException, SAXException, IOException, NullPointerException {
// In some cases 'new StreamSource(File file)' causes the InputStream to the file to remain open if the
// parsing is interrupted with a fatal error and SAXException is thrown.
// Therefore we must open and close the streams ourselves here.
StreamSource[] xmlSources = new StreamSource[xmlFiles.length];
try {
for (int i = 0; i < xmlFiles.length; i++) {
xmlSources[i] = new StreamSource(new FileInputStream(xmlFiles[i]));
xmlSources[i].setSystemId(xmlFiles[i]);
}
validate(xmlSources);
} finally {
for (StreamSource xmlSource : xmlSources) {
if (xmlSource != null) {
IOUtils.closeQuietly(xmlSource.getInputStream());
// Here we rely on the fact that getInputStream() returns the same object that we created
// the StreamSource from.
}
}
}
}