JDK-4968789 : java.io.File or classloader can't open file by URL with %XX encoded characters
  • Type: Bug
  • Component: core-libs
  • Sub-Component: java.io
  • Affected Version: 1.4.2
  • Priority: P3
  • Status: Closed
  • Resolution: Won't Fix
  • OS: windows_xp
  • CPU: x86
  • Submitted: 2003-12-15
  • Updated: 2004-03-26
  • Resolved: 2004-03-26
Description

Name: gm110360			Date: 12/15/2003


FULL PRODUCT VERSION :
java version "1.4.1_01"
Java(TM) 2 Runtime Environment, Standard Edition (build 1.4.1_01-b01)
Java HotSpot(TM) Client VM (build 1.4.1_01-b01, mixed mode)

FULL OPERATING SYSTEM VERSION :

Microsoft Windows XP [version 5.1.2600]

ADDITIONAL OPERATING SYSTEMS :

Windows 95, 98, SE, ME, 2000, and probably affects also
internationalized versions of Unix, Solaris, or Linux.

EXTRA RELEVANT SYSTEM CONFIGURATION :
This is both a general issue, and a OS-specific issue, as
this affects the way system-specific filepaths are
converted to URL and back to filepaths. This transformation
should be reversible, at least for the critical case of
class loaders. But its implementation is OS-specific and
imply porting testers for each platform, and the
internationalization team.

This may affect many (most?) versions of Java (JRE/JDK,
J2SE/J2EE.

A DESCRIPTION OF THE PROBLEM :
Building and testing junit on Windows may fail if junit
(and its sources) is installed in a path that contains non
ASCII characters (for example a space, but also any
international character), such as in "C:\Program
Files\Java\junit3.8.1" where the downloaded zip was
downloaded, or "C:\Documents and
Files\<username>\Javatools" (a subdirectory of a user's
home directory on Windows)

     [java] .........LoadFileData
("C:\PROGRA~\Java\JUNIT3~1.1
\junit3.8.1","junit/tests/runner/BaseTestRunnerTest$NonStati
c.class")
     [java] ..........LoadFileData("C:\PROGRA~1
\Java\JUNIT3~1.1
\junit3.8.1","junit/tests/runner/ClassLoaderTest.class")
     [java] .testJarClassLoading():
     [java] test.jar expected at /C:/Program%
20Files/Java/junit3.8.1/junit3.8.1/junit/tests/runner/test.j
ar
     [java] LoadJarData("/C:/Program%
20Files/Java/junit3.8.1/junit3.8.1/junit/tests/runner/test.j
ar","junit/tests/runner/LoadedFromJar.class")
     [java] E.................
     [java] Time: 8,953
     [java] There was 1 error:
     [java] 1) testJarClassLoading
(junit.tests.runner.TestCaseClassLoaderTest)
java.lang.ClassNotFoundException:
junit.tests.runner.LoadedFromJar
     [java]     at
junit.runner.TestCaseClassLoader.lookupClassData
(TestCaseClassLoader.java:126)
     [java]     at
junit.runner.TestCaseClassLoader.loadClass
(TestCaseClassLoader.java:101)
     [java]     at
junit.tests.runner.TestCaseClassLoaderTest.testJarClassLoadi
ng(TestCaseClassLoaderTest.java:35)
     [java]     at
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
     [java]     at
sun.reflect.NativeMethodAccessorImpl.invoke
(NativeMethodAccessorImpl.java:39)
     [java]     at
sun.reflect.DelegatingMethodAccessorImpl.invoke
(DelegatingMethodAccessorImpl.java:25)

     [java] FAILURES!!!
     [java] Tests run: 119,  Failures: 0,  Errors: 1

     [java] Java Result: 1

BUILD SUCCESSFUL
  Total time: 1 minute 24 seconds



The bug is not directly within junit. It seems to be an
issue of the classloader classes in the JRE/JDK, which
converts the Windows path given to load a class into a URL,
or more intimately within the java.io.File classes, which
fails to reconvert an URL into a valid Windows path ("/"
characters are correctly converted, but not hexadecimal
byte references such as "%20" which are not translated back
to their original form needed to successfully open a file):


STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
1. run a Java class from a path containing spaces or
international (non ASCII) characters.
2. in that class, use getClass().getResource("filename") to
locate a valid file in the same path, for example a JAR
file. An valid absolute URL containing "/" and "%XX"
references is returned, for example "file:/C:/Program%
20Files/Java/junit3.8.1"
3. Use url.getFile() to extract the filepath from that URL
as a String object, for example '/C:/Program%
20Files/Java/junit3.8.1/junit/tests/runner/test.jar"
4. Try to use this path to instanciate a new class or
resource loader, this fails.

Another example, simply by using the jar tool: if you
specify the same URL "/C:/Program%20Files/..." for the JAR
file to open with the jar tool, this fails, as java.io.File
will not convert "%20" correctly before calling the Win32
API with the incorrect name: "C:\Program%20Files\..."

If you only replace "%20" by a space, this jar tool use
succeeds.


EXPECTED VERSUS ACTUAL BEHAVIOR :
java.io.File porting on Windows should convert a file URL
into a valid Windows filepath, by not only converting "/"
into "\", or remoing the initial "/" before the drive
letter. It should also part the URL correctly, and convert %
XX sequences into their actual byte values.

The transformation of a filepath into a URL should always
be reversible in java.io.File...

ERROR MESSAGES/STACK TRACES THAT OCCUR :
C:\PROGRA~1\Java\JUNIT3~1.1>jar tvf /C:/Program%20Files/Java/junit3.8.1/junit3.8
.1/junit/tests/runner/test.jar
java.io.FileNotFoundException: C:\Program%20Files\Java\junit3.8.1\junit3.8.1\jun
it\tests\runner\test.jar (Le chemin d'acc������s sp������cifi������ est introuvable)
        at java.io.FileInputStream.open(Native Method)
        at java.io.FileInputStream.<init>(FileInputStream.java:103)
        at java.io.FileInputStream.<init>(FileInputStream.java:66)
        at sun.tools.jar.Main.run(Main.java:185)
        at sun.tools.jar.Main.main(Main.java:904)

C:\PROGRA~1\Java\JUNIT3~1.1>jar tvf "/C:/Program Files/Java/junit3.8.1/junit3.8.
1/junit/tests/runner/test.jar"
     0 Mon Jan 13 03:49:36 CET 2003 META-INF/
    58 Mon Jan 13 03:49:36 CET 2003 META-INF/MANIFEST.MF
     0 Mon Jan 13 03:18:08 CET 2003 junit/
     0 Mon Jan 13 03:49:18 CET 2003 junit/tests/
     0 Mon Jan 13 03:49:18 CET 2003 junit/tests/runner/
  1507 Mon Jan 13 03:49:18 CET 2003 junit/tests/runner/LoadedFromJar.class

C:\PROGRA~1\Java\JUNIT3~1.1>jar tvf "/C:/Progra~1/Java/junit3.8.1/junit3.8.1/jun
it/tests/runner/test.jar"
     0 Mon Jan 13 03:49:36 CET 2003 META-INF/
    58 Mon Jan 13 03:49:36 CET 2003 META-INF/MANIFEST.MF
     0 Mon Jan 13 03:18:08 CET 2003 junit/
     0 Mon Jan 13 03:49:18 CET 2003 junit/tests/
     0 Mon Jan 13 03:49:18 CET 2003 junit/tests/runner/
  1507 Mon Jan 13 03:49:18 CET 2003 junit/tests/runner/LoadedFromJar.class

C:\PROGRA~1\Java\JUNIT3~1.1>


REPRODUCIBILITY :
This bug can be reproduced always.

---------- BEGIN SOURCE ----------
Download the junit ZIP package into a subfolder of "C:\Program Files", and
unzip it there. Then set your CLASSPATH to include the extracted "junit.jar" as
indicated in the online documentation. Then start any of the three suggested
self tests: this fails in the class loader test.
Then extract the sources with "jar xf src.jar", and build it with ant.
The compilation and reconstruction of a new junit ZIP distribution succeeds,
but the final self test fails.
You may locate the problem in TestCaseClassLoaderTest.java,
in the member function testJarClassLoading()
The new classloader object is effectively created to test loading from
the "test.jar", however in this case the path is a URL, that loadClass() cannot
use (this exception will be reported internally during loadClass() as it uses a
File.open() method that fails to convert the absolute URL into a valid Windows
file path.
---------- END SOURCE ----------

CUSTOMER WORKAROUND :
None for Windows NT/2000/XP users with non Administrative
accesses rights and that can only create files in their own
document directory.

For other users, applications using custom classloaders
should not be installed within directories whose path may
contain non ASCII characters or characters that are
reserved by the URL standard...

The impact on standard users is blocking (no progress is
possible).

On users with enough access rights so that they can create
or use directories whose absolute path can conform to the
basic ASCII URL encoding, it is alarming.

Standard installation of many JAR packages may fail because
of this, if they need to implement custom class loaders.

For now, care must be taken when extracting a filepath from
an URL, and before using it for File IO, the application
must perform a OS-specific conversion instead of using the
perfectly valid and canonical URL associated to a unique
file path.
(Incident Review ID: 179905) 
======================================================================

Comments
EVALUATION Yes, URL.getFile() has the problems noted by submitter but this will not change due to compatibility, and it is the reason that this issue is handled correctly by the URI class. ###@###.### 2004-03-25
25-03-2004