JDK-4307912 : Filesystem silently translates illegal characters, causing compiler failure
  • Type: Bug
  • Component: core-libs
  • Sub-Component: java.io
  • Affected Version: 1.3.0
  • Priority: P4
  • Status: Closed
  • Resolution: Won't Fix
  • OS: solaris_7
  • CPU: sparc
  • Submitted: 2000-01-28
  • Updated: 2000-02-03
  • Resolved: 2000-02-03
Related Reports
Relates :  
JDK-1266364 - Unicode escape in package name isn't processed properly
Relates :  
JDK-4277355 - Compilation problem with the unicode on win32
Description

Name: laC46010			Date: 01/28/2000



The class name and package name containing Unicode Escape are incorrectly 
reflected on solaris file-system.

The test pkgs00704 is valid compile positive test (see source and log below).
This test fails on all JDK versions by different reasons.
It is compiled succesfully on all JDK versions solaris but the two classes with 
the same name belonging to different packages (pkgs00704\u00e8 and 
pkgs00704\u00e9)are placed to the same directory on JDK 1.2 and later. 
As the result one of correspoding class-files rewrite another. 
On JDK 1.02 and 1.1* these classes are placed to the different directory 
but the execution fails.
Note that if this test is compiled and executed on win32 then it passes.

The test name00203 is valid compile positive test (see source and log below).
This test fails on all JDK versions by different reasons.
It is compiled succesfully on all JDK versions solaris but the two classes with 
the different names (name00203\u00e8 and name00203\u00e9) are placed to the same 
class-file on JDK 1.2 and later. As the result one of correspoding class-files 
rewrite another. The name of class-file is name002003?.class where character 
code of ? is 0x3f.
On JDK 1.02 and 1.1* these classes are placed to the different class-files 
but the execution fails.
Note that if this test is compiled and executed on win32 then it passes.

Source pkgs00704
-------------------------pkgs00704a/pkgs00704a.java--------------------------
package javasoft.sqe.tests.lang.pkgs007.pkgs00704\u00e8;
public class pkgs00704a {
  public static int get() {
     return 8;
  }
}
-------------------------------pkgs00704a.java-------------------------------
package javasoft.sqe.tests.lang.pkgs007.pkgs00704\u00e9;
public class pkgs00704a {
  public static int get() {
       return 4;
  }
}
-------------------------------pkgs00704.java--------------------------------
public static int run(String argv[],PrintStream out) {
   if(javasoft.sqe.tests.lang.pkgs007.pkgs00704\u00e8.pkgs00704a.get()!=8) {
       out.println("pkgs00704a.get() != 8 ");
       return 2/*STATUS_FAILED*/;
   }
   if(javasoft.sqe.tests.lang.pkgs007.pkgs00704\u00e9.pkgs00704a.get()!=4) {
       out.println("pkgs00704a.get() != 4 ");
       return 2/*STATUS_FAILED*/;
   }
   return 0/*STATUS_PASSED*/;
}
-----------------------------------------------------------------------------
log on JDK 1.3 solaris
_________________________________________________________________________
javac -d . pkgs00704a.java pkgs00704a/pkgs00704a.java pkgs00704.java 

java -cp . javasoft.sqe.tests.lang.pkgs007.pkgs00704.pkgs00704

Exception in thread "main" java.lang.NoClassDefFoundError: 
javasoft/sqe/tests/lang/pkgs007/pkgs00704?/pkgs00704a (wrong name: 
javasoft/sqe/tests/lang/pkgs007/pkgs00704?/pkgs00704a)
        at java.lang.ClassLoader.defineClass0(Native Method)
        at java.lang.ClassLoader.defineClass(ClassLoader.java:453)
        at 
java.security.SecureClassLoader.defineClass(SecureClassLoader.java:101)
        at java.net.URLClassLoader.defineClass(URLClassLoader.java:251)
        at java.net.URLClassLoader.access$100(URLClassLoader.java:59)
        at java.net.URLClassLoader$1.run(URLClassLoader.java:198)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.net.URLClassLoader.findClass(URLClassLoader.java:191)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:291)
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:290)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
        at 
javasoft.sqe.tests.lang.pkgs007.pkgs00704.pkgs00704.run(pkgs00704.java:17)
        at 
javasoft.sqe.tests.lang.pkgs007.pkgs00704.pkgs00704.main(pkgs00704.java:10)
_________________________________________________________________________

log on JDK 1.1 solaris
_________________________________________________________________________
javac -d . pkgs00704a.java pkgs00704a/pkgs00704a.java pkgs00704.java 

export CLASSPATH=.:/export/ld25/java/dest/jdk1.1/solaris/lib/classes.zip

java javasoft.sqe.tests.lang.pkgs007.pkgs00704.pkgs00704

java.lang.NoClassDefFoundError: 
javasoft/sqe/tests/lang/pkgs007/pkgs00704��/pkgs00704a
        at 
        at 
javasoft.sqe.tests.lang.pkgs007.pkgs00704.pkgs00704.main(pkgs00704.java:10)
_________________________________________________________________________



Source name00203
-------------------------------name00203.java--------------------------------
class name00203\u00e9 {
    static int i = 4;
}
class name00203\u00e8 {
    static int i = 8;
}
public class name00203 { 
    public static int run(String argv[],PrintStream out) {
       if(name00203\u00e9.i != 4) {
           out.println("name00203\u00e9.i != 4");
           return 2/*STATUS_FAILED*/;
       }
       if(name00203\u00e8.i != 8) {
           out.println("name00203\u00e8.i != 8");
           return 2/*STATUS_FAILED*/;
       }
       return 0/*STATUS_PASSED*/;
    }
}
-----------------------------------------------------------------------------
log on JDK 1.3 solaris
_________________________________________________________________________
javac -d . name00203.java 

java -cp . javasoft.sqe.tests.lang.name002.name00203.name00203

Exception in thread "main" java.lang.NoClassDefFoundError: 
javasoft/sqe/tests/lang/name002/name00203/name00203? (wrong name: 
javasoft/sqe/tests/lang/name002/name00203/name00203?)
        at java.lang.ClassLoader.defineClass0(Native Method)
        at java.lang.ClassLoader.defineClass(ClassLoader.java:453)
        at 
java.security.SecureClassLoader.defineClass(SecureClassLoader.java:101)
        at java.net.URLClassLoader.defineClass(URLClassLoader.java:251)
        at java.net.URLClassLoader.access$100(URLClassLoader.java:59)
        at java.net.URLClassLoader$1.run(URLClassLoader.java:198)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.net.URLClassLoader.findClass(URLClassLoader.java:191)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:291)
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:290)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
        at 
javasoft.sqe.tests.lang.name002.name00203.name00203.run(name00203.java:22)
        at 
javasoft.sqe.tests.lang.name002.name00203.name00203.main(name00203.java:19)
_________________________________________________________________________

log on JDK 1.1 solaris
_________________________________________________________________________
javac -d . name00203.java 

export CLASSPATH=.:/export/ld25/java/dest/jdk1.1/solaris/lib/classes.zip

java javasoft.sqe.tests.lang.name002.name00203.name00203

java.lang.NoClassDefFoundError: 
javasoft/sqe/tests/lang/name002/name00203/name00203��
        at 
        at 
javasoft.sqe.tests.lang.name002.name00203.name00203.main(name00203.java:19)
_________________________________________________________________________

======================================================================
Comments
EVALUATION The fact that the example works correctly on win32 indicates that the class files themselves are generated correctly -- there are no platform dependencies that should affect this. The problem occurs when the host platform is unable to handle the pathnames that are created for the class files, derived from the identifiers used to name the classes and their containing packages within the source code. The problem here is that a '?' character is being silently substituted for characters that are illegal on the host platform. It is a limitation of the compiler that it can only compile programs that contain characters that are legal on the host platform, at least when these characters appear in class and package names. Unicode escapes cannot be used to get around this. The problem is that the host filesystem may impose restrictions on the characters that may appear in pathnames, and the Java IO subsystem does not provide any way to query which characters are acceptable, or to recover from attempts to use illegal characters. It is thus impossible to write truly platform-independent code using the current java.io classes. See the analysis of a similar problem, in Win32, by xueming.shen@eng in the evaluation of bug 4277355. The JLS remarks that an ecape convention may be used to encode the names of classes in filenames when the class and package names contain characters that are not permitted in filenames by the host platform. Unfortunately, given that there is no way to determine which characters are permitted, we are left with making pessimistic assumptions. If we really want to fix this bug, i.e., to allow these test cases to compile and execute correctly on all conformant Java implementations, regardless of host platform, it will be necessary to use the escape convention for all characters in the filename, with the exception of the seven-bit ASCII characters. Since the platform classes, whose names must be identical everywhere (so the VM can find them) obey this restriction, there should be no adverse compatibility implications, but non-English speakers would be rightfully annoyed that the names of the generated classfiles no longer bear any resemblance to the names of their classes as they appear in the source program. It has been observed that we can avoid dependencies on the host filesystem by compiling into JAR files, in effect, using our own "virtual file system" over which we have full control. To really work, however, we would have to store source files this way too, which would be highly inconvenient. In any case, this would be incompatible with current practice and user expectations. While we may add compilation into JAR files as an option for other reasons (see 4304048), it is not an adequate solution to the present difficulty. Another unnaceptable approach would be to give the compiler special knowledge of all host platforms. Note, however, that each locale would count as a separate platform. This would be an unmaintainable mess. The bottom line is that we either must accept the claim by xueming.shen@eng that examples like these should not be expected to work, or we need support in the Java platform for platform-independent file naming. william.maddox@Eng 2000-02-02 I agree with xeuming.shen@eng that examples like this should not be expected to work. The only remotely plausible solution, using the escape convention suggested in the JLS, cannot be implemented very well, as noted above, due to the pessimistic assumptions that must be made. The new I/O work will hopefully address the lack of a way to determine what filenames are legal and at least let the compiler generate an error when it can't store a class file in the filesystem without mangling the name. I think that would be preferable to any sort of escaping mechanism. -- mr@eng 2000/2/3
02-08-0169