JDK-4866151 : File returned by listFiles cannot be opened: FileNotFoundException
  • Type: Bug
  • Component: core-libs
  • Sub-Component: java.io
  • Affected Version: 1.4.1
  • Priority: P3
  • Status: Closed
  • Resolution: Not an Issue
  • OS: linux
  • CPU: x86
  • Submitted: 2003-05-19
  • Updated: 2003-05-24
  • Resolved: 2003-05-24
Description

Name: nt126004			Date: 05/19/2003


FULL PRODUCT VERSION :
java version "1.4.1_01"
Java(TM) 2 Runtime Environment, Standard Edition (build 1.4.1_01-b01)
Java HotSpot(TM) Client VM (build 1.4.1_01-b01, mixed mode)


FULL OS VERSION :
Linux homemp3 2.4.20 #5 Thu Mar 20 09:01:11 CET 2003 i686 unknown
GNU/Debian

Also reproduced on Solaris :
SunOS bugkiller 5.8 Generic_108528-17 sun4u sparc SUNW,Ultra-5_10


EXTRA RELEVANT SYSTEM CONFIGURATION :
ext3 filesystem

A DESCRIPTION OF THE PROBLEM :
I have a file containing somewhat weird characters in its name, resulting from grabbing a CD on Windows and storing the files to an SMB-mounted directory on a Linux server. Trying to open the resulting file from Java on the Linux server fails because the result of listFiles on the respective directory causes a FileNotFoundException. The bug seems to have been reported before against 1.3.1 (# 4498715). It seems to be a major interoperability problem.

STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
I used the following C code to create the file:

#include <stdio.h>

int main(char* args[])
{
    FILE* f = fopen("newdir/abc\366def", "w");
    fprintf(f, "nonsense");
    fclose(f);
}

Create a directory "newdir" in the current directory, compile and run the
C program. Then, use the following Java code to reproduce the bug:

import java.io.*;

public class T
{
    public static void main(String[] args)
    {
        try
        {
            File dir = new File("./newdir");
            File[] fa = dir.listFiles();
            for (int i=0; i<fa.length; i++)
            {
                System.out.println(fa[i]);
                for (int j=0; j<fa[i].getPath().length(); j++)
                    System.out.print(", "+(int) fa[i].getPath().charAt(j));
                System.out.println();
            }
            java.io.File f = fa[0];
            java.io.FileInputStream fis = new java.io.FileInputStream(f);
            fis.close();
        }
        catch (Exception e)
        {
            e.printStackTrace();
        }
    }
}



EXPECTED VERSUS ACTUAL BEHAVIOR :
EXPECTED -
uhl@homemp3:/tmp$ ./T
uhl@homemp3:/tmp$ ls newdir/
abc?def
uhl@homemp3:/tmp$ ls newdir/ | od -bc
0000000 141 142 143 366 144 145 146 012
          a   b   c 366   d   e   f  \n
0000010
uhl@homemp3:/tmp$ javac T.java
uhl@homemp3:/tmp$ java T
./newdir/abc?def
, 46, 47, 110, 101, 119, 100, 105, 114, 47, 97, 98, 99, 65533, 100, 101, 102

ACTUAL -
uhl@homemp3:/tmp$ ./T
uhl@homemp3:/tmp$ ls newdir/
abc?def
uhl@homemp3:/tmp$ ls newdir/ | od -bc
0000000 141 142 143 366 144 145 146 012
          a   b   c 366   d   e   f  \n
0000010
uhl@homemp3:/tmp$ javac T.java
uhl@homemp3:/tmp$ java T
./newdir/abc?def
, 46, 47, 110, 101, 119, 100, 105, 114, 47, 97, 98, 99, 65533, 100, 101, 102
java.io.FileNotFoundException: ./newdir/abc?def (No such file or directory)
        at java.io.FileInputStream.open(Native Method)
        at java.io.FileInputStream.<init>(FileInputStream.java:103)
        at T.main(T.java:19)


ERROR MESSAGES/STACK TRACES THAT OCCUR :
java.io.FileNotFoundException: ./newdir/abc?def (No such file or directory)
        at java.io.FileInputStream.open(Native Method)
        at java.io.FileInputStream.<init>(FileInputStream.java:103)
        at T.main(T.java:19)


REPRODUCIBILITY :
This bug can be reproduced always.

---------- BEGIN SOURCE ----------
#include <stdio.h>

int main(char* args[])
{
    FILE* f = fopen("newdir/abc\366def", "w");
    fprintf(f, "nonsense");
    fclose(f);
}

Create a directory "newdir" in the current directory, compile and run the
C program. Then, use the following Java code to reproduce the bug:

import java.io.*;

public class T
{
    public static void main(String[] args)
    {
        try
        {
            File dir = new File("./newdir");
            File[] fa = dir.listFiles();
            for (int i=0; i<fa.length; i++)
            {
                System.out.println(fa[i]);
                for (int j=0; j<fa[i].getPath().length(); j++)
                    System.out.print(", "+(int) fa[i].getPath().charAt(j));
                System.out.println();
            }
            java.io.File f = fa[0];
            java.io.FileInputStream fis = new java.io.FileInputStream(f);
            fis.close();
        }
        catch (Exception e)
        {
            e.printStackTrace();
        }
    }
}

---------- END SOURCE ----------
(Review ID: 186048) 
======================================================================

Comments
EVALUATION The root problem here is that \366 is not a valid character in the default charset (whatever it is, probably ASCII), so when the File.listFiles method reads the filename it translates \366 to \uFFFD, the Unicode replacement character. The given code passes this back to the FileInputStream constructor, which eventually translates the pathname back into ASCII or whatever, and since \uFFFD is not mappable in that charset it's mapped to '?', and of course then the file can't be opened because instead of opening ./newdir/abc\366def the runtime is actually trying to open ./newdir/abc?def, which doesn't exist. The treatment of filenames as strings encoded in the default charset is deeply baked into the treatment of pathnames and cannot be changed. This particular problem can be worked around by running the JRE in a locale whose charset preserves the encoded value \366. On both Solaris and Linux the en_US locale, e.g., uses ISO-8859-1 as the default, and \366 is a legal character in that charset, so running the program like this: $ LC_ALL=en_US java T should produce the expected result. -- ###@###.### 2003/5/23
05-11-0189