Bug ID: JDK-5098433 REG: DnD of File-List between JVM is broken for non ASCII file names

Other	JDK 7
5.0-poolResolved	7 b78Fixed

EVALUATION I opened separated CR 6576211 for the Linux specific part of the problem. I tested the suggested fix with WToolkit. All worked fine except NullPointerException. The exception was caused by some security fix and it is not connected to the suggested fix.
05-07-2007
WORK AROUND Both drag source and drop target shoulbe run with appropriate locale. For example, on Gentoo Linux, for files with Russian names created in unicode locale, you could use >export LC_ALL=ru_RU.utf8 >java <java program>
06-03-2007
EVALUATION The problem should be solved when 4899439 will be fixed.
06-03-2007
EVALUATION I found that the problem is in File.getCanonicalPath() method. To translate the list of files packed in Transferable object into byte array, we DataTransferer extracts canonical file names as Strings. The file names stores in File object in correct way. But to get canonical names of the files java.io.File uses native function canonicalize(). This function assumes that the file name is not unicode. So it returns truncated bytes instead of canonical filenames. the next test shows the problem without D&D. === BEGIN OF SOURCE import java.io.File; import java.io.IOException; import java.awt.*; public class UnicodeCanonicalPath { public static void main (String [] args) { String russian_word = "\u043a\u043e\u043b"; File file = new File(russian_word); Frame frame = new Frame(); frame.setLayout(new FlowLayout()); TextField field1 = new TextField(russian_word); TextField field2 = null; try { // getCanonicalPath() truncates bytes below field2 = new TextField(file.getCanonicalPath()); } catch (IOException e) { e.printStackTrace(); } frame.add(field1); frame.add(field2); frame.pack(); frame.setVisible(true); } } ===== END OF SOURCE You can easy find that instead of {0x043a}{0x043e}{0x043b} the getCanonicalPath() function returns {0x3f}{0x3f}{0x3f}.
05-03-2007
EVALUATION The suggested fix is failed on the Linux platform. Currently we store transferred list of file names as a window property using XChangeProperty() method. We retrieve the data stored on the selection owner with XTextPropertyToStringList() function with ��STRING�� as an encoding atom. I should investigate the possibility to use not only ��STRING�� but ��UTF8_STRING�� encoding atom like it was done in XDdecoratedWindow and XBaseWindow classes. As I know UTF8_STRING is supported in X ORG and in XFree86 after version 4.2. Also a possibility to use Xutf8TextListToTextProperty method should be checked.
12-02-2007
EVALUATION The next fix is solves the problem on the WToolkit. But here could be some problems with Xtoolkit (See 5079469 DnD of File-List across JVM adds two empty items to the list). I am currently verify the behaviour on the Linux platform. ------- DataTransferer.java ------- * /tmp/sccs.r2aaNb Wed Feb 7 15:09:28 2007 --- DataTransferer.java Wed Feb 7 14:11:51 2007 *********** * 1299,1310 ** throw new IOException(pae.getMessage()); } ! for (int i = 0; i < files.length; i++) { ! byte[] bytes = files[i].getBytes(); ! bos.write(bytes, 0, bytes.length); ! bos.write(0); ! } ! bos.write(0); // Source data is an InputStream. For arbitrary flavors, just grab the // bytes and dump them into a byte array. For text flavors, decode back --- 1299,1312 ---- throw new IOException(pae.getMessage()); } ! for (int i = 0; i < files.length; i++) { ! byte[] bytes = files[i].getBytes(getDefaultUnicodeEncoding()); ! if (i != 0) { ! bos.write(0); ! bos.write(0); ! } ! bos.write(bytes, 0, bytes.length); ! } // Source data is an InputStream. For arbitrary flavors, just grab the // bytes and dump them into a byte array. For text flavors, decode back ------- awt_DnDDS.cpp ------- * /tmp/sccs.X2aGPb Wed Feb 7 15:15:40 2007 --- awt_DnDDS.cpp Wed Feb 7 15:15:37 2007 ************* * 636,642 ** dropfiles->pt.x = m_dropPoint.x; dropfiles->pt.y = m_dropPoint.y; dropfiles->fNC = m_fNC; ! dropfiles->fWide = FALSE; // good guess! dataout += sizeof(DROPFILES); } --- 636,642 ---- dropfiles->pt.x = m_dropPoint.x; dropfiles->pt.y = m_dropPoint.y; dropfiles->fNC = m_fNC; ! dropfiles->fWide = TRUE; dataout += sizeof(DROPFILES); } *********** * 803,809 **** dropfiles->pt.x = m_dropPoint.x; dropfiles->pt.y = m_dropPoint.y; dropfiles->fNC = m_fNC; ! dropfiles->fWide = FALSE; // good guess! dataout += sizeof(DROPFILES); } --- 803,809 ---- dropfiles->pt.x = m_dropPoint.x; dropfiles->pt.y = m_dropPoint.y; dropfiles->fNC = m_fNC; ! dropfiles->fWide = TRUE; dataout += sizeof(DROPFILES); }
07-02-2007
SUGGESTED FIX --- old/src/share/classes/sun/awt/datatransfer/DataTransferer.java 2008-06-30 15:13:19.203125000 +0400 +++ new/src/share/classes/sun/awt/datatransfer/DataTransferer.java 2008-06-30 15:13:18.156250000 +0400 @@ -1314,11 +1314,22 @@ } for (int i = 0; i < files.length; i++) { - byte[] bytes = files[i].getBytes(); - if (i != 0) bos.write(0); - bos.write(bytes, 0, bytes.length); + byte[] bytes = files[i].getBytes(getDefaultUnicodeEncoding()); + if (i != 0 && i != files.length) { + bos.write(0); + bos.write(0); + } + + bos.write(0, bytes.length); } + // According to MSDN the byte array have to be double NULL-terminated. + // The array contains Unicode characters, so each NULL-terminator is + // a pair of bytes + + bos.write(new byte [] {0,0}); + bos.write(new byte [] {0,0}); + // Source data is an InputStream. For arbitrary flavors, just grab the // bytes and dump them into a byte array. For text flavors, decode back // to a String and recur to reencode according to the requested format. --- old/src/windows/native/sun/windows/awt_DnDDS.cpp 2008-06-30 15:13:25.031250000 +0400 +++ new/src/windows/native/sun/windows/awt_DnDDS.cpp 2008-06-30 15:13:24.062500000 +0400 @@ -649,7 +649,7 @@ dropfiles->pt.x = m_dropPoint.x; dropfiles->pt.y = m_dropPoint.y; dropfiles->fNC = m_fNC; - dropfiles->fWide = FALSE; // good guess! + dropfiles->fWide = TRUE; // good guess! dataout += sizeof(DROPFILES); } @@ -816,7 +816,7 @@ dropfiles->pt.x = m_dropPoint.x; dropfiles->pt.y = m_dropPoint.y; dropfiles->fNC = m_fNC; - dropfiles->fWide = FALSE; // good guess! + dropfiles->fWide = TRUE; // good guess! dataout += sizeof(DROPFILES); }
07-02-2007
EVALUATION This is a common behavior. I tried to set Russian locale and English for the application without Unicode support. The situation is the same (incorrect encoding). I found that in DROPFILES structure we set fWide field in FALSE. (for more information about DROPFILES look throug MSDN documentation) This looks incorrect. I changed fWide value to TRUE. NOw instead of wrong encoding I see nulls with non-cyrilic characters. This brings my to idea that the way we convert filelists into the native byte array is not appropriate for unicode and should be rewritten. See also: awt_DnDDS.cpp; DROPFILES structure is prepeared and used here See also: AwtDataTransferer->GetData and GetDataHere methods.
05-02-2007
EVALUATION The test given in the description fails starting with the tiger build 11 if the following Regional and Language Options are in effect: Regional Options, Standards and formats: Japanese Advanced, Language for non-Unicode programs: English (United States) If Regional and Language Options are: Regional Options, Standards and formats: Japanese Advanced, Language for non-Unicode programs: Japanese then the test passes. Most likely, the issue is caused by the fix for 4637798 (Use Unicode APIs on Windows NT/2000/XP) integrated in the tiger build 11: the fix for 4637798 did incomplete "unicodization" of AWT/DnD. Our implementation of IDataObject should provide files (data in CF_HDROP format) in Unicode. ###@###.### 2004-09-15
15-09-2004

Relates :	JDK-4899439 - File uses strings for names but file names are byte arrays on OS
Relates :	JDK-6576211 - DnD of File-List between JVM is broken for non ASCII file names - XToolkit