JDK-5005426 : Buffered stream data is discarded by IllegalStateException in 1.4.2 and Tiger
  • Type: Bug
  • Component: core-libs
  • Sub-Component: java.nio.charsets
  • Affected Version: 5.0,5.0u13,6
  • Priority: P3
  • Status: Resolved
  • Resolution: Fixed
  • OS: solaris_10,windows_xp
  • CPU: generic,x86
  • Submitted: 2004-03-01
  • Updated: 2005-03-04
  • Resolved: 2005-03-04
The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.
Other Other JDK 6
1.4.2_09Fixed 5.0u17Fixed 6 b27Fixed
Related Reports
Duplicate :  
Relates :  
Relates :  
Description
There is such big issues as buffered Data lost 
This is because flush operation is not done with IllegalStateException.

The attached test program works well in 1.3.1_0X(1.4.1_0X also),
but not in 1.4.2_0X(1.5.beta38).

REPRODUCE :
 (1) Compile DataLost.java 
 (2) Launch "java DataLost windows-31j" in 1.4.2_03(or Tigerbeta38)

     You will see the stack trace as follows.

K:\nio-lost-data>java DataLost windows-31j   
java.lang.IllegalStateException: Current state = FLUSHED, new state = CODING_END

        at java.nio.charset.CharsetEncoder.throwIllegalStateException(CharsetEnc
oder.java:939)
        at java.nio.charset.CharsetEncoder.encode(CharsetEncoder.java:535)
        at sun.nio.cs.StreamEncoder$CharsetSE.flushLeftoverChar(StreamEncoder.ja
va:358)
        at sun.nio.cs.StreamEncoder$CharsetSE.implClose(StreamEncoder.java:414)
        at sun.nio.cs.StreamEncoder.close(StreamEncoder.java:160)
        at java.io.OutputStreamWriter.close(OutputStreamWriter.java:222)
        at java.io.PrintWriter.close(PrintWriter.java:219)
        at DataLost.testMain(DataLost.java:26)
        at DataLost.main(DataLost.java:5)


The java.lang.IllegalStateException occurs in 1.4.2( and Tiger b38 ),
not in 1.3.1(1.4.0).


INVESTIGATION:

The following test program outputs some string to a file, "TestFile".
This program generates an exception intentionally.
 (Please don't consider the sample code has problem.)
Please consider FileOutputStreamEX class simulates an exception 
in PrintWriter.

  
===== Test Program ( DataLost.java ) ====
import java.io.*;
class DataLost {

    public static void main( String arg[] ) {
	testMain("TestFile", arg[0]);
    }

    public static void testMain( String filename, String charsetName ) {
	try {
	    FileOutputStreamEX fos = new FileOutputStreamEX( filename );
	    OutputStreamWriter mosw = new OutputStreamWriter( fos, charsetName);
	    PrintWriter pw = new PrintWriter( mosw );

	    fos.dontClose();
	    for( int i=0;i<100;i++ ){
		pw.write(i+"  ");
	    }
	    try {
		pw.close();             //  1st PrintWriter Close
	    }
	    catch ( Exception e ) {
	    }

	    fos.canClose();
	    try {
		pw.close();             //  2nd PrintWriter Close
	    }
	    catch (Exception e ) {
		e.printStackTrace();
	    }
	}
	catch (Exception e ) {
	    e.printStackTrace();
	}
    }
}

class FileOutputStreamEX extends BufferedOutputStream {
				   
    private boolean status;

    FileOutputStreamEX( String str ) throws IOException {
	super( new FileOutputStream( str ) );
    }

    public void dontClose() {
	status = false;
    }

    public void canClose() {
	status = true;
    }

    public void close() throws IOException {
	if ( status == false ) {
	    throw new IOException(" can not close ");
	}
	super.close();
    }
}
==========================================

In the above program, an exception occurs in "1st PrintWriter Close" line.
Then the program is expected  to flush data and close correctly 
in "2nd PrintWriter Close" line.
However, the flush doesn't run because of exception occurrence.

This problem happens to 
  sun.nio.cs.StreamEncoder$CharsetSE.implClose() .
If exception, OutputStream.close is not launched.
As a result, the buffered data in OutputStream is discarded.

The followings shows when this behavior come to be a problem.


==== Sample servlet code =====================
....
public class F extends HttpServlet{
	public void doGet(HttpServletRequest req,HttpServletResponse res)throws ServletException,IOException{

		res.setContentType("text/html;charset=SJIS");
		PrintWriter out = res.getWriter();

		ServletContext sc = getServletContext();
      		RequestDispatcher rd = sc.getRequestDispatcher("/jsp/f.jsp");
		rd.forward(req, res);                   <===(1)

		out.println("</body></html><===(2)
		out.close();                            <=== (3)
        }
}
.....
==============================================

The above is mini code to explain the problem in licensees app. server.

According to the specifications of servlet, a program have to do as in (2) and (3)
once response and request are forwarded.

In 1.4.2(maybe Tiger also), when an exception occurs in (2) and (3),
IllegalStateException occurs and flush won't be done as the test program(DataLost.java)
shows.
It means that the stream data will not be sent to client.

In 1.4.0, the program works well.


PROBLEM: 

- Buffered Data written in stream is discarded.

=============================================================================

###@###.### 2004-03-26

Comments
CONVERTED DATA BugTraq+ Release Management Values COMMIT TO FIX: mustang
08-07-2004

EVALUATION A similar approach to the suggested fix would appear to be the most appropriate way to address this issue which can lead to data loss in some corner cases as demonstrated by the provided test case ###@###.### 2004-04-14 The problem is that the flush method of a CharsetEncoder is being invoked twice, and despite spec that claims this is explicitly allowed, the implementation enforces an internal state change ST_END => ST_FLUSHED, causing the second flush to always fail. I'm not exactly sure what the best fix is, but eliminating the ST_FLUSHED state entirely causes the submitter's program to succeed. ###@###.### 2004-06-14 We need to examine all the whole state transitions for Charset-X-Coder.java more carefully, to make sure the promises made by flush() are kept, among others. It is not entirely clear to me why there is a need for a flush method, given that an end-of-input argument to decode is available. Too late to fix in Tiger. It's a little optimistic to expect good error recovery from an exception thrown in close(). ###@###.### 2004-06-18
18-06-2004

SUGGESTED FIX The followings are suggested fix by the licensee. *** org/StreamEncoder.java 2 27 16:42:32 2004 --- StreamEncoder.java 2 27 19:25:14 2004 *************** *** 411,417 **** --- 411,419 ---- } void implClose() throws IOException { + try { flushLeftoverChar(null, true); + for (;;) { CoderResult cr = encoder.flush(bb); if (cr.isUnderflow()) *************** *** 425,434 **** --- 427,450 ---- } if (bb.position() > 0) writeBytes(); + } catch (IllegalStateException e) { + throw new IOException("IllegalStateException"); + } catch (IllegalArgumentException e) { + throw new IOException("IllegalArgumentException"); + } catch (MalformedInputException e) { + throw new IOException("MalformedInputException"); + } catch (UnmappableCharacterException e) { + throw new IOException("UnmappableCharacterException"); + } catch (CharacterCodingException e) { + throw new IOException("CharacterCodingException"); + } catch (CoderMalfunctionError e) { + throw new IOException("CoderMalfunctionError"); + } finally { if (ch != null) ch.close(); else out.close(); + } } String encodingName() { ###@###.### 2004-03-01 ===========================================================================
01-03-2004