United StatesChange Country, Oracle Worldwide Web Sites Communities I am a... I want to...
Bug ID: JDK-5005426 Buffered stream data is discarded by IllegalStateException in 1.4.2 and Tiger
JDK-5005426 : Buffered stream data is discarded by IllegalStateException in 1.4.2 and Tiger

Details
Type:
Bug
Submit Date:
2004-03-01
Status:
Resolved
Updated Date:
2005-03-04
Project Name:
JDK
Resolved Date:
2005-03-04
Component:
core-libs
OS:
solaris_10,windows_xp
Sub-Component:
java.nio.charsets
CPU:
x86,generic
Priority:
P3
Resolution:
Fixed
Affected Versions:
5.0,5.0u13,6
Fixed Versions:

Related Reports
Backport:
Backport:
Backport:
Duplicate:
Relates:
Relates:

Sub Tasks

Description
There is such big issues as buffered Data lost 
This is because flush operation is not done with IllegalStateException.

The attached test program works well in 1.3.1_0X(1.4.1_0X also),
but not in 1.4.2_0X(1.5.beta38).

REPRODUCE :
 (1) Compile DataLost.java 
 (2) Launch "java DataLost windows-31j" in 1.4.2_03(or Tigerbeta38)

     You will see the stack trace as follows.

K:\nio-lost-data>java DataLost windows-31j   
java.lang.IllegalStateException: Current state = FLUSHED, new state = CODING_END

        at java.nio.charset.CharsetEncoder.throwIllegalStateException(CharsetEnc
oder.java:939)
        at java.nio.charset.CharsetEncoder.encode(CharsetEncoder.java:535)
        at sun.nio.cs.StreamEncoder$CharsetSE.flushLeftoverChar(StreamEncoder.ja
va:358)
        at sun.nio.cs.StreamEncoder$CharsetSE.implClose(StreamEncoder.java:414)
        at sun.nio.cs.StreamEncoder.close(StreamEncoder.java:160)
        at java.io.OutputStreamWriter.close(OutputStreamWriter.java:222)
        at java.io.PrintWriter.close(PrintWriter.java:219)
        at DataLost.testMain(DataLost.java:26)
        at DataLost.main(DataLost.java:5)


The java.lang.IllegalStateException occurs in 1.4.2( and Tiger b38 ),
not in 1.3.1(1.4.0).


INVESTIGATION:

The following test program outputs some string to a file, "TestFile".
This program generates an exception intentionally.
 (Please don't consider the sample code has problem.)
Please consider FileOutputStreamEX class simulates an exception 
in PrintWriter.

  
===== Test Program ( DataLost.java ) ====
import java.io.*;
class DataLost {

    public static void main( String arg[] ) {
	testMain("TestFile", arg[0]);
    }

    public static void testMain( String filename, String charsetName ) {
	try {
	    FileOutputStreamEX fos = new FileOutputStreamEX( filename );
	    OutputStreamWriter mosw = new OutputStreamWriter( fos, charsetName);
	    PrintWriter pw = new PrintWriter( mosw );

	    fos.dontClose();
	    for( int i=0;i<100;i++ ){
		pw.write(i+"  ");
	    }
	    try {
		pw.close();             //  1st PrintWriter Close
	    }
	    catch ( Exception e ) {
	    }

	    fos.canClose();
	    try {
		pw.close();             //  2nd PrintWriter Close
	    }
	    catch (Exception e ) {
		e.printStackTrace();
	    }
	}
	catch (Exception e ) {
	    e.printStackTrace();
	}
    }
}

class FileOutputStreamEX extends BufferedOutputStream {
				   
    private boolean status;

    FileOutputStreamEX( String str ) throws IOException {
	super( new FileOutputStream( str ) );
    }

    public void dontClose() {
	status = false;
    }

    public void canClose() {
	status = true;
    }

    public void close() throws IOException {
	if ( status == false ) {
	    throw new IOException(" can not close ");
	}
	super.close();
    }
}
==========================================

In the above program, an exception occurs in "1st PrintWriter Close" line.
Then the program is expected  to flush data and close correctly 
in "2nd PrintWriter Close" line.
However, the flush doesn't run because of exception occurrence.

This problem happens to 
  sun.nio.cs.StreamEncoder$CharsetSE.implClose() .
If exception, OutputStream.close is not launched.
As a result, the buffered data in OutputStream is discarded.

The followings shows when this behavior come to be a problem.


==== Sample servlet code =====================
....
public class F extends HttpServlet{
	public void doGet(HttpServletRequest req,HttpServletResponse res)throws ServletException,IOException{

		res.setContentType("text/html;charset=SJIS");
		PrintWriter out = res.getWriter();

		ServletContext sc = getServletContext();
      		RequestDispatcher rd = sc.getRequestDispatcher("/jsp/f.jsp");
		rd.forward(req, res);                   <===(1)

		out.println("</body></html><===(2)
		out.close();                            <=== (3)
        }
}
.....
==============================================

The above is mini code to explain the problem in licensees app. server.

According to the specifications of servlet, a program have to do as in (2) and (3)
once response and request are forwarded.

In 1.4.2(maybe Tiger also), when an exception occurs in (2) and (3),
IllegalStateException occurs and flush won't be done as the test program(DataLost.java)
shows.
It means that the stream data will not be sent to client.

In 1.4.0, the program works well.


PROBLEM: 

- Buffered Data written in stream is discarded.

=============================================================================

###@###.### 2004-03-26

                                    

Comments
SUGGESTED FIX

The followings are suggested fix by the licensee.

*** org/StreamEncoder.java        2 27 16:42:32 2004
--- StreamEncoder.java    2 27 19:25:14 2004
***************
*** 411,417 ****
--- 411,419 ----
        }

        void implClose() throws IOException {
+           try {
                flushLeftoverChar(null, true);
+
                for (;;) {
                    CoderResult cr = encoder.flush(bb);
                    if (cr.isUnderflow())
***************
*** 425,434 ****
--- 427,450 ----
                }
                if (bb.position() > 0)
                    writeBytes();
+           } catch (IllegalStateException e) {
+               throw new IOException("IllegalStateException");
+           } catch (IllegalArgumentException e) {
+               throw new IOException("IllegalArgumentException");
+           } catch (MalformedInputException e) {
+               throw new IOException("MalformedInputException");
+           } catch (UnmappableCharacterException e) {
+               throw new IOException("UnmappableCharacterException");
+           } catch (CharacterCodingException e) {
+               throw new IOException("CharacterCodingException");
+           } catch (CoderMalfunctionError e) {
+               throw new IOException("CoderMalfunctionError");
+           } finally {
                if (ch != null)
                    ch.close();
                else
                    out.close();
+           }
        }

        String encodingName() {
        
###@###.### 2004-03-01
===========================================================================
                                     
2004-03-01
EVALUATION

A similar approach to the suggested fix would appear to be the most appropriate
way to address this issue which can lead to data loss in some corner cases
as demonstrated by the provided test case
###@###.### 2004-04-14

The problem is that the flush method of a CharsetEncoder is being invoked
twice, and despite spec that claims this is explicitly allowed, the
implementation enforces an internal state change ST_END => ST_FLUSHED,
causing the second flush to always fail.
I'm not exactly sure what the best fix is, but eliminating the ST_FLUSHED
state entirely causes the submitter's program to succeed.

###@###.### 2004-06-14

We need to examine all the whole state transitions for
Charset-X-Coder.java more carefully, to make sure the promises
made by flush() are kept, among others.
It is not entirely clear to me why there is a need for a flush
method, given that an end-of-input argument to decode is available.
Too late to fix in Tiger.

It's a little optimistic to expect good error recovery from an exception
thrown in close().

###@###.### 2004-06-18
                                     
2004-06-18
CONVERTED DATA

BugTraq+ Release Management Values

COMMIT TO FIX:
mustang


                                     
2004-07-08



Hardware and Software, Engineered to Work Together