JDK-8218280 : LineNumberReader throws "Mark invalid" exception if CRLF straddles buffer.
  • Type: Bug
  • Component: core-libs
  • Sub-Component: java.io
  • Affected Version: 8u221,11,12,13
  • Priority: P4
  • Status: Resolved
  • Resolution: Fixed
  • OS: windows_10
  • CPU: x86_64
  • Submitted: 2019-02-04
  • Updated: 2019-11-22
  • Resolved: 2019-04-29
The Version table provides details related to the release that this issue/RFE will be addressed.

Unresolved : Release in which this issue/RFE will be addressed.
Resolved: Release in which this issue/RFE has been resolved.
Fixed : Release in which this issue/RFE has been fixed. The release containing this fix may be available for download as an Early Access Release or a General Availability Release.

To download the current JDK release, click here.
JDK 11 JDK 13 Other
11.0.5Fixed 13 b19Fixed openjdk8u232Fixed
Description
A DESCRIPTION OF THE PROBLEM :
The java.io.Reader API says essentially that, if a reader supports mark/reset, marking a certain number of characters using reader.mark() will allow the caller to reset the position back to the beginning of those characters later using reader.reset(), as long as the caller didn't ask the reader to read more characters than I had marked.

LineNumberReader automatically reads two characters internally if it finds a CRLF sequence, normalizing the sequence to LF. This results in two calls to the underlying read() method. If the caller has called reader.mark(1), and if a CRLF straddles the end of the buffer, then reading the second character will invalidate the mark. Subsequently calling reset() to reset the mark will result in "java.ioException: Mark invalid".

Why this happens is obvious, once one is aware of it. However from the point of view of a consumer of the Reader API, the reader is not following its contract. If a consumer has a Reader reference, not knowing whether it is a LineNumberReader or any other sort of reader, it expects that, if mark/reset is supported, then mark/reset should work. In other words, the API contract says that some readers will support mark/reset and some won't. The API does _not_ say that some of those which support mark/reset may fail arbitrarily.

Moreover even if LineNumberReader were intended to behave differently (breaking the Liskov Substitution Principle if not breaking the contract), this is not documented anywhere. In summary, LineNumberReader will behave unexpectedly in a certain edge case and throw an exception, _not related to any fault of the caller_.

STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
1. Create a LineNumberReader with a small buffer size.
2. Provide test data in which CRLF straddles the end of the buffer.
3. Before reading the LF character, set reader.mark(1).
4. After reading the LF character try to reset using reader.reset().

EXPECTED VERSUS ACTUAL BEHAVIOR :
EXPECTED -
The reader should reset to its position before calling mark(1). In the example below, the code should print "bar".
ACTUAL -
java.io.IOException: Mark invalid
	at java.io.BufferedReader.reset(BufferedReader.java:512)
	at java.io.LineNumberReader.reset(LineNumberReader.java:277)
	...

---------- BEGIN SOURCE ----------
final String string = "foo\r\nbar";
try (final Reader reader = new LineNumberReader(new StringReader(string), 5)) {
  reader.read();
  reader.read();
  reader.read();
  reader.read();
  reader.mark(1);
  reader.read();
  reader.reset(); //error!
  System.out.print((char)reader.read());
  System.out.print((char)reader.read());
 System.out.println((char)reader.read());
}
---------- END SOURCE ----------

CUSTOMER SUBMITTED WORKAROUND :
Use "reader.mark(2)" instead of "reader.mark(1)". (But how would a caller know it needs to do that with the general Reader API?)

FREQUENCY : always



Comments
Thanks. Approved.
22-08-2019

Yes 8u does pass the test after patching.
22-08-2019

Does the new test pass on 8u after patching?
21-08-2019

Backporting this patch elimiates a bug in LineNumberReader. Patch applies cleaning to 8u if you account for the change in the pathname for the file. JDK8u tests show failures similar to a clean JDK8u build with no plausible new additions.
16-08-2019

Fix Request Backporting this patch fixes a bug in lineNumberReader. Patch applies cleanly to 11u. Test fails without patch but passes with patch. Tier1 tests pass with this patch
12-08-2019

To reproduce the issue, run the attached test case. JDK 11.0.2 - Fail JDK 12-ea+30 - Fail JDK 13-ea+6 - Fail Output: Exception in thread "main" java.io.IOException: Mark invalid at java.base/java.io.BufferedReader.reset(BufferedReader.java:517) at java.base/java.io.LineNumberReader.reset(LineNumberReader.java:279) at JI9059238.main(JI9059238.java:16)
04-02-2019