JDK-8308798 : Wrong behavior of DecimalFormat with RoundingMode.UP in special case
  • Type: CSR
  • Component: core-libs
  • Sub-Component: java.text
  • Priority: P4
  • Status: Closed
  • Resolution: Withdrawn
  • Submitted: 2023-05-24
  • Updated: 2025-04-23
  • Resolved: 2025-04-23
Related Reports
CSR :  
Description
Summary
-------

Correct `DecimalFormat`’s implementation to handle an edge case when formatting a fractional floating-point value would return a zero string, violating the *UP*, *CEILING*, and *FLOOR* `RoundingMode` contracts.

Problem
-------

`DecimalFormat`, when formatting a floating-point value, will first make a conversion from a floating-point value to a string representation, (from here on, out referred to as **s**). **s** is then truncated and rounded under the `DecimalFormat` pattern and `RoundingMode` to create the resultant string returned.

In this particular edge case, if the maximum fractional digits given by the `DecimalFormat` pattern is less than the amount of leading zeroes in the fractional portion of **s**, `DecimalFormat` incorrectly returned a zero string, instead of considering rounding. By not considering rounding, formatting certain floating-point values violated the *UP*, *CEILING*, and *FLOOR* `RoundingMode` contracts. Specifically, the stipulation: "Note that this rounding mode never decreases the magnitude of the calculated value".

For example, consider a `DecimalFormat` with pattern “0.0” and `RoundingMode.UP` which formats the numerical value `.001`. This numerical value, which is a representation of the floating-point value `0.001000000000000000020816681711721685132943093776702880859375` is converted to the decimal string “0.001” by `DecimalFormat`. The existing implementation assumes it can return zero as the fractional precision (1) given by the `DecimalFormat` pattern is less than the count of fractional leading zeros (2) in the decimal string. However, returning the formatted string "0.0" violates the `RoundingMode.UP` contract and should have been "0.1".


Solution
--------

Ensure that rounding is considered, even if the maximum fractional digits allowed by the `DecimalFormat` pattern is less than the leading fractional zeroes given by **s** for formatting a fractional floating-point value. 

Given the double value `0.00010000000000000000479217...` represented by the numerical value `0.0001`,
the following is an example of the behavioral changes under every RoundingMode with a "0.00" pattern `DecimalFormat`. Unless marked as different, the results are the same as before the change.

    0.0001 formatted with 0.00 and mode UP gives 0.01 (previously, 0.00)
    0.0001 formatted with 0.00 and mode DOWN gives 0.00
    0.0001 formatted with 0.00 and mode CEILING gives 0.01 (previously, 0.00)
    0.0001 formatted with 0.00 and mode FLOOR gives 0.00
    0.0001 formatted with 0.00 and mode HALF_UP gives 0.00
    0.0001 formatted with 0.00 and mode HALF_DOWN gives 0.00
    0.0001 formatted with 0.00 and mode HALF_EVEN gives 0.00
    -0.0001 formatted with 0.00 and mode UP gives -0.01 (previously, -0.00)
    -0.0001 formatted with 0.00 and mode DOWN gives -0.00
    -0.0001 formatted with 0.00 and mode CEILING gives -0.00
    -0.0001 formatted with 0.00 and mode FLOOR gives -0.01 (previously, -0.00)
    -0.0001 formatted with 0.00 and mode HALF_UP gives -0.00
    -0.0001 formatted with 0.00 and mode HALF_DOWN gives -0.00
    -0.0001 formatted with 0.00 and mode HALF_EVEN gives -0.00 

Note: This change does not affect the case where the `DecimalFormat` maximum fractional digits is equivalent to the leading fractional zeroes given by **s**. Rounding behavior for that case remains the same. For example, the double value `0.0500000000000000027755...` given by the numerical value `0.05` would still format to `0.1` under a "0.0" DecimalFormat pattern with `HALF_UP`. (This behavior correctly occurs before and after this change).

Specification
-------------

N/A (behavioral change only)


Comments
Withdrawing this CSR for now. The current issue at hand now has a proposed solution of adding a new method that uses C-style formatting. However, this method would come first for util.Formatter, and text.DecimalFormat would be able to leverage it later as well. This CSR can be re-considered when we have a concrete plan forward. JDK-8313963 is also withdrawn as well.
23-04-2025

The JDK 24 rampdown 1 deadline is fast-approaching; a reminder to ourselves to reconsider this work for JDK 25.
03-12-2024

Changing fixVersion to 24 as JDK 23 is in ramp down 2.
20-07-2024

I see both of your points. I am going to take a step back and look at it from a wider lens. I am inclined to do "either follow and generalize the spec of Formatter, which in fact formats the decimal produced by Double.toString()" as this is what DecimalFormat does, with the exception of the fringe case handling. However, as you stated, the class needs specification regarding this. I will also discuss the proposed solutions with Naoto.
07-05-2024

[~jlu], stepping back, I would summarize this issue as "DecimalFormat has double-rounding hazards" rather than just "wrong behavior RoundingMode.UP in special case." I think it would be preferable to address the larger category of double-rounding hazards in one bug fix rather than split them out by rounding mode.
03-05-2024

[~jlu] DecimalFormat has no rigorous spec, so there's latitude in the implementation. Take the example discussed in a comment on https://bugs.openjdk.org/browse/JDK-7131459 . The full expansion of double 0.8055d is 0.80549999999999999378275106209912337362766265869140625 If DecimalFormat is used with format "0.0000" (4 fractional digits), it correctly produces "0.8055" But if the format is like "0.000000000000000000000000000000" (30 fractional digits) the output is the incorrect "0.805500000000000000000000000000" whereas the correct output is "0.805499999999999993782751062099" To me, this means that the DecimalFormat's implementation is very ad hoc. It might be tailored to handle some special fringe cases, but it shows weaknesses as well. As I see it, there are two ways out: * either follow and generalize the spec of Formatter, which in fact formats the decimal produced by Double.toString() * or specify that DecimalFormat formats the full decimal expansion of a double/float. Other approaches are too ad hoc.
03-05-2024

Hi [~darcy], I'd like to clarify, when you say "specification update" are you referring to adding specification with the associated fix, or did you mean to update the wording in the solution with the "different rounding modes..." Also, for your previous comment with the "0.05... 0.15... ... 0.95..." examples. If we are rounding to the tenths, the implementation will calculate the value through a path independent of this fix. I will look into these cases as well, but just wanted to clarify that point.
03-05-2024

[~rgiulietti], I agree, I think DecimalFormat could benefit from some specification regarding rounding/floating-point. I can file a separate issue for that, perhaps we should also discuss the compatibility concerns there around aligning DecimalFormat with Formatter. I forget if it was previously mentioned, but I think aligning the classes is along the lines of reverting https://bugs.openjdk.org/browse/JDK-7131459 (which changed DecimalFormat to prevent double rounding, unlike Formatter) IMHO, for this particular edge case, (regarding the UP, CEILING, and FLOOR modes) I think it is more clear to make the fix distinct from the aforementioned issue. That is, I think that these should be two separate issues.
03-05-2024

Moving to Provisional to indicate this issue is worth solving; however, I think the specification update needs to more fully cover different rounding modes and conditions before the request can be Finalized.
02-05-2024

Let's say you want to format the double 0.1d to 17 fractional digits, rounding with modes UP, HALF_UP, or CEILING. The full expansion of double 0.1d is 0.1000000000000000055511151231257827021181583404541015625 The 18th fractional digit is the first 5 after the long string of 0 followed by something non-zero, so rounding with any of the above modes means producing the decimal number 0.10000000000000001 But new DecimalFormat("0.00000000000000000") (17 zeros in the fractional part) outputs "0.10000000000000000" in any of the above modes. It might be surprising and even perceived as a bug. But at least it is consistent with what Formatter on format "%.17f" and (non-negotiable) HALF_UP does. Indeed, Formatter works "as if" by first converting 0.1d to "0.1", regardless of the requested precision. Then, "as if" manipulating "0.1" to fulfill the request of 17 digits, it produces the output. Of course, at this point it cannot just invent the 1 at the 17th position out of thin air, so it will produce "0.10000000000000000" This is specified behavior: "If the precision is less than the number of digits which would appear after the decimal point in the string returned by Double.toString(double), then the value will be rounded using the round half up algorithm. Otherwise, zeros may be appended to reach the precision." I think DecimalFormat and Formatter should be consistent between each other, so DecimalFormat should not attempt to be more accurate than Formatter. We should be careful to specify the behavior of DecimalFormat in a way that the outputs are consistent with Formatter, and implement it correctly.
02-05-2024

Hi [~jlu], Let me clarify my concerns; I haven't fully thought through whether this is is not a problem. Let's say you're rounding under one of the to-nearest rounding modes and the numerical value is in 0.5 in the position *after* the one being rounded to. For example, if you were rounding to the tenth's position, then the value are like one of 0.05... 0.15... ... 0.95... With an intermediate rounding to string, it isn't clear to me all these operation will round correctly. HTH
01-05-2024

Hi [~darcy], if I interpreted your comment correctly, you are talking about a case such as a DecimalFormat with the pattern "0.0" formatting the double `0.0500000001...` This would correctly be formatted to "0.1" under `HALF_UP`, (before and after this change). This fix does not affect that case, as the fractional digits (1) is equal to the leading zeroes in the value returned by `Double.toString(0.0500000001...)`, also (1). The case that is affected would be if a pattern "0.0" was used to format the double `0.0050000000000000001...` In that case, (before and after this change) the formatted value is still "0.0" under `HALF_UP`. I have updated the CSR to make this more clear, as well as provided further examples under all `RoundingModes`.
01-05-2024

Moving back to Draft. [~jlu], I support this issue being fixed. However, I'm not sure the full scope of the behavioral changes implied by the intermediate conversion to string in the implementation has been captured. For example, it is conceivable that double values near half-way cases, e.g. 0.0500000001... etc. would trigger analagous issues for the half-way rounding modes. Please investigate if those cases occurs too before re-Proposing the CSR.
30-04-2024

[~jlu] Ah, I see.
25-04-2024

Hi [~rgiulietti], I filed the CSR because while only an implementation only change, it has behavioral compatibility impact which IIUC requires a CSR. For this specific issue, I believe the scope should be focused on fixing only this edge case bug. There is a separate issue filed that addresses the lack of floating-point specification for DecimalFormat, (that you mentioned previously).
25-04-2024

[~jlu] Not sure about the reason for the CSR if nothing changes in the specification...
25-04-2024

Hi Joe, Raffaello, I plan to resume work here for JDK 23. I have made updates and re-proposed the CSR for more feedback on the latest version.
23-04-2024

Setting fixVersion to 23 as a reminder for us to revisit the issue.
07-12-2023

A direct rounding from a (binary) floating-point value to decimal might need a lot of computational resources, depending on the precision implied by the `DecimalFormat` pattern. For example, if the pattern has 50 zeros after the decimal separator, high precision arithmetic (like `BigDecimal`) is rather unavoidable, even for harmless looking numbers as `0.002485d`. `j.u.Formatter` chooses to first convert the floating-point value to decimal using the algorithm of `Double.toString(double)`, then to round the resulting decimal if necessary, using `RoundingMode.HALF_UP`. This might incur double-rounding (as with `0.002485d`), but is relatively cheap, as the first conversion is highly optimized. Thus, while a clean mathematical semantics should be the ideal goal, other considerations like execution costs and consistency with `j.u.Formatter` might play an important role as well.
27-07-2023

Sorry for the delay in reviewing. I'm moving this request back to Provisional; there is an issue to be solved, but I'm not convinced the current approach is the right one. Decimal <-> binary conversion can be subtle and tricking and some formulations of the process introduce extra (potential) for errors or confusions. I think the numerically best way is to define the semantics in terms of a conversion between a binary format, such as 64-bit double, and a decimal format implied by the pattern being used. Such a formulations leads to a numerically well-defined answer in terms of inputs and output. For example, for a double number with a numerical value of X, there can be decimal values of the given format bracketing X and then IEEE 754-style rounding rules can be used to pick the correct one. (Leaving out special cases of exact conversion, out of range, etc.)
27-07-2023

I think the behavior of DecimalFormat should be consistent with j.u.Formatter. Here's an example of a difference: ``` var df = new DecimalFormat("0.00000"); df.setRoundingMode(RoundingMode.HALF_UP); df.format(0.002485) -> "0.00248" ``` ``` String.format("%.5f", 0.002485) -> "0.00249" ``` The main issue with DecimalFormat is that it is rather under-specified when it comes to floating-point types, regardless of rounding modes.
25-07-2023

In short, it is kind of both. What the end user "encounters" in terms of this special case would be the default decimal expansion of the double. However, the implementation itself is based on the Binary to ASCII to String representation of the number. In this special case, either representation could technically "fulfill" the condition, as the implementation need not consider anything beyond the fact that the number is non zero, the rounding mode, and if it is under-flowing to zero. For example, whether it be 0.0001 or 0.000100000000000000004792173602..., I believe the desired phrasing depends on the viewpoint. I have updated the wording in the CSR, problem -> is phrased as to what the end user experiences (since they are the ones experiencing the problem) and solution -> is phrased in terms of the implementation (since the implementation is what will fix the issue). Please let me know if you disagree with any of these decisions and I can update accordingly.
26-05-2023

Moving to Provisional, not Approved. The phase "if the number has more zeros between the decimal and first non zero digit than fractional digits in the pattern" is not necessarily well-defined. Is what is meant "In the exact decimal expansion of the numerical value stored in a finite double..." or "In the default decimal expansion of double..." which was improved/corrected a few releases ago. Something else?
26-05-2023