JDK-8267758 : Double nested Stream.flatMap buffer the entire Stream before processing it
  • Type: Bug
  • Component: core-libs
  • Sub-Component: java.util.stream
  • Affected Version: 11,16,17
  • Priority: P4
  • Status: Resolved
  • Resolution: Won't Fix
  • Submitted: 2021-05-25
  • Updated: 2021-06-09
  • Resolved: 2021-06-09
Related Reports
Relates :  
Relates :  
Relates :  
Description
ADDITIONAL SYSTEM INFORMATION :
Microsoft Windows 10 Pro 10.0.19042

A DESCRIPTION OF THE PROBLEM :
Nesting a `flatMap` call inside another `flatMap` produces a complete stream buffering as the one described at https://bugs.java.com/bugdatabase/view_bug.do?bug_id=8075939

The inner Stream seems to be completely evaluated, making it impossible to use Streams comfortably in some lazy-evaluated applications (cartesian product, for instance) that would seem natural given the API.

STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
Execute a `flatMap` operation nested in another `flatMap` operation.

EXPECTED VERSUS ACTUAL BEHAVIOR :
EXPECTED -
a
ax
axx
axxx
axxxx
ACTUAL -
OutOfMemoryException as sd::repeat is evaluated from start to finish filling the heap with repetitions of the string "x".

---------- BEGIN SOURCE ----------
    public static void main(String[] args) {
        var max = Integer.MAX_VALUE;
        Stream.of("a", "b", "c")
            .flatMap(s -> Stream.of("x", "y")
                .flatMap(sd -> IntStream.rangeClosed(0, max)
                    .mapToObj(sd::repeat)))
            .map(s -> s + "u")
            .limit(5)
            .forEach(System.out::println);
    }
---------- END SOURCE ----------

CUSTOMER SUBMITTED WORKAROUND :
None found using the Streams API.

FREQUENCY : always



Comments
See comment JDK-8267359: https://bugs.openjdk.java.net/browse/JDK-8267359?focusedCommentId=14426679&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14426679
09-06-2021

The observations on Windows 10: JDK 11: Failed, OutOfMemoryError thrown JDK 16: Failed. JDK 17ea+6: Failed.
26-05-2021