JDK-8267359 : Stream.flatMap() consumes entire flatmapped stream when Stream.iterator() is called
  • Type: Bug
  • Component: core-libs
  • Sub-Component: java.util.stream
  • Affected Version: 11.0.9.1,16,17
  • Priority: P4
  • Status: Resolved
  • Resolution: Won't Fix
  • Submitted: 2021-05-19
  • Updated: 2021-06-09
  • Resolved: 2021-06-09
Related Reports
Relates :  
Relates :  
Relates :  
Description
Following up on https://bugs.openjdk.java.net/browse/JDK-8075939, there's still a case where Stream.flatMap() doesn't get short circuited, when using Stream.iterator():

// --------------------------------------------------------
Iterator<Integer> it =
    Stream.of("a", "b")
        .flatMap(s -> Stream
            .of(1, 2, 3, 4)
            .filter(i -> { System.out.println(i); return true; }))
        .iterator();

it.hasNext(); // This consumes the entire flatmapped stream
it.next();
// --------------------------------------------------------

The above program prints:

1
2
3
4

This program never stops:

// --------------------------------------------------------
Iterator<Integer> it =
    Stream.of("a", "b")
        .flatMap(s -> Stream
            .iterate(1, i -> i)
            .filter(i -> { System.out.println(i); return true; }))
        .iterator();

it.hasNext();
it.next();
// --------------------------------------------------------

Comments
The iterator/spliterator terminal operations transform a push-based model to a pull-based model, where there is no notion of cancellation. The pull-model buffers elements pushed to it. Modifying the stream implementation to support a push or pull approach would increase its complexity (we prototyped such an approach, it was complex). Given the iterator/spliterator terminal operations are "escape hatches" for when an existing operation does not suffice, and for many cases the result is unaffected (assuming no side effects) the additional complexity is not worth it. One especially problematic area are infinite streams (as pointed out bu Daniel) or "effectively" infinite streams producing a large quantity of elements. These have to be used with care, otherwise non-termination or an OutOfMemoryException might result. It occurs for the case of push to pull, but also for nested flatMap cases (since cancellation is not a part of the stream or spliterator API). The best i can suggest is the addition of some guidance on such usages, perhaps something in the package-info.java of the java.util.stream package.
09-06-2021

There's some conversation on Twitter about this. Follow the thread up and down from here: https://twitter.com/stuartmarks/status/1395079086087630848
20-05-2021

Related issue: https://bugs.openjdk.java.net/browse/JDK-8267452
20-05-2021