When and how to perform one to 0..n mapping Stream mapMulti over flatMap as of Java 16
Stream::mapMulti is a new method that is classified as an intermediate operation.
It requires a BiConsumer<T, Consumer<R>> mapper of the element about to be processed a Consumer. The latter makes the method look strange at the first glance because it is different from what we are used to at the other intermediate methods such as map, filter, or peek where none of them use any variation of *Consumer.
The purpose of the Consumer provided right within the lambda expression by the API itself is to accept any number elements to be available in the subsequent pipeline. Therefore, all the elements, regardless of how many, will be propagated.
Explanation using simple snippets
One to some (0..1) mapping (similar to
filter)Using the
consumer.accept(R r)for only a few selected items achieves filter-alike pipeline. This might get useful in case of checking the element against a predicate and it's mapping to a different value, which would be otherwise done using a combination offilterandmapinstead. The followingStream.of("Java", "Python", "JavaScript", "C#", "Ruby") .mapMulti((str, consumer) -> { if (str.length() > 4) { consumer.accept(str.length()); // lengths larger than 4 } }) .forEach(i -> System.out.print(i + " ")); // 6 10One to one mapping (similar to
map)Working with the previous example, when the condition is omitted and every element is mapped into a new one and accepted using the
consumer, the method effectively behaves likemap:Stream.of("Java", "Python", "JavaScript", "C#", "Ruby") .mapMulti((str, consumer) -> consumer.accept(str.length())) .forEach(i -> System.out.print(i + " ")); // 4 6 10 2 4One to many mapping (similar to
flatMap)Here things get interesting because one can call
consumer.accept(R r)any number of times. Let's say we want to replicate the number representing the String length by itself, i.e.2becomes2,2.4becomes4,4,4,4. and0becomes nothing.Stream.of("Java", "Python", "JavaScript", "C#", "Ruby", "") .mapMulti((str, consumer) -> { for (int i = 0; i < str.length(); i++) { consumer.accept(str.length()); } }) .forEach(i -> System.out.print(i + " ")); // 4 4 4 4 6 6 6 6 6 6 10 10 10 10 10 10 10 10 10 10 2 2 4 4 4 4
Comparison with flatMap
The very idea of this mechanism is that is can be called multiple times (including zero) and its usage of SpinedBuffer internally allows to push the elements into a single flattened Stream instance without creating a new one for every group of output elements unlike flatMap. The JavaDoc states two use-cases when using this method is preferable over flatMap:
- When replacing each stream element with a small (possibly zero) number of elements. Using this method avoids the overhead of creating a new Stream instance for every group of result elements, as required by flatMap.
- When it is easier to use an imperative approach for generating result elements than it is to return them in the form of a Stream.
Performance-wise, the new method mapMulti is a winner in such cases. Check out the benchmark at the bottom of this answer.
Filter-map scenario
Using this method instead of filter or map separately doesn't make sense due to its verbosity and the fact one intermediate stream is created anyway. The exception might be replacing the .filter(..).map(..) chain called together, which comes handy in the case such as checking the element type and its casting.
int sum = Stream.of(1, 2.0, 3.0, 4F, 5, 6L)
.mapMultiToInt((number, consumer) -> {
if (number instanceof Integer) {
consumer.accept((Integer) number);
}
})
.sum();
// 6
int sum = Stream.of(1, 2.0, 3.0, 4F, 5, 6L)
.filter(number -> number instanceof Integer)
.mapToInt(number -> (Integer) number)
.sum();
As seen above, its variations like mapMultiToDouble, mapMultiToInt and mapMultiToLong were introduced. This comes along the mapMulti methods within the primitive Streams such as IntStream mapMulti(IntStream.IntMapMultiConsumer mapper). Also, three new functional interfaces were introduced. Basically, they are the primitive variations of BiConsumer<T, Consumer<R>>, example:
@FunctionalInterface
interface IntMapMultiConsumer {
void accept(int value, IntConsumer ic);
}
Combined real use-case scenario
The real power of this method is in its flexibility of usage and creating only one Stream at a time, which is the major advantage over flatMap. The two below snippets represent a flatmapping of Product and its List<Variation> into 0..n offers represented by the Offer class and based on certain conditions (product category and the variation availability).
ProductwithString name,int basePrice,String categoryandList<Variation> variations.VariationwithString name,int priceandboolean availability.
List<Product> products = ...
List<Offer> offers = products.stream()
.mapMulti((product, consumer) -> {
if ("PRODUCT_CATEGORY".equals(product.getCategory())) {
for (Variation v : product.getVariations()) {
if (v.isAvailable()) {
Offer offer = new Offer(
product.getName() + "_" + v.getName(),
product.getBasePrice() + v.getPrice());
consumer.accept(offer);
}
}
}
})
.collect(Collectors.toList());
List<Product> products = ...
List<Offer> offers = products.stream()
.filter(product -> "PRODUCT_CATEGORY".equals(product.getCategory()))
.flatMap(product -> product.getVariations().stream()
.filter(Variation::isAvailable)
.map(v -> new Offer(
product.getName() + "_" + v.getName(),
product.getBasePrice() + v.getPrice()
))
)
.collect(Collectors.toList());
The use of mapMulti is more imperatively inclined compared to the declarative approach of the previous-versions Stream methods combination seen in the latter snippet using flatMap, map, and filter. From this perspective, it depends on the use-case whether is easier to use an imperative approach. Recursion is a good example described in the JavaDoc.
Benchmark
As promised, I have wrote a bunch of micro-benchmarks from ideas collected from the comments. As long as there is quite a lot of code to publish, I have created a GitHub repository with the implementation details and I am about to share the results only.
Stream::flatMap(Function) vs Stream::mapMulti(BiConsumer) Source
Here we can see the huge difference and a proof the newer method actually works as described and its usage avoid the overhead of creating a new Stream instance with each processed element.
Benchmark Mode Cnt Score Error Units
MapMulti_FlatMap.flatMap avgt 25 73.852 ± 3.433 ns/op
MapMulti_FlatMap.mapMulti avgt 25 17.495 ± 0.476 ns/op
Stream::filter(Predicate).map(Function) vs Stream::mapMulti(BiConsumer) Source
Using chained pipelines (not nested, though) is fine.
Benchmark Mode Cnt Score Error Units
MapMulti_FilterMap.filterMap avgt 25 7.973 ± 0.378 ns/op
MapMulti_FilterMap.mapMulti avgt 25 7.765 ± 0.633 ns/op
Stream::flatMap(Function) with Optional::stream() vs Stream::mapMulti(BiConsumer) Source
This one is very interesting, escpecially in terms of usage (see the source code): we are now able to flatten using mapMulti(Optional::ifPresent) and as expected, the new method is a bit faster in this case.
Benchmark Mode Cnt Score Error Units
MapMulti_FlatMap_Optional.flatMap avgt 25 20.186 ± 1.305 ns/op
MapMulti_FlatMap_Optional.mapMulti avgt 25 10.498 ± 0.403 ns/op
To address the scenario
When it is easier to use an imperative approach for generating result elements than it is to return them in the form of a Stream.
We can see it as now having a limited variant of the yield statement C#. The limitations are that we always need an initial input from a stream, as this is an intermediate operation, further, there’s no short-circuiting for the elements we’re pushing in one function evaluation.
Still, it opens interesting opportunities.
E.g., implementing a stream of Fibonacci number formerly required a solution using temporary objects capable of holding two values.
Now, we can use something like:
IntStream.of(0)
.mapMulti((a,c) -> {
for(int b = 1; a >=0; b = a + (a = b))
c.accept(a);
})
/* additional stream operations here */
.forEach(System.out::println);
It stops when the int values overflow, as said, it won’t short-circuit when we use a terminal operation that does not consume all values, however, this loop producing then-ignored values might still be faster than the other approaches.
Another example inspired by this answer, to iterate over a class hierarchy from root to most specific:
Stream.of(LinkedHashMap.class).mapMulti(MapMultiExamples::hierarchy)
/* additional stream operations here */
.forEach(System.out::println);
}
static void hierarchy(Class<?> cl, Consumer<? super Class<?>> co) {
if(cl != null) {
hierarchy(cl.getSuperclass(), co);
co.accept(cl);
}
}
which unlike the old approaches does not require additional heap storage and will likely run faster (assuming reasonable class depths that do not make recursion backfire).
Also monsters like this
List<A> list = IntStream.range(0, r_i).boxed() .flatMap(i -> IntStream.range(0, r_j).boxed() .flatMap(j -> IntStream.range(0, r_k) .mapToObj(k -> new A(i, j, k)))) .collect(Collectors.toList());
can now be written like
List<A> list = IntStream.range(0, r_i).boxed()
.<A>mapMulti((i,c) -> {
for(int j = 0; j < r_j; j++) {
for(int k = 0; k < r_k; k++) {
c.accept(new A(i, j, k));
}
}
})
.collect(Collectors.toList());
Compared to the nested flatMap steps, it loses some parallelism opportunity, which the reference implementation didn’t exploit anyway. For a non-short-circuiting operation like above, the new method likely will benefit from the reduced boxing and less instantiation of capturing lambda expressions. But of course, it should be used judiciously, not to rewrite every construct to an imperative version (after so many people tried to rewrite every imperative code into a functional version)…