Why do we need flatMap (in general)?

FlatMap, known as "bind" in some other languages, is as you said yourself for function composition.

Imagine for a moment that you have some functions like these:

def foo(x: Int): Option[Int] = Some(x + 2)
def bar(x: Int): Option[Int] = Some(x * 3)

The functions work great, calling foo(3) returns Some(5), and calling bar(3) returns Some(9), and we're all happy.

But now you've run into the situation that requires you to do the operation more than once.

foo(3).map(x => foo(x)) // or just foo(3).map(foo) for short

Job done, right?

Except not really. The output of the expression above is Some(Some(7)), not Some(7), and if you now want to chain another map on the end you can't because foo and bar take an Int, and not an Option[Int].

Enter flatMap

foo(3).flatMap(foo)

Will return Some(7), and

foo(3).flatMap(foo).flatMap(bar)

Returns Some(15).

This is great! Using flatMap lets you chain functions of the shape A => M[B] to oblivion (in the previous example A and B are Int, and M is Option).

More technically speaking; flatMap and bind have the signature M[A] => (A => M[B]) => M[B], meaning they take a "wrapped" value, such as Some(3), Right('foo), or List(1,2,3) and shove it through a function that would normally take an unwrapped value, such as the aforementioned foo and bar. It does this by first "unwrapping" the value, and then passing it through the function.

I've seen the box analogy being used for this, so observe my expertly drawn MSPaint illustration:

This unwrapping and re-wrapping behavior means that if I were to introduce a third function that doesn't return an Option[Int] and tried to flatMap it to the sequence, it wouldn't work because flatMap expects you to return a monad (in this case an Option)

def baz(x: Int): String = x + " is a number"

foo(3).flatMap(foo).flatMap(bar).flatMap(baz) // <<< ERROR

To get around this, if your function doesn't return a monad, you'd just have to use the regular map function

foo(3).flatMap(foo).flatMap(bar).map(baz)

Which would then return Some("15 is a number")

It's the same reason you provide more than one way to do anything: it's a common enough operation that you may want to wrap it.

You could ask the opposite question: why have map and flatten when you already have flatMap and a way to store a single element inside your collection? That is,

x map f
x filter p

can be replaced by

x flatMap ( xi => x.take(0) :+ f(xi) )
x flatMap ( xi => if (p(xi)) x.take(0) :+ xi else x.take(0) )

so why bother with map and filter?

In fact, there are various minimal sets of operations you need to reconstruct many of the others (flatMap is a good choice because of its flexibility).

Pragmatically, it's better to have the tool you need. Same reason why there are non-adjustable wrenches.