Making overview with linked detailed maps in QGIS?

It is possible to create an overview map for multiple detail maps in Print Composer. The workflow is as follows (see the user docs on this topic as well):

1. Create the detail maps
2. Add another map item and set it to overview
3. Configure the overview map by adding the detail maps to its settings

With help of the answer above, I figured out how to make the overview!

The problem was how, in the third step, to make the overview show all of my detailed maps in one map.

I made a map (step 2) and then I added one overview for each of my detailed maps using the plus button. For each of my detailed maps, I checked the box 'lock layer for map item'.

Then I zoomed out on the map canvas in QGIS, and chose layers to be shown on the overview map.

Then I went back to the Print Composer, and with the overview map frame activated I clicked 'Set to map canvas extent'.

My overview map now shows where my detailed maps are located.