Clean .bbl files with latexmk -C

By default, latexmk treats .bbl files as non-regeneratable. This is because the .bib from which they are made is not always available. See the documentation for a fuller explanation, especially the explanation of the $bibtex_use variable. To get latexmk -C to delete .bbl files, put the following in an initialization file:

$bibtex_use = 2;

Another option to the presumably canonical answer John gave is specifying $clean_ext in a latexmkrc file, which contains your personalized global options. This file can be in one of the places I outlined in my answer to Latexmk: makeglossaries, Biber and pdfLaTeX / Where to put latexmkrc?. Also see the latexmk manual.

My complete list of extensions looks like this:

$clean_ext = 'synctex.gz synctex.gz(busy) run.xml tex.bak bbl bcf fdb_latexmk run tdo %R-blx.bib'

(As a result of -- I think -- SyncTeX, latexmk, Texmaker, biblatex, Biber, todonotes)