Bibliography in two columns, section title in one

You can redefine the thebibliography environment to use two columns and use the optional argument of multicols to place the title; add the following lines to the preamble of your document:

\usepackage{multicol}

\makeatletter
\renewenvironment{thebibliography}[1]
     {\begin{multicols}{2}[\section*{\refname}]%
      \@mkboth{\MakeUppercase\refname}{\MakeUppercase\refname}%
      \list{\@biblabel{\@arabic\c@enumiv}}%
           {\settowidth\labelwidth{\@biblabel{#1}}%
            \leftmargin\labelwidth
            \advance\leftmargin\labelsep
            \@openbib@code
            \usecounter{enumiv}%
            \let\p@enumiv\@empty
            \renewcommand\theenumiv{\@arabic\c@enumiv}}%
      \sloppy
      \clubpenalty4000
      \@clubpenalty \clubpenalty
      \widowpenalty4000%
      \sfcode`\.\@m}
     {\def\@noitemerr
       {\@latex@warning{Empty `thebibliography' environment}}%
      \endlist\end{multicols}}
\makeatother

Now in the body of your document, simply use

\bibliographystyle{abbrv}
\bibliography{mybib}

Building upon Gonzalo's answer, I suggest to use the etoolbox package to selectively change the definitions of \thebibliography and \endthebibliography. Add the following to your preamble:

\usepackage{multicol}
\usepackage{etoolbox}
\patchcmd{\thebibliography}{\section*{\refname}}
    {\begin{multicols}{2}[\section*{\refname}]}{}{}
\patchcmd{\endthebibliography}{\endlist}{\endlist\end{multicols}}{}{}

EDIT: How does it work? The \patchcmd command needs five arguments, but you may ignore (as I did) the two last ones, which generate patching success/error messages. Argument no. 1 is the command to be "patched" (selectively changed), argument no. 2 is the part of the command definition that should be replaced, argument no. 3 is the replacement. (Because thebibliography is an environment, I had to patch the two commands that define it.)

The easiest way to tackle this problem is to use bibpreamble and bibpostamble:

\usepackage{multicols}

\renewcommand{\bibpreamble}{\begin{multicols}{2}}
\renewcommand{\bibpostamble}{\end{multicols}}

In this manner, only the references are placed in two columns and your title is left out. Nevertheless, it keeps the heading of the bibliography in the chapter-style. To change that, you could use e.g. bibsection:

\usepackage{natbib}

\renewcommand\bibsection{\section{\bibname}}