How can I put a matrix in a figure caption?

Boxing the smallmatrix first also solves the problem. For this, you have to define a box using \newsavebox{<box>} and store the contents using \savebox{<box>}{<stuff>}:

\newsavebox{\smlmat}% Box to store smallmatrix content
\savebox{\smlmat}{$\left(\begin{smallmatrix}2&-2\\-1&2\end{smallmatrix}\right)$}

Here is a complete minimal example:

\documentclass{amsart}
\begin{document}

\newsavebox{\smlmat}% Box to store smallmatrix content
\savebox{\smlmat}{$\left(\begin{smallmatrix}2&-2\\-1&2\end{smallmatrix}\right)$}

\begin{figure}
\begin{picture}(0,0)(-72,-75)
\put(12,-68){\small$\alpha_2$}
\put(-15,-71){\small$\alpha_1$}
\put(-50,-60){\small$-\alpha_1$}
\put(-47,-44){\small$2\alpha_1+\alpha_2$}
\put(-60,-10){\small$-2\alpha_1-\alpha_2$}
\end{picture}
\caption{Some roots, associated to the Cartan matrix~\usebox{\smlmat}.}
\end{figure}

\end{document}​

You can use the optional argument of \caption:

\caption[Roots associated to the Cartan matrix]{Some roots, associated to the Cartan matrix $\left(\begin{smallmatrix}2&-2\\-1&2\end{smallmatrix}\right)$.}

After all, the matrix wouldn't look good in a list of figures.

Another way of avoiding the error is to load the caption package an using the singlelinechek=off option; this approach, however, will fail if a List of Figures needs to be produced:

\documentclass{amsart}
\usepackage{graphics,  epsfig, psfrag}  
\usepackage{caption}

\begin{document}

\begin{figure}
\captionsetup{singlelinecheck=off}
\begin{picture}(0,0)(-72,-75)
\put(12,-68){\small$\alpha_2$}
\put(-15,-71){\small$\alpha_1$}
\put(-50,-60){\small$-\alpha_1$}
\put(-47,-44){\small$2\alpha_1+\alpha_2$}
\put(-60,-10){\small$-2\alpha_1-\alpha_2$}
\end{picture}
\caption{Some roots, associated to the Cartan matrix $\left(\protect\begin{smallmatrix}2&-2\\-1&2\protect\end{smallmatrix}\right)$.}
\end{figure}

\end{document}