Caption on side by side matrices

Not entirely sure whether this satisfies all your constraints (since I'm not putting the sub-captions in the LoF):

\documentclass{article}
\begin{document}
\begin{figure}
  \begin{minipage}{.5\linewidth}
    \centering
    \[A=\left[\begin{array}{cc}
      a & b \\
      c & d
    \end{array}\right]\]
    Short caption for matrix~$A$
  \end{minipage}%
  \begin{minipage}{.5\linewidth}
    \centering
    \[A'=\left[\begin{array}{cc}
      e & f \\
      g & h
    \end{array}\right]\]
    Short caption for matrix~$A'$
  \end{minipage}
  \caption{A caption to the entire figure}
\end{figure}
\end{document}

The idea is to place two minipages of .5\linewidth inside a figure. Yes, the figure environment can contain anything inside of it, even mathematics, even a tabular, even an image. It merely acts as a place holder that floats around.

Within each minipage the contents can be supplied in whichever way you like. As I've done in the above example, a display equation is used, together with regular centred text below. You can modify the "short captions" to be \small or \footnotesize, if needed.

If the requirement is to have the "short captions" form part of the LoF, some more work is required.

In the following MWE you will find a solution for your problem using package subfig:

%http://tex.stackexchange.com/questions/84021/caption-on-side-by-side-matrices
\documentclass{article}

\usepackage{subfig}
\usepackage{amsmath}
\usepackage{array}

\begin{document}

\listoffigures

\section{My Work}

\begin{figure}[htb]
  \centering
  \subfloat[Image A][The Image A%
    \label{subfig:imageA}]{%
    \begin{math}
    \left[\begin{array}{cc}
    a & b \\
    c & d
    \end{array}\right]
    \end{math}
    }% 
   \qquad   % Distance Image A and B 
  \subfloat[Image B][The Image B%
    \label{subfig:ImageB}]{% 
    \begin{math}
    \left[\begin{array}{cc}
    e & f \\
    g & h
    \end{array}\right]
    \end{math}
  } 
  \caption{Image with two subfigures}
  \label{fig:ImageAB}
\end{figure} 

In figure~\ref{fig:ImageAB} you see two subfigure~\ref{subfig:imageA} and \ref{subfig:ImageB}.

\end{document}

With \subfloat you define a sub figure, \qquad is the distance between the two sub figures. You can reference the complete figure or both of the sub figures (see last sentence in MWE). A list of figures shows your figure as wanted.