Curly braces and nested arrays

You have a typo in your inner array

change

     \begin{array}{1}
                   %%%

to

   \begin{array}{l}
                %%%

To give you another possibility using cases from amsmath or mathtools.

\documentclass{article}
\usepackage{mathtools}
\begin{document}
\begin{equation}
DTW(A,B)=\begin{cases}
    0 \quad \phantom{\infty}\text{if}\,\, n=0 \,\,\text{and}\,\, m=0  \\
    \infty \quad \phantom{0}  \text{if}\,\, n=0 \,\,\text{and}\,\, m=0 \\
    {d(H(A),H(B))+\min \begin{cases}
        a\\
        b\\
        c\\
    \end{cases}}
      \end{cases}
    \end{equation} 
  \end{document}