Loop counter not interpreted as number

Yes, your impression is right. One way to enforce evaluation is to use \numexpr.

\documentclass{article}
\usepackage{tikz}
\usetikzlibrary{positioning}

\begin{document}

\begin{tikzpicture}[
    node distance=0pt, 
    box/.style={outer sep=0pt, draw, thick, minimum height=0.6cm}]

    \node[box](0) {0};

    \foreach \i in {0,..., 31} {                
        \node[box, right=of \i] (\the\numexpr\i + 1\relax)
        {\the\numexpr\i+1\relax};    
    }

    \end{tikzpicture}
\end{document}

The less hacky way, though, is to use evaluate.

\documentclass[tikz,border=3.14mm]{standalone}
\usetikzlibrary{positioning}

\begin{document}

\begin{tikzpicture}[
    node distance=0pt, 
    box/.style={outer sep=0pt, draw, thick, minimum height=0.6cm}]

    \node[box](0) {0};

    \foreach \i [evaluate=\i as \nexti using {int(\i+1)}] in {0,..., 31} {                
        \node[box, right=of \i] (\nexti)
        {\nexti};    
    }

    \end{tikzpicture}
\end{document}

Here is a pic version thereof.

\documentclass[tikz,border=3.14mm]{standalone}
\usetikzlibrary{positioning}

\begin{document}

\begin{tikzpicture}[
    node distance=0pt, 
    box/.style={outer sep=0pt, draw, thick, minimum height=0.6cm},
    pics/boxrow/.style 2 args={code={
    \node[box] (#1){#1};
    \foreach \i [evaluate=\i as \nexti using {int(\i+1)}] in
    {#1,\the\numexpr#1+1,...,#2} {                
        \node[box, right=of \i] (\nexti)     {\nexti};    
    }
    }}]
 \path (0,0) pic{boxrow={0}{30}}
 (0,-1) pic{boxrow={5}{18}};
\end{tikzpicture}
\end{document}

Even another solution:

\documentclass[tikz, border=2mm]{standalone}
\usetikzlibrary{positioning}

\begin{document}

\begin{tikzpicture}[
    node distance=0pt, 
    box/.style={outer sep=0pt, draw, thick, minimum height=0.6cm}]

    \node[box](0) {0};

    \foreach \i [remember=\i as \lasti (initially 0), count=\ni] in {1, ..., 31} {                
        \node[box, right=of \lasti] (\ni) {\ni};    
    }

    \end{tikzpicture}
\end{document}

A variation of Ignasi's answer where you can easily change upper and lower bound of the \foreach-range.

\documentclass[tikz, border=2mm]{standalone}
\usetikzlibrary{positioning}

\begin{document}

\begin{tikzpicture}[
  node distance=0pt, 
  box/.style={outer sep=0pt, draw, thick, minimum height=0.6cm}
]
  \foreach \i [remember=\i as \lasti (initially a)] in {0, ..., 31} {
    \if a\lasti
      \node[box](\i) {\i};
    \else
      \node[box, right=of \lasti](\i) {\i};
    \fi
  }
\end{tikzpicture}
\end{document}