\scalebox doesn't work inside a TikZ scope

If you add some non-TikZ commands like \scalebox into a tikzpicture you should use a \node for this. Otherwise you might get strange results as TikZ has special settings (i.e. nullfont) in the normal picture code area.

The following looks OK to me, you might want to adjust the positioning.

\documentclass[border=5cm]{standalone}

\usepackage{tikz}
\usetikzlibrary{positioning}
\usetikzlibrary{shapes}

\newcommand{\hexlattice}[1]{\begin{tikzpicture}[hexa/.style= {shape=regular polygon,regular polygon sides=6,minimum size=1cm, draw,inner sep=0,anchor=south,rotate=30}]
    \foreach \j in {0,...,#1}{%
        \foreach \i in {-1,...,#1}{%
            \node[hexa] (h\i;\j) at ({(\i-\j/2)*sin(60)},{\j*0.75}) {};} } 
    \end{tikzpicture}}

\begin{document}
    \begin{tikzpicture}[scale=1.1,every node/.style={minimum size=1cm},on grid]

    \begin{scope}[yshift=-120] 
        \draw[black, dashed, thin, xshift=30mm, yshift=0.5mm,xslant=-.8] (1,4) rectangle (7,6);
        \node [anchor=south west] at (0,4) {\scalebox{0.8}[0.4]{\hexlattice{5}}};
    \end{scope}

    \begin{scope}[yshift=0]
        \draw[black, dashed, thin, xshift=30mm, xslant=-.8] (1,4) rectangle (7,6);
        \node [anchor=south west] at (0,4) {\scalebox{0.8}[0.4]{\hexlattice{5}}};
    \end{scope} 
    \end{tikzpicture}
\end{document}

I would like to politely convince you not to nest tikzpicture environments without \saveboxes. I have seen this leading to uncontrollable results in way too many cases. If you really want to nest a tikzpicture, you can put it in a \savebox. However, in many cases, it is not at all needed to resort to the \savebox trick. In the case of your MWE, you could just use a pic. This allows you to control the color and the other parameters of the grid, which a \savebox does not allow you to access when you are using it with \usebox.

Other than that I use orthonormal projections (rather than slants) to arrive at

\documentclass[tikz,border=3.14mm]{standalone}
\usepackage{tikz-3dplot} 
\usetikzlibrary{shapes,3d}

\begin{document}
\tdplotsetmaincoords{70}{30}
\begin{tikzpicture}[tdplot_main_coords,scale=1.1,
    hexa/.style= {shape=regular polygon,regular polygon
sides=6,minimum size=1cm, draw,inner sep=0,anchor=south,rotate=30},
hexlattice/.pic={
\foreach \j in {0,...,#1}{%
        \foreach \i in {-1,...,#1}{%
            \node[hexa] (h\i;\j) at ({(\i-(1+pow(-1,\j))*1/4)*sin(60)},{\j*0.75}) {};} } 
}]

    \begin{scope}[canvas is xy plane at z=0,transform shape]
     \path[clip,postaction={draw,dashed}] (1,4) rectangle (7,6);
     \pic[scale=0.5] at (2,2) {hexlattice=12};
    \end{scope} 

    \begin{scope}[canvas is xy plane at z=4,transform shape]
     \path[clip,postaction={draw,dashed}] (1,4) rectangle (7,6);
     \pic[scale=0.5] at (2,2) {hexlattice=12};
    \end{scope} 
\end{tikzpicture}
\end{document}

Here you can change the view angle, and as you see the pics can be rescaled via scale=0.5, say, there is no need to resort to \scaleboxes.