Drawing absolute value bars around a fraction with different height of numerator and denominator

Here, I use \stretchleftright{|}{...}{|} to achieve it.

\documentclass{article} 
\usepackage{scalerel,amsmath}
\begin{document} 
\[
\cos\phi=\stretchleftright{|}{\frac{
        \begin{pmatrix}1\\1\\1\end{pmatrix}
    \circ
        \begin{pmatrix}1\\2\\3\end{pmatrix}
    }{
        \sqrt3\cdot\sqrt{1^2+2^2+3^2}
    }}{|} 
\]
\end{document}

I'd like to suggest a solution that's different from either of your preferred choices.

\documentclass{article}
\usepackage{mathtools} % for '\DeclarePairedDelimiter' macro
\DeclarePairedDelimiter{\abs}{\lvert}{\rvert}

\begin{document}
\[
\cos\phi= \frac{1}{\sqrt3 \cdot\sqrt{1^2+2^2+3^2}}
          \abs*{\begin{pmatrix}1\\1\\1\end{pmatrix}\circ
                \begin{pmatrix}1\\2\\3\end{pmatrix}}  
\]
\end{document}