How continue an equation next line

I suggest using the multlined environment from mathtools to break the radicand . You also can choose not to break it, but to type in medsize (~80 % of \displaystyle) with the \mediummath command from nccmath. I took the opportunity to clean your code:

\documentclass[11pt,a4paper,twocolumn]{article}
\usepackage[utf8]{inputenc}
\usepackage{mathtools, nccmath}

\begin{document}

\begin{equation*}
    \bigl| \dot{r} \bigr| =
    \sqrt{\begin{multlined}[b] -\!\bigl(e^{-t} ( \cos t + \sin t )\bigr)^2 + \\ \bigl(e^{-t} (\cos t + \sin t)\bigr)^2 + \bigl(-e^{-t}\bigr)^2 \end{multlined}}
\end{equation*}
\begin{align*}
    \bigl| \dot{r} \bigr| & =
    \sqrt{\medmath{ -\!\bigl(e^{-t} ( \cos t + \sin t )\bigr)^2 + \bigl(e^{-t} (\cos t + \sin t)\bigr)^2 + \bigl(-e^{-t}\bigr)^2}} \\
         & = \sqrt{ -\!\bigl(e^{-t} ( \cos t + \sin t )\bigr)^2 + \bigl(e^{-t} (\cos t + \sin t)\bigr)^2 + \bigl(-e^{-t}\bigr)^2}
\end{align*}

\end{document}

As the equation* environment doesn't permit line breaks, consider using a multline* environment. And, instead of creating a multi-line surd expression with a \sqrt instruction, I suggest you use (...)^{1/2} notation.

Two additional comments about your code: Do try to use \left and \right less frequently, and don't needlessly encase various terms in curly braces.

\documentclass[11pt,a4paper,twocolumn]{article}
\usepackage{amsmath} % for 'multline*' env.
\begin{document}
...
\hrule % just to illustrate column width
\begin{multline*}
    | \dot{r} | = \bigl\{
    - e^{-t} ( \cos t + \sin t )^{2} \\
    + e^{-t} ( \cos t + \sin t )^{2}
    + (-e^{-t})^{2} \bigr\}^{1/2}
\end{multline*}
\end{document}

If you are interested in a square root expression and not a power notation that Mico's answer provides, have a look at the following code:

\begin{equation*}
    \left| \dot{r} \right| =
    \sqrt{
        \begin{aligned}
        &{- \, {e}^{-t} \left( \cos t + \sin t \right)}^{2} \, + \, \\ 
        &{{e}^{-t}  \left( \cos t + \sin t \right)}^{2} \, + \, {(- \, {e}^{-t})}^{2}
        \end{aligned}   
        }
\end{equation*}

Ouptut