How can I make a "vertical dotfill" environment?

Yes, do not reinvent the wheel:

\documentclass{article}
\usepackage{lipsum}
\usepackage[most]{tcolorbox}
\newtcolorbox{dottedenvironment}{enhanced,
boxrule=0pt,boxsep=1em,colback=white, arc=3mm, frame hidden,
borderline={1pt}{2pt}{red!30!black,dotted}}
\begin{document}

\begin{dottedenvironment}
\lipsum[2]  
\end{dottedenvironment}

\end{document}

Edit: As stated in the comments, It seems a requeriment that boxes should be breakable even nested. I do not know an automatic approach and I think that is not a good idea (*), but manually you can break a nested tcolorbox

(*) Note that enforce breakable option is a bit insane.

\documentclass{article}
\usepackage[paperwidth=8cm,paperheight=8cm]{geometry}
\usepackage{ltablex}
\usepackage{lipsum}
\usepackage[most]{tcolorbox}
\newtcolorbox{dottedenvironment}{enhanced,enforce breakable, boxrule=0pt,boxsep=1em,colback=white, arc=3mm, frame hidden,
borderline={1pt}{2pt}{red!30!black,dotted}}
\def\dummytext{The key message for success is to keep it simple. }
\def\moretext{\dummytext\dummytext\par}
\begin{document}
\begin{dottedenvironment}
\moretext
\begin{dottedenvironment}
\dummytext
\tcbbreak
\moretext\end{dottedenvironment}
\tcbbreak
\end{dottedenvironment}
\end{document}

ConTeXt can do this easily using textbackground. You can also include floats (which will not be surrounded by dots because they could appear elsewhere).

\startuseMPgraphic{dottedframe}
begingroup;
  for i=1 upto nofmultipars :
    draw ( llcorner multipars[i]
      -- lrcorner multipars[i]
      -- urcorner multipars[i]
      -- ulcorner multipars[i]
      -- cycle )
      enlarged (EmWidth,EmWidth)
      dashed withdots ;
  endfor ;
endgroup;
\stopuseMPgraphic

\definetextbackground
  [DottedFrame]
  [mp=dottedframe,
   location=paragraph,
   before={\blank[2*big]}]

\starttext

\starttextbackground[DottedFrame]
Text and/or environments and/or equations

Text and/or environments and/or equations

Text and/or environments and/or equations

\startplacefigure[location=here,title={The famous Dutch cow}]
  \externalfigure[cow]
\stopplacefigure

Text and/or environments and/or equations

Text and/or environments and/or equations

Text and/or environments and/or equations
\stoptextbackground

\stoptext