How can I detect when I'm at the top of column?

Is it what you'd like to get?

\documentclass{article}
\usepackage[latin]{babel}% for avoiding too many underfull boxes
\usepackage{multicol}
\usepackage{lipsum}

\newenvironment{segmented}
 {%
  \noindent\begin{minipage}{\linewidth}
 }
 {%
  \end{minipage}\par
  \bigskip
  \cleaders\vbox{\hrule width\linewidth}\vskip0.4pt
  \bigskip
 }

\begin{document}

\begin{multicols}{2}
\begin{segmented}
\lipsum[1]  
\end{segmented}
\begin{segmented}
\lipsum[2]  
\end{segmented}
\begin{segmented}
\lipsum[3]  
\end{segmented}
\end{multicols}

\end{document}

There's probably a better way to do this, but @Werner's link gave me this idea. It relies on the colaction feature of multicol:

\documentclass{article}
\usepackage[colaction]{multicol}
\usepackage{etoolbox}
\usepackage{lipsum}

\numdef{\columnnumber}{0}
\newcommand{\conditionalrule}[1]{
    \ifnumequal{#1}{\columnnumber}{\hrule}{}
    \global\numdef{\columnnumber}{#1}
    }

\newenvironment{segmented}{%
\noindent\begin{minipage}{\linewidth}
    \docolaction{\conditionalrule{1}}{}{\conditionalrule{2}}\bigskip
}{\bigskip
\end{minipage}}

\begin{document}
    \begin{multicols}{2}
        \begin{segmented}
            \lipsum[1]  
        \end{segmented}
        \begin{segmented}
            \lipsum[2]  
        \end{segmented}
        \begin{segmented}
            \lipsum[3]  
        \end{segmented}
    \end{multicols}
\end{document}

If anyone has a more elegant/efficient solution, I'd be happy to accept their answer, so I'm holding off on accepting my own answer for a little while.

You can let TeX determine the position (which works within or outside multicol):

\documentclass[]{article}
\usepackage{multicol,lipsum}

\usepackage{xparse}

\ExplSyntaxOn

\dim_new:N \l__condline_dim
\box_new:N \l__condline_box

% #1 skip produced
% #2 vertical correction of material
% #3 material 
% #4 break penalty after (if forbidden the only place to break is before

\cs_new:Npn \condline:nn #1#2#3#4 
  {
    \par
    \hbox_set_to_wd:Nnn \l__condline_box \columnwidth {\hfil #3 \hfil}
%
    \dim_set:Nn \l__condline_dim
        { \box_ht:N \l__condline_box +
          \box_dp:N \l__condline_box }
%
    \skip_vertical:n { #1 / 2 - \prevdepth + #2 - \l__condline_dim / 2 }
    \nobreak    
%
    \cleaders \copy \l__condline_box \vskip \l__condline_dim
    \nobreak    
%
    \skip_vertical:n { #1 / 2 - \parskip  - #2 - \l__condline_dim / 2 }
    \penalty #4 \relax 
%
    \dim_set:Nn \prevdepth { 0pt }
  }

% document level command
%
% \condline * < correction > [ skip ] { material }
%
% all arguments optional

\DeclareDocumentCommand\condline 
  { s  D<>{3pt} O{\baselineskip} G{$*\quad*\quad*$} }
  { \IfBooleanTF #1
      { \condline:nn {#3} {#2} {#4} {10000} }
      { \condline:nn {#3} {#2} {#4} {0} }
  }

\ExplSyntaxOff

\begin{document}
\begin{multicols}{3}
  \lipsum[1]
  \condline{...........}
  line1\\  line2\\  line3\\  line4\\ line5\\  
  add two extra lines
  and the stars below will vanish!\\
 % line9\\  line10\\
  LAST LAST LAST
  \condline{$* \quad * \quad * $}
  \lipsum[2]
\end{multicols}

  \condline<-5pt>{\hrulefill}

\begin{multicols}{2}
  \lipsum[3]
  \condline{\hrulefill}
  \lipsum[4]
  \condline[2\baselineskip]{---With star this line would vanish }
  \lipsum[5]
  \condline{\dotfill}
  \lipsum[6]
\end{multicols}

\end{document} 

(the code is a bit of a mixture of expl3 + ordinary LaTeX as expl3 doesn't yet have all tools for text processing (like handling penalties))

The result is:

And if we add 2 lines in the first multicol and use the star form of \condline in the second one we will get:

The right bottom column now appears to be too short but that is understandable as now the only break is before the \condline and the line vanishes if the break is taken there (just like the line with asterisks in the first multicol vanished at the break.