how to have a caption on top of longtable?

\begin{longtable}{|p{2cm}|p{3cm}|p{7cm}|p{3cm}|}
\caption{my caption}\\    %%%%<===
\hline

and you should put the \label after \caption, which makes more sense.

Another way: use head and first head:

\documentclass{article}
\usepackage{longtable}
\begin{document}

\begin{longtable}{|p{2cm}|p{3cm}|p{7cm}|p{3cm}|}
\caption{my caption}
\label{variability_impl_mech}
\endfirsthead
\endhead
\hline
1 & 2 & 3 & 4\\ 
%\hline 
\hline
1 & 2 & 3 & 4\\
\hline
1 & 2 & 3 & 4\\
\hline
.
.
.
1 & 2 & 3 & 4\\
\hline
\end{longtable}

\end{document}