How can make a protocol in latex?

Here is a starting point - setting the protocol in a tabularx, wrapped in a protocol environment:

\documentclass{article}

\usepackage{tabularx}

\newcounter{protocol}
\newenvironment{protocol}[1]
  {\par\addvspace{\topsep}
   \noindent
   \tabularx{\linewidth}{@{} X @{}}
    \hline
    \refstepcounter{protocol}\textbf{Protocol \theprotocol} #1 \\
    \hline}
  { \\
    \hline
   \endtabularx
   \par\addvspace{\topsep}}

\newcommand{\sbline}{\\[.5\normalbaselineskip]}% small blank line

\begin{document}

\begin{protocol}{Secure Multi-Party Shuffling Scheme}
\textit{Inputs.} For all $i \in [n]$, party~$P_i$ holds an input~$x_i$. 
  Let~$C$ denote the probabilistic sorting network of~[LP90] and~$d$ denote its depth.
\sbline
\textit{Goal.} Parties jointly compute a random shuffle of their inputs.
\sbline
\textit{The protocol:}
\begin{enumerate}
  \item \textbf{Setup.}
  \begin{enumerate}
    \item
    Parties run \textsf{Build-Quorums} to agree on $n$ quorums $Q_1, \ldots, Q_n$.

    \item
    Parties in $Q_1$ run \textsf{Gen-Rand} and \textsf{VSS-Reconst} repeatedly to generate 
    a sequence~$R$ of $\Theta(\log^2 n)$ random bits.

    \item
    Parties in $Q_1$ send $R$ to all other quorums in the following way. For all $i \in \{2, \ldots, n\}$, parties
    in~$Q_i$ receive~$R$ from $Q_{\lfloor i/2 \rfloor}$, and then send it to all parties in~$Q_{2i}$ and $Q_{2i+1}$.

    \item
    For all $i \in [n]$ and $j \in [m]$, parties assign~$Q_i$ to~$P_i$ and $Q_{(j \textrm{mod} n)}$
    to the $j$-th gate of~$C$, and connect the gates based on the random butterfly tournament described in~[LP90] and the
    random sequence~$R$.
  \end{enumerate}
\end{enumerate}

\end{protocol}

\end{document}

Source: Secure Multi-Party Shuffling