Using section sign (§) for cross-references to sections

You could use the cleveref package and redefine \crefname{section}:

\documentclass{article}

\usepackage[utf8]{inputenc}

\usepackage{cleveref}
\crefname{section}{§}{§§}
\Crefname{section}{§}{§§}

\begin{document}

\section{foo}\label{sec:foo}

Some text.

\section{bar}

As explained in \cref{sec:foo}~\dots

\end{document}

hyperref provides \autoref{<label>} that checks the counter used in the reference and sets a label with a prepended \<counter>autorefname. Here's a small example:

\documentclass{article}
\usepackage{hyperref}% http://ctan.org/pkg/hyperref
\renewcommand{\sectionautorefname}{\S}
\begin{document}
\section{A section}
See~\autoref{another-section}.
\section{Another section}\label{another-section}
\end{document}​

Often forgotten is the macro \p@<counter>. If LaTeX generates a reference value, this is not just \the<counter> but \p@<counter>\the<counter>. If a new counter is defined, \p@<counter> is defined empty. But it can be redefined to add a prefix, for example. Thus there is no need for additional packages.

\documentclass{article}

\makeatletter
\renewcommand*{\p@section}{\S\,}
\renewcommand*{\p@subsection}{\S\,}
\makeatother

\begin{document}

\section{Hello World}
\label{sec:hello}
\subsection{Subsection A}
\subsection{Subsection B}
\subsection{Subsection C}
\label{sec:C}

See \ref{sec:C} inside \ref{sec:hello}.
\end{document}