Different labels for subsection titles and references

Redefine \p@subsection (the prefix using for the references to subsections):

\documentclass{report}

\renewcommand\thesection{\arabic{chapter}.\arabic{section}}
\renewcommand\thesubsection{\Roman{subsection}}
\makeatletter
\renewcommand\p@subsection{\thesection.}
\makeatother

\begin{document}

\chapter{Chapter}

\section{Section}

\subsection{Subsection}\label{subsection}

\noindent I have the subsections labelled in Roman in the title, because 1.1.I would just be long.

\noindent Here is a reference to the subsection: (\ref{subsection})

\end{document}

By the way, never use the combination \\ + blank line; this will generate underfull boxes warnings.

References can automatically be prefixed by \p@<counter>, see example:

\documentclass{report}
\usepackage{parskip}

\renewcommand\thesection{\arabic{chapter}.\arabic{section}}
\renewcommand\thesubsection{\Roman{subsection}}

\makeatletter
\renewcommand*{\p@subsection}{\thesection.}
\makeatother

\begin{document}
\chapter{Chapter}

Here is a reference to the subsection: \ref{subsection}

\section{Section}

\subsection{Subsection}\label{subsection}

\end{document}

Result: