Remove last dot in title numbering

Here's how you can obtain the output you're after in the KOMA-script classes:

\documentclass[appendixprefix=true]{scrreprt}

\usepackage{etoolbox}
\makeatletter
\g@addto@macro{\appendix}{%
  \patchcmd{\@@makechapterhead}% <cmd>
    {\endgraf\nobreak\vskip.5\baselineskip}% <search>
    {\hspace*{-.5em}:\space}% <replace>
    {}{}% <success><failure>
  \patchcmd{\@chapter}% <cmd>
    {\addchaptertocentry{\thechapter}}% <search>
    {\addchaptertocentry{Appendix~\thechapter:}}% <replace>
    {}{}% <success><failure>
  \addtocontents{toc}{%
    \protect\patchcmd{\protect\l@chapter}% <cmd>
      {1.5em}% <search>
      {6.5em}% <replace>
      {}{}}% <success><failure>
}
\renewcommand{\autodot}{}% Remove all end-of-counter dots
\makeatother

\begin{document}

\tableofcontents

\chapter{First chapter}
\section{First section}
\subsection{First subsection}

\appendix
\chapter{Last chapter}

\end{document}

The dots are removed through a redefinition of \autodot, while the Appendix formatting is done via a \patchcmd of \@@makechapterhead when you call \appendix. The final patch adjusts the width of the \numberline box from 1.5em to 6.5em. This adjustment is specific to the chapter-related entry in the ToC as it deals with \l@chapter.

etoolbox provides the patching capability.

Since KOMA-Script version 3.20 you can use \DeclareTOCStyleEntry. So here is a new suggestion:

\documentclass[
  %appendixprefix=true,%<- removed
  11pt,a4paper,
  numbers=noenddot% <- added
]{scrreprt}[2016/05/10]% needs at least version 3.20

\DeclareTOCStyleEntry[
  level=\chaptertocdepth,
  indent=0pt,
  numwidth=2.3em,
  dynnumwidth,
  linefill=\hfill,
  entryformat=\appendixtocformat,
  entrynumberformat=\appendixtocnumberformat,
  pagenumberformat=\appendixtocpagenumberformat
]{tocline}{appendixchapter}
\newcommand*\appendixtocformat[1]{{\usekomafont{chapterentry}#1}}
\newcommand*\appendixtocnumberformat[1]{{\def\autodot{:}\appendixname\ #1}}
\newcommand*\appendixtocpagenumberformat[1]
  {{\usekomafont{chapterentry}\usekomafont{chapterentrypagenumber}#1}}

\usepackage{xpatch}
\xapptocmd\appendix
  {%
    \renewcommand*{\chapterformat}{%
      \mbox{\appendixname{\nobreakspace}\thechapter:%
        \IfUsePrefixLine{}{\enskip}}%
    }%
    \renewcommand*{\chaptermarkformat}{\appendixname\ \thechapter:\enskip}%
    \xpatchcmd{\addchaptertocentry}
      {\addtocentrydefault{chapter}{#1}{#2}}
      {\addtocentrydefault{appendixchapter}{#1}{#2}}%
      {}{\PatchFailed}%
  }{}{\PatchFailed}

\begin{document}
\tableofcontents
\chapter{First chapter}
\section{First section}
\subsection{First subsection}
\appendix
\chapter{Last chapter}
\end{document}

Run three times to get:

Here you are

\documentclass{book}
\begin{document}
\makeatletter
\def\@makechapterhead#1{\vspace*{50\p@}{\parindent\z@\raggedright\normalfont\ifnum\c@secnumdepth>\m@ne\if@mainmatter\huge\bfseries\@chapapp\space\thechapter: \fi\fi\interlinepenalty\@M\Huge\bfseries#1\par\nobreak\vskip40\p@}}
\makeatother
\chapter{CHAPI}
\section{SECI}
\subsection{SUBSECTI}
\appendix
\chapter{CHAPII}
\section{SECII}
\subsection{SUBSECTII}
\end{document}