How to "simplify" the syntax of a particular coordinate system in TikZ?

If you have always three (or less) coordinates, there is no problem. TikZ has this out of the box. You only need

\begin{scope}[x={(a)},y={(b)},z={(c)}]
  <in this scope your syntax works>
\end{scope}

Full MWE:

\documentclass[tikz]{standalone}
\begin{document}
\begin{tikzpicture}
  \fill (1,4) circle (1pt) coordinate (a) node[above] {$A$}
        (0,0) circle (1pt) coordinate (b) node[below] {$B$}
        (5,0) circle (1pt) coordinate (c) node[below] {$C$};
  \draw (a) -- (b) -- (c) -- cycle;
  \fill[red] (barycentric cs:a=1/3,b=1/3,c=1/3) circle (1pt) node[below] {$G$};
  \begin{scope}[x={(a)},y={(b)},z={(c)}]
  \draw[blue] (1/3,1/3,1/3) circle[radius=2pt];
  \end{scope}
\end{tikzpicture}
\end{document}

This is, however, not a true barycentric coordinate system because it does not do the normalization. In your example where the components add up to one this is fine, though.

If you want to use something that resembles your syntax, allows for an arbitrary number of coordinates and normalizes the coordinates, then consider this:

\documentclass[tikz]{standalone}
\makeatletter
\tikzdeclarecoordinatesystem{b}
{%
  {%
    \pgf@xa=0pt% point
    \pgf@ya=0pt%
    \pgf@xb=0pt% sum
    \pgfutil@tempcnta=0%
    \tikz@b@dolist#1,,%
    \pgfmathparse{1/\the\pgf@xb}%
    \global\pgf@x=\pgfmathresult\pgf@xa%
    \global\pgf@y=\pgfmathresult\pgf@ya%
  }%
}%

\def\tikz@b@dolist#1,{%
  \def\tikz@temp{#1}%
  \ifx\tikz@temp\pgfutil@empty%
  \else
    \pgf@process{\pgfpointanchor{\csname tikz@bcoord@\the\pgfutil@tempcnta\endcsname}{center}}%
    \pgfmathparse{#1}%
    \advance\pgf@xa by\pgfmathresult\pgf@x%
    \advance\pgf@ya by\pgfmathresult\pgf@y%
    \advance\pgf@xb by\pgfmathresult pt%
    \advance\pgfutil@tempcnta1%
    \expandafter\tikz@b@dolist%
  \fi%
}%
\tikzset{barycentric coordinates/.code={
    \pgfutil@tempcnta=0%
    \tikz@bc@dolist#1,,%
}}
\def\tikz@bc@dolist#1,{%
  \def\tikz@temp{#1}%
  \ifx\tikz@temp\pgfutil@empty%
  \else
    \expandafter\edef\csname tikz@bcoord@\the\pgfutil@tempcnta\endcsname{#1}%
    \advance\pgfutil@tempcnta1%
    \expandafter\tikz@bc@dolist%
  \fi%
}%
\makeatother
\begin{document}
\begin{tikzpicture}
  \fill (1,4) circle (1pt) coordinate (a) node[above] {$A$}
        (0,0) circle (1pt) coordinate (b) node[below] {$B$}
        (5,0) circle (1pt) coordinate (c) node[below] {$C$}
        (4,3) circle (1pt) coordinate (d) node[below] {$D$}     ;
  \draw (a) -- (b) -- (c) -- (d) -- cycle;
  \fill[red] (barycentric cs:a=1/3,b=1/3,c=1/3) circle (1pt) node[below] {$G$};
  \begin{scope}[barycentric coordinates={a,b,c}]
  \draw (b cs:1,1,1) circle[radius=2pt];
  \end{scope}
  \fill[red] (barycentric cs:a=1/4,b=1/4,c=1/4,d=1/4) circle (1pt) node[below] {$H$};
  \begin{scope}[barycentric coordinates={a,b,c,d}]
  \draw (b cs:1,1,1,1) circle[radius=2pt];
  \end{scope}
\end{tikzpicture}
\end{document}

You still have to write b cs: but other than it is the syntax you are proposing. (One could eliminate the need to add b cs:, see here, but this will make this code even more hacky, and this will AFAIK only work for 2 or 3 coordinates anyway.)

Since this is not infinitely well known, let me also mention ternary axes that can be loaded with \usepgfplotslibrary{ternary} in pgfplots. They, too, parse coordinates in the way you want.

I prefer the notation A=1/3,B=1/3,C=1/3 [More concise, more readable] but it's a matter of taste. Here how I proceed with tkz-euclide

\documentclass[border=.25cm]{standalone}
\usepackage{tkz-euclide}

\begin{document}
\begin{tikzpicture}
\tkzDefPoints{1/4/A,0/0/B,5/0/C,4/3/D}
\tkzDefBarycentricPoint(A=1/3,B=1/3,C=1/3) \tkzGetPoint{G}
\tkzDefBarycentricPoint(A=1/4,B=1/4,C=1/4,D=1/4) \tkzGetPoint{H}

\tkzDrawPolygon(A,...,D)
\tkzDrawPoints[red](G,H)
\tkzDrawPoints(A,B,C,D)

\tkzLabelPoints(A,B,C,D,G,H)
\end{tikzpicture}
\end{document}

 % why not
 %  \tkzDefBarycentricPoint(A=1,B=1,C=1) \tkzGetPoint{G}
 %  \tkzDefBarycentricPoint(A=1,B=1,C=1,D=1) \tkzGetPoint{H}