Put a dot above a bold symbol in an equation?

It is $\dot{\mathbf{x}}$. The syntax is \dot{<symbol>}. You are missing the braces.

\documentclass{article}
\begin{document}
  $\dot{\mathbf{x}} \dot{x}$
\end{document}

Actually, you only get this error if you use the amsmath package - and that is why it is always a good idea to give a complete "minimum working example" of your problem, for example

\documentclass{article}
\usepackage{amsmath}
\begin{document}
$\dot \mathbf x$ % doesn't work
\end{document}

Even with amsmath, $\dot x$ works correctly without braces. But the usual cause of weird error messages like \select@group has an extra }, which is what I got from $\dot \mathbf x$ with amsmath, is that something (in this case \dot) is effectively being applied just to the next token in the input, and $\dot \mathbf$ doesn't make any sense on its own.

Note also that $\dot{\mathbf x}$ and $\mathbf{\dot x}$ are not the same. The second one gives a bold-face dot over the bold-face x, unlike the first one.