Rotation Matrix of rotation around a point other than the origin

Your first formula is correct. Remember, the point to which this is applied appears on the RIGHT: $$ T(x,y) * R * T(-x,-y) (P) $$ So to evaluate the expression above, we first translate $P$ by $(-x, -y)$, then rotate the result, then translate back. Let's see what happens when $P$ is the point $(x, y, 1)$. That amounts to evaluating the following product: \begin{align} f((x, y)) &= \begin{bmatrix}1&0&x\\ 0& 1&y\\0&0&1\end{bmatrix} \begin{bmatrix}\cos(\theta) & -\sin(\theta) & 0\\\sin(\theta) & \cos(\theta) & 0 \\ 0&0&1\end{bmatrix} \begin{bmatrix}1&0&-x\\ 0& 1&-y\\0&0&1\end{bmatrix} \begin{bmatrix}x\\ y\\1\end{bmatrix}\\ &= \begin{bmatrix}1&0&x\\ 0& 1&y\\0&0&1\end{bmatrix} \begin{bmatrix}\cos(\theta) & -\sin(\theta) & 0\\\sin(\theta) & \cos(\theta) & 0 \\ 0&0&1\end{bmatrix} \begin{bmatrix}0\\ 0\\1\end{bmatrix}\\ &= \begin{bmatrix}1&0&x\\ 0& 1&y\\0&0&1\end{bmatrix} \begin{bmatrix}0\\ 0\\1\end{bmatrix}\\ &= \begin{bmatrix}x\\ y\\1\end{bmatrix}\\ \end{align} as expected: the point $(x, y)$ remains fixed by this composite transformation.

The point is, that you're shifting the coordinate system, not the point.
So you don't actually shift the point to the origin, you shift the origin to the point, and then back.

These matrices are left-side multiplicated with vector positions, so the order of multiplication is from right to left - on the right side is the first operation, on the left side - the last one.