GNU sort stable sort when sort does not know sort order

You could use awk to start a new sort for each block:

% awk -v cmd="sort -k2,2" '$1 != prev {close(cmd); prev=$1} {print | cmd}' foo
C 1
C 2
A 1
A 2
B 1
B 2

• $1 != prev {close(cmd); prev=$1} - when the saved value is different, we have a new block, so we close any previously started sort
• {print | "sort -k2,2"}' pipes the output to sort, starting it if it isn't already running (awk can keep track of commands it starts)

You could use a Schwartzian transform (this is basically the decorate-sort-undecorate approach you alluded to in a comment, but likely more performant than muru's fine answer due to using a single sort invocation as opposed to multiple) - using awk add a prefix column that increments with a change in value in the first column, sort by the prefix column followed by the "second" column(whose ordinal position has temporarily shifted to 3 due to the presence of the prefix column), and finally get rid of the prefix column

awk '{print ($1 in a? c+0: ++c)"\t" $0; a[$1]}' file | sort -k1,1n  -k3,3 | cut -f 2-