Topological ground state degeneracy of SU(N), SO(N), Sp(N) Chern-Simons theory
After stating the solution, I'll try to give some physical insights to the best of my knowledge and some more references.
The dimension of the required state space is given by the Verlinde formula, having the following form for a general compact semisimple Lie group $G$ on a Riemann surface with genus $g$ corresponding to the level $k$:
$$ \mathrm{dim} V_{g,k} =(C (k+h)^r)^{g-1} \sum_{\lambda \in \Lambda_k}\prod_{\alpha \in \Delta}(1-e^{i\frac{\alpha .(\lambda+\rho)}{k+h}})^{(1-g)}$$
(Please see Blau and Thompson equation 1.2.). Here, $C$ is the order of the center, $h$ is dual Coxeter invariant, $\rho$ is half the sum of the positive roots, and $r$ is the rank of $G$. $g$ is the genus, $\Delta$ is the set of roots and $\Lambda_k$ is the set of integrable highest weights of the Kac-Moody algebra $G_k$.
For the torus ($g=1$), this formula simplifies to:
$$ \mathrm{dim} V_{\mathrm{Torus},k} = \# \Lambda_k $$
i.e., the dimension is equal to the number of integrable highest weights of the Kac-Moody algebra $G_k$.
The integrable highest weights of a level-$k$ Kac-Moody algebra are given by the following constraints:
$$ \lambda - \mathrm{dominant}, 0 \leq \sum_{i=1}^r \frac{2 \lambda. \alpha^{(i)}}{ \alpha^{(i)}. \alpha^{(i)}}\leq k$$
Where $ \alpha^{(i)}$ are the simple roots, please see, for example, the following review by Fuchs on Kac-Moody algebras.
(My favorite reference for the representation theory of Kac-Moody algebras is the Goddard and Olive review which seems not available on line)
For example for $SU(3)_k$ whose dominant weights are $2$-tuples of nonnegative numbers $(n_1, n_2)$, the above condition reduces to:
$$\mathrm{dim} V^{SU(3)}_{\mathrm{Torus},k} = \# (n_1\geq 0, n_2\geq 0, 0\leq n_1 + n_2 \leq k )= \frac{(k+1)(k+2)}{2}$$
To perform the computations for the more general cases, one can use the seminal review by Slansky.
The Verlinde formula was discovered before the Chern-Simons theory came into the world. Originally it is the dimension of the space of conformal blocks for the WZW model. This formula has been derived in a large variety of ways, please, see footnote 26 in the Fuchs review. It is still an active research topic, please see for example a new derivation in this recent article by Gukov.
The Chern-Simons theory may be the most sophisticated example in which the Dirac quantization postulates can be carried out in spirit. (More precisely their generalization in geometric quantization). I mean starting from a phase space and utilizing a specified set of rules to associate a Hilbert space to it. In the case of the Chern-Simons theory, the phase space is the set of solutions of the classical equations of motion. The classical equations of motion require the field strength to vanish, in other words the connection to be flat.
This phase space (the moduli space of flat connections) is finite dimensional, it has a Kähler structure and it can geometrically quantized as a Kähler manifold, just like the case of the harmonic oscillator. Thus the problem can be reduced in principle to a problem in quantum mechanics.
The case of the torus is the easiest because everything can be carried out explicitly in the Abelian and the non-Abelian case, please see the following explicit construction by Bos and Nair, (a more concise treatment appears in Dunne's review).
In the case of the torus, the moduli space of flat connections in the Abelian case is also a torus and in the non-Abelian case it is:
$$\mathcal{M} = \frac{T \times T}{W}$$
where $T$ is the maximal torus of $G$. Basically, a Fock quantization can be carried away, but there is a further restriction on the admissible wave functions coming from the invariance requirement under the large gauge transformations (please see for example, the Dunne's review ). The invariant wave functions are called non-Abelian theta functions and they are just in a one to one correspondence with the Kac-Moody algebra integrable highest weights. (In the Abelian case, the wave functions are the Jacobi theta functions).
In the higher genus case, although the quantization program leading to the Verlinde formula can be carried out in principle, few explicit results are known, please see the following article by Lisa Jeffrey (and also the following lecture notes). The dimension of these moduli spaces is known. In addition. Witten in an ingenious work computed their symplectic volumes and their cohomology ring in some cases.
Witten's idea is that as in the case of a simple spin, the dimension of the Hilbert space in the semiclassical limit ($k \rightarrow \infty$) becomes proportional to the volume and the leading exponent of $k$ is the complex dimension of the moduli space (please observe for example, that in the case of $SU(3)$ on the torus, the leading exponent is $2$ which is the rank of $SU(3)$ which is the dimension of the maximal torus $T$).