Conjugate Representations

Any hermitian inner product on a complex vector space $V$ gives an isomorphism between $\bar V$ and $V^*$. Furthermore, if $V$ has a hermitian inner product that is preserved by the representation of the Lie group/algebra, then this gives an isomorphism of representations between $\bar V$ and $V^*$. Thus in such cases you are asking about representations being self-dual (which is equivalent to the representation having an invariant bilinear form).

For semi-simple Lie groups, every representation has an invariant hermitian inner product (the representation comes from the simply connected compact real form and you can average over this group to get an invariant hermitian inner product-- see Weyl's unitarian trick).

You can determine if a representation of a semi-simple Lie algebra is self-dual by using weights. If $\mu$ is the highest weight of $V$ then the highest weight of $V^*$ is $-\omega_0 \mu$, where $\omega_0$ is the unique element of the Weyl group mapping the positive Weyl chamber to its negative. The element $\omega_0$ is $-1$ for the groups $SU(2)$, $SO(2n+1)$, $Sp(n)$, and $SO(4n)$. So for all of these groups $V \simeq V^* (\simeq \bar V)$. A reference for this is section 6.4 (and it's exercises) of Sepanski's Compact Lie groups.


Allow me to stick with Lie algebras. As I write in the comment to the other answer, we can agree that in the compact (semisimple) case, complex conjugation acts on the weight lattice via $-w_0$; and since that's also how duality acts on it, and since every irrep is determined by its highest weight, we have

for a complex representation $\rho: \mathfrak{g} \rightarrow End_{\mathbb C}(\mathbb C^n)$ of a compact semisimple real Lie algebra $\mathfrak{g}$, with complexified representation $\rho_{\mathbb C}: \mathfrak{g} \rightarrow End_{\mathbb C}(\mathbb C^n)$, the conjugate of $\rho$ is the restriction to $\mathfrak{g}$ of the dual of $\rho_\mathbb C$: $$\bar \rho = \rho_{\mathbb C \vert \mathfrak{g}}^*$$

In particular, as the other answer states, we have that all representations are self-conjugate for the compact real forms of types $A_1, B_n, C_n, D_{2n}$ (i.e. $\mathfrak{su}_2, \mathfrak{so}_{2n+1}, \mathfrak{sp}_n, \mathfrak{so}_{4n}$), as well as the compact real forms of types $E_7, E_8, F_4$ and $G_2$.

Note that even in the remaining cases, some representations (e.g. the adjoint representation) are always self-dual hence self-conjugate. For all this discussed in terms of duality, cf. Irreducible Dual Representation and finding highest weight of dual of a representation of a semisimple lie algebra and further references therein.

However, for non-compact real forms, I think this is wrong. Namely, the action of complex conjugation on the weight lattice is in general no longer given by $-w_0$ (whereas dualising still is). Rather, the action of complex conjugation on the weight lattice which we need now is described by what is called the "twisted Galois action" (denoted e.g. $\tau(\sigma)$ in Springer's Linear Algebraic Groups 15.5.2, as $\sigma^*$ in Tits' article in the Boulder Proceedings, as $_\Delta \sigma$ in Borel-Tits' Groupes réductifs 6.2., as $t(\sigma)$ in my thesis 3.1.3.2), which conveniently can be read from the Satake-Tits diagram of the real form.

Since the arrows in such a diagram must correspond to a diagram automorphism, and the list in the compact case above consists $\color{red}{\text{with one exception}}$ of those Dynkin diagrams which have no diagram automorphism at all, we conclude more generally:

All representations of all real forms of types $A_1, B_n, C_n, E_7, E_8, F_4$ and $G_2$ are self-conjugate (but $\color{red}{\text{not all of type } D!})$

(And they are also self-dual, which maybe contributed to the impression that conjugacy would still be basically the same as duality.) (Edit: I see I recovered content of highly upvoted posts from MathOverflow here.) But further we have e.g.:

All representations of all split forms are self-conjugate.

(which is kind of obvious writing down split forms, where "all matrix entries are real"), and this includes for example all $\mathfrak{sl}_n(\mathbb R)$ i.e. split forms of type $A_n$, which for $n \ge 2$ have many representations which are not self-dual.

Conversely, for all quasi-split forms which are non-split, there is an arrow in the Satake diagram, and hence there will be (fundamental) representations which are not self-conjugate. One class of examples for such quasi-split but non-split forms is given by the simple complex Lie algebras seen as real Lie algebras. E.g. the Lorentz Lie algebra $\mathfrak{so}(3,1)$, a six-dimensional Lie algebra which can be viewed as the underlying real Lie algebra of $\mathfrak{sl}_2(\mathbb C)$:

$\mathfrak{so}(3,1)$ has two fundamental $2$-dimensional representations which are conjugate to each other, although each of them is self-dual.

(The Satake diagram of this consists of two white nodes without an edge between them, but with an arrow between them; quasi-split form of type $D_2 \simeq A_1 \times A_1$.)

I wrote a lengthy related answer recently here: https://math.stackexchange.com/a/3258221/96384. Another example in there shows that each of the the three real forms of type $A_2$ has two fundamental representations (which are dual to each other, and are called "the fundamental" and "the antifundamental" one by physicists). For the split form, each of these two is self-conjugate (and so would be every representation of the split form), whereas both for the quasi-split and the compact form, the conjugate of the a fundamental representation is equivalent to its dual, i.e. in physicists terminology, conjugation switches the fundamental and the antifundamental one; which matches the fact that the Satake-Tits diagram of the split form has no arrows, whereas the ones for the other two forms have arrows.

To end with an answer to your question: Given a real form $\mathfrak{g}$, determine the action of complex conjugation on the root system, and hence the weight lattice, from the Satake-Tits diagram (it is an involutive action that stabilises a fixed basis of the roots, w.r.t which the weight lattice is ordered). Denote this action by $\sigma^*$. Any irreducible complex representation of $\mathfrak{g}$ is given as a restriction to $\mathfrak{g}$ of some highest-weight complex irrep

$V(\lambda)$

for a dominant integral weight $\lambda$. Now the conjugate representation of that will be equivalent to

$V(\sigma^*\lambda)$;

in other words, your irrep is self-conjugate iff $\sigma^*\lambda = \lambda$. That depends on both $\lambda$ and the Galois action.

(Reiterating things said above, for the compact forms, $\sigma^* = -w_0$ i.e. conjugation does the same as dualising; for split forms, $\sigma^*=id$; for $\lambda=$ the highest root i.e. $V(\lambda) =$adjoint rep, $\sigma^* \lambda = \lambda$, regardless of what $\sigma^*$ is otherwise on your form.)

When you say "the fundamental" and "the antifundamental representation", probably you mean what mathematicians would call the defining representation (in one of the classical types $A-D$) and its dual, which are inequivalent only in type $A_{n \ge 2}$ to begin with. In math terminology, there are as many fundamental representations as the rank of the root system. (Cf. 1, 2, 3 for this clash of terminologies.) If they all are self-conjugate (which in type $A$ they are iff your two special ones are self-conjugate, e.g. in the split case), then every representation is self-conjugate (because for this, the $\sigma^*$-action must be trivial). However, as said, even if some or all of the fundamental reps are not self-conjugate, infinitely many irreducible representations still will be self-conjugate. Also, e.g. in type $D_{n \ge 3}$, the defining representation is always (self-dual and) self-conjugate, but there might still be other (fundamental) representations which are not, depending on what form you're looking at.