On the proof that every positive continuous random variable with the memoryless property is exponentially distributed

Why do we use $G(x)=1−F(x)$ instead of just $F(x)$?

Because the memoryless property is that: $\mathsf P(X>t+s\mid X>s)=\mathsf P(X>t)$

So we can use this to state: $$\begin{align}\mathsf P(X>s+t) =&~\mathsf P(X>s)\mathsf P(X>s+t\mid X>s)\\=&~\mathsf P(X>s)\mathsf P(X>t)\\[2ex] 1-F_X(s+t)=&~ (1-F_X(s))(1-F_X(t))\\[1ex] F_X(s+t)=&~ F_X(s)+F_X(t)-F_X(s)F_X(t)\\[3ex] G_X(s+t) =&~ G_X(s)G_X(t) & G_X(z):=1-F_X(z)\end{align}$$

Using $G$ gives a more useful result.

What does the professor mean when he says that you can get real numbers by taking the limit of rational numbers.

Every real number is the limits of some sequence of rational numbers. $$\forall r\in\Bbb R :\lim_{n\in\Bbb N, n\to\infty}\frac{\lfloor{rn}\rfloor}{n}=r$$

In the video, he just says that $G(x)=G(1)^x$ looks like an exponential and thus, $G(x)=G(1)^x=e^{x\ln G(1)}$.   How did he know that this is an exponential?

Because it looks like it.   It is an easily recognised pattern.

By definition of the $\ln()$ function, for any $a$ (except zero), $a^x = e^{x\ln a}$.

Thus $G(1)^x=e^{x\ln G(1)}$

In the proposed order:

1. You can use $G(x)$ instead of $(1-F(x))$ just to make you demonstration less clumsy by not carrying that difference around.
   For instance, the first line would be more readable:

   Try $s=t$, this gives us $G(2t)=G(t)^2$, $G(3t)=G(t)^3$, ..., $G(kt)=G(t)^k$
   Try $s=t$, this gives us $1-F(2t)=(1-F(t))^2$, $1-F(3t)=(1-F(t))^3$, ..., $1-F(kt)=(1-F(t))^k$

2. Consider the set $E$ of exponents $x$ for which $G(xt)=G(t)^x$.
   Until that point of the proof, all they proved was that $\mathbb{Q}^+\subset E$.
   They did not prove, in particular, that $\mathbb{R}^+\subset E$.
   Now, $\mathbb{R}^+$ is the closure of $\mathbb{Q}^+$ (i.e., the set of all possible limits $a$ for all possibile sequences $(a_i)$ in $\mathbb{Q}^+$) and that is why they mentioned "taking limit of $m/n$".
   Once $\mathbb{R}^+\subset E$, $G(xt)=G(t)^x$ would be valid for all $x\in\mathbb{R}^+$.

3. Well, $G(1)$ is, indeed, equal to some constant $\kappa\in(0,1)$.
   So saying that $G(x)=G(1)^x$ is the same as saying $G(x)=\kappa^x$, which is an exponential function.
   Since $\kappa = \ln(e^\kappa) = \ln(\exp(\kappa)) = e^{\ln(\kappa)} = \exp(\ln(x))$ (for $\ln$ and $\exp$ are inverse function of one another), then $$G(x) = G(1)^x = \exp(\ln(G(1)))^x = \left(e^{\ln(G(1))}\right)^x = e^{x\ln(G(1))} = e^{x\ln(\kappa)} = e^{x(-\lambda)} = e^{-\lambda x}$$ for some $\lambda>0$, as $\kappa\in(0,1) \implies \ln(\kappa)<0$.