What is QED about "Cavity QED"?
The terminology "cavity QED" is used to signal that one is treating the electromagnetic field as itself quantised and there is a cavity with mirrors in the problem. The alternative would be to treat the electric field of the cavity mode as a classical variable. Instead one uses raising and lowering operators for both the field mode and the internal state of the atom; that is what makes it "QED"-like.
Having said that, one usually moves quickly to a master-equation style of calculation where one does not track all the modes of the em field. One keeps one mode (the cavity mode in question) explicitly in the analysis, and treats the others as a bath, typically at zero temperature.
What would I be missing in Cavity QED if I had never learnt about quantum electrodynamcis?
Excellent question, especially the way it is posed!
The bottom line is that what the OP calls "standard quantum optics" is essentially the non-relativistic sector of QED. Many textbooks indeed do not come from the QED perspective and instead introduce effective descriptions (such as Master equations) which only contain the important parts of the physics. That said QED and quantum optics are not really separated fields, but continuously transition into each other. For example if you look at what happens in very strong laser fields, you are often right at the interface of quantum optics and QED.
What makes cavity QED cool is that we can squeeze out more "quantumness" by trapping the light in a box, which makes its interaction with matter much stronger. This causes the dynamics to become more correlated on a quantum level and one can see the effects thereof both in the light and in the matter sector.
The strong coupling regime is only the first step in this, by now even much stronger couplings are accessible, such as the ultra-strong coupling regime (see also What are the "strong", "ultrastrong" and "deep strong" coupling regimes of the Rabi model?).
The point is that the light-matter interaction can become non-pertubative in these regimes. One result of this is that a few aspects of QED, which are not so relevant in other parts of quantum optics, become relevant there (for example the choice of gauge, virtual photons and much more).
That said, what is certainly still absent from cavity QED are relativistic effects, which you certainly get in QED. Most importantly there are no electron-positron loops in the theory and so there is also no Schwinger pair production. As mentioned before, these and many other effects (such as quantum radiation reaction and other cool stuff) become relevant for example in strong laser fields, and of course you can get all kinds of phenomena in QED which do not have much to do with quantum optics any more.
So what would you be missing in Cavity QED if you had never learnt about quantum electrodynamcis? Probably everything, unless you knew how to quantize light from somewhere else. I doubt you would though, "quantum electrodynamics" is the theory of how to do that after all (and the cool stuff that happens when you throw matter into the mix).
Well if you are representing light as a quantum field and not as a classical field, then I suppose cavity QED is QED.
The only difference is that cQED is usually done with nonrelativistic particles, so maybe cQED is best understood as nonrelativistic QED in a strongly coupled environment. What this also means is that things like pair-production, positrons, etc. are irrelevant, and you don't have to UV-complete your theory since there is a natural cutoff.
In my opinion, you would not be missing very much by not learning QED before cQED unless you want to understand cQED at very high energies where relativistic effects are important (or tiny shifts caused by relativistic QED like the Lamb shift).