How does a charged field's coupling to the electromagnetic field translate to a worldline coupling?
I think the best way to see where the four-point interaction disappears is to follow the derivation of the worldline formalism. Christian Schneider's PhD thesis has a good walkthrough for scalars and spinors in section 4.2; the majority of this answer paraphrases it.
Starting with the scalar QED Lagrangian you write down at the beginning of the question, you get the path integral
$$\mathcal{Z} = \int \mathcal{D} \phi \mathcal{D} \phi^* \mathcal{D}A \, \mathrm{e}^{iS[\phi, \phi^*, A]}$$
where $S$ is the action from the scalar QED lagrangian. The next step is to rewrite the lagrangian in a Gaussian form: by essentially completing the square we find
$$S \supset \phi^* (D^2 + m^2) \phi.$$
The next trick is to Wick rotate to Euclidean space (changing $D^2 \to -D^2$ and getting rid of the oscillatory integral), so we can integrate out the matter field to give
$$\mathcal{Z}_\mathrm{E} = \int \mathcal{D} A \det (-D^2 + m^2) \mathrm{e}^{- \frac{1}{4}\int F_{\mu \nu} F^{\mu \nu}}. $$
This is where the four-point interaction appears to vanish: we have integrated out over all the scalar field loops and hidden the interactions in the functional determinant. What follows is a series of integral tricks to write the functional determinant as an integral over all worldlines. We eventually get
$$\mathcal{Z}_\mathrm{E} = \mathcal{N} \int_0^\infty \frac{\mathrm{d}T}{T} \int \mathcal{D} x \int \mathrm{D} A \, \mathrm{e}^{-\frac{1}{4} \int F_{\mu \nu} F^{\mu \nu}}\,\mathrm{e}^{-\int_0^T \mathcal{L}_\mathrm{eff}[x, A] \, \mathrm{d} \tau},$$
where $\mathcal{L}_\mathrm{eff}$ is the worldline effective action, and $T$ is a parameter introduced that can be thought of as the worldline length. The effects of the four-point interaction are hidden in the $x$ and and $T$ integrals; if you sum them to all orders you'll get the same result as the standard QFT one.
As noted on p6 of this paper, the fact that the worldline method integrates over diagrams including the scalar interactions is one of its strengths; it can greatly simplify certain calculations.