What is the relationship between generalized functions and things like the Riesz representation theorem?
I thought I should expand a bit on my earlier answer.
As I see it, there are two questions here: do generalized functions make sense on any Hilbert space and if not is there an analogue of a classical Riesz representation theorem which is valid here?
To answer the first question, the first thing we need to rule out is the $\delta$ function as a functional on $L^{2}$ I would argue that the big issue here is that $L^{2}$ functions do not exist pointwise--you can change their value on any set of measure zero without changing the function. Because of this, there is no hope of making sense of $\delta$ over the Hilbert space you are used to.
The natural question then becomes what space we should work on. I would argue that the most important property of generalized functions is that they are all 'smooth' in the distributional sense (i.e. infinitely differentiable). If this is the property that you want them to have, then you need at least that level of regularity on the test functions you apply the generalized functions to (since by duality we apply all the derivatives to the test function to define the distributional derivative). This is essentially the trade-off one usually sees when defining objects through duality: the more structure you put on the test objects, the less structure you need on your dual objects to get nice properties. This natural path leads one to the theory of distributions, which are the continuous linear functionals on the space $C_{0}^{\infty}$ with a particularly nasty (not even metrizible) but entirely natural topology on it.
For a sequence of test functions to converge in $C_{0}^{\infty}$ we need two things: they need to live in a single common compact set (so no infinitely expanding supports) and all of the partial derivatives have to converge in the infinity norm, i.e. $\|\partial^{\alpha}\varphi_{n}-\partial^{\alpha}\varphi\|_{\infty}\to0$ for all multiindices $\alpha.$ Continuity of a functional linear functional $T$ in the dual means that if $\varphi_{n}\to\varphi$ in this topology, then $T\varphi_{n}\to T\varphi.$ As a comment, this is not good enough to specify the topology since we are not working on a metric space. To prove that this space is not metrizible, we would actually have to go through the nitty gritty details of its construction. It's not too terrible if you have a background in topological vector spaces, but it's not edifying if you don't so I won't. The important property for our purposes is that there is no metric space with the same topology as the one we gave to $C_{0}^{\infty}.$ In particular, it does not admit a Hilbert space structure. Now, Hilbert spaces are all self-dual (this is the Riesz theorem you are used to), which means that $C_{0}^{\infty*}$also does not admit a Hilbert space structure. The answer to the first question is therefore no. Generalized functions do not make sense over a Hilbert space.
Before going into the second question, let me comment that there are a lot of 'Riesz representation theorems.' All that these theorems have in common is that over many nice spaces there is a natural class of continuous linear functionals (on a Hilbert space, it is given by the inner product; on $L^{p}$ it is given by multiplication by an $L^{q}$ function and justified by Holder's inequality; on $C_{0}$ it is given by integration against a Borel measure). The common trait of Riesz-type theorems is that they say that all functionals belong to this natural class of functionals. The obvious class in our case is integration against Borel measures (since our functions are still continuous), but we cannot apply the classical Riesz-Markov theorem since we changed the topology of the space we are working over. It turns out the result is false in any case: the objects we are working with have too much structure for measures to be able to distinguish between all of them.
The fact that the $\delta$ function is represented as a measure when viewed as a functional over $C_{0}^{\infty}$ is a manifestation of the fact that all positive distributions (that is all continuous linear functionals $T$ on $C_{0}^{\infty}$ with the property that if $f\geq0$ then $Tf\geq0$) are represented as measures.
For an example of why we cannot apply Riesz-Markov here, I do not believe that the generalized function given by the derivative of the Dirac delta is represented as a measure.
The Dirac delta is simply the functional $\delta:f\mapsto f(0)$ on $C_0(\mathbb{R})$. It can also be seen as a integral (via Riesz-Markov if you want, but one can check it directly): $$ \delta(f)=\int_\mathbb{R}\,f\,d\mu_\delta, $$ where $\mu_\delta$ is the measure defined on the full power set of $\mathbb{R}$ by $$ \mu_\delta(A)=\begin{cases}1&\text{ if } 0\in A\\ 0&\text{ if }0\not\in A\end{cases} $$
The only "confusion" with the delta arises if you want (as it was done intuitively a long time ago, but is actually impossible) to think of $\mu_\delta$ as $g(t)dt$ for some function $g$.
To answer your question proper, as others have already mentioned, the "Riesz theorem" that you think about (the one about Hilbert spaces) is not relevant here. The relevant one is Riesz-Markov.