Analyzing the P.D.E that describes the displacement of a circular membrane
In my opinion, the first question asks to show that the dynamics of $u$ conserves some total energy. Generally, a membrane could oscillate and damp (or resonate), whereas oscillation is the only case with conservative energy. Following the total energy of a membrane, it suffices to show something similar to (but not exactly is, as you will see very soon) $$ \frac{\rm d}{{\rm d}t}\int_{B(0,a)}\frac{1}{2}\left(u_t^2+c^2\left\|\nabla u\right\|^2\right){\rm d}V=0, $$ provided the boundary condition $u=\mathbf{n}\cdot\nabla u$ on $\partial B(0,a)$. Here $B(0,a)$ denotes the disk-like domain centered at $0$ with a radius of $a$, while $\mathbf{n}$ stands for the outward unit normal of $B(0,a)$ along its boundary $\partial B(0,a)$.
The above time derivative is straightforward to figure out. \begin{align} &\frac{\rm d}{{\rm d}t}\int_{B(0,a)}\frac{1}{2}\left(u_t^2+c^2\left\|\nabla u\right\|^2\right){\rm d}V\\ &=\int_{B(0,a)}\left(u_tu_{tt}+c^2\nabla u\cdot\nabla u_t\right){\rm d}V\\ &=\int_{B(0,a)}u_tu_{tt}{\rm d}V+c^2\int_{B(0,a)}\nabla u\cdot\nabla u_t{\rm d}V\\ &=\int_{B(0,a)}u_tu_{tt}{\rm d}V+c^2\int_{B(0,a)}\left[\nabla\cdot\left(u_t\nabla u\right)-u_t\Delta u\right]{\rm d}V\\ &=\int_{B(0,a)}u_t\left(u_{tt}-c^2\Delta u\right){\rm d}V+c^2\int_{B(0,a)}\nabla\cdot\left(u_t\nabla u\right){\rm d}V\\ &=c^2\int_{B(0,a)}\nabla\cdot\left(u_t\nabla u\right){\rm d}V\\ &=c^2\int_{\partial B(0,a)}u_t\nabla u\cdot{\rm d}\mathbf{S}\\ &=c^2\int_{\partial B(0,a)}u_t\mathbf{n}\cdot\nabla u{\rm d}S\\ &=c^2\int_{\partial B(0,a)}uu_t{\rm d}S\\ &=\frac{\rm d}{{\rm d}t}\int_{\partial B(0,a)}\frac{1}{2}c^2u^2{\rm d}S. \end{align} Therefore, we conclude that the real energy conservation reads $$ \frac{\rm d}{{\rm d}t}\left[\int_{B(0,a)}\frac{1}{2}\left(u_t^2+c^2\left\|\nabla u\right\|^2\right){\rm d}V-\int_{\partial B(0,a)}\frac{1}{2}c^2u^2{\rm d}S\right]=0. $$ This conservation implies that the membrane only oscillates.
As we try to solve the equation, noteworthy is the periodicity of $u$ with respect to $\theta$, i.e., $$ u(r,\theta,t)=u(r,\theta+2\pi,t) $$ holds for all $r$ and $t$. Thanks to this periodicity, $u$ observes the following Fourier expansion $$ u(r,\theta,t)=\sum_{n\in\mathbb{Z}}\hat{u}_n(r,t)e^{in\theta}. $$ Assume that this series converges absolutely (because we are only interested in the classical solutions). As such, the operators $\partial_t$ and $\Delta$ both commute with $\sum_{n\in\mathbb{Z}}$. The wave equation yields $$ 0=u_{tt}-c^2\Delta u=\sum_{n\in\mathbb{Z}}\left(\left(\hat{u}_n\right)_{tt}-c^2\left(\left(\hat{u}_n\right)_{rr}+\frac{1}{r}\left(\hat{u}_n\right)_r-\frac{n^2}{r^2}\hat{u}_n\right)\right)e^{in\theta}. $$ Note that $\left\{e^{in\theta}\right\}_{n\in\mathbb{Z}}$ is linearly independent, for which the coefficient in front of each $e^{in\theta}$ must vanish, i.e., $$ \left(\hat{u}_n\right)_{tt}=c^2\left(\left(\hat{u}_n\right)_{rr}+\frac{1}{r}\left(\hat{u}_n\right)_r-\frac{n^2}{r^2}\hat{u}_n\right). $$ In addition, the boundary condition $u(a,\theta,t)=u_r(a,\theta,t)$ implies $$ \sum_{n\in\mathbb{Z}}\hat{u}_n(a,t)e^{in\theta}=\sum_{n\in\mathbb{Z}}\left(\hat{u}_n\right)_r(a,t)e^{in\theta}\iff\hat{u}_n(a,t)=\left(\hat{u}_n\right)_r(a,t). $$
A natural frequency can be understood as an eigenvalue of $\partial_t^2$, subject to the boundary condition. That is, we need to figure out $\nu$ that satisfies $$ u_{tt}=\nu u, $$ subject to $u(a,\theta,t)=u_r(a,\theta,t)$. Note that $$ \nu\hat{u}_n=\left(\hat{u}_n\right)_{tt}=c^2\left(\left(\hat{u}_n\right)_{rr}+\frac{1}{r}\left(\hat{u}_n\right)_r-\frac{n^2}{r^2}\hat{u}_n\right). $$ Thus we need to figure out $\nu$ that satisfies $$ \left(\hat{u}_n\right)_{rr}+\frac{1}{r}\left(\hat{u}_n\right)_r-\left(\frac{n^2}{r^2}+\frac{\nu}{c^2}\right)\hat{u}_n=0, $$ subject to $$ \hat{u}_n(a,t)=\left(\hat{u}_n\right)_r(a,t). $$ Plus, since the membrane is smooth at $r=0$, it is a must that $$ u(-r,\theta,t)=u(r,\theta+\pi,t) $$ holds for all $r$, $\theta$ and $t$. This gives, by taking the derivative with respect to $r$, $$ -u_r(-r,\theta,t)=u_r(r,\theta+\pi,t). $$ Thus the boundary condition implies $$ u(-a,\theta,t)=u(a,\theta+\pi,t)=u_r(a,\theta+\pi,t)=-u_r(-a,\theta,t), $$ which is equivalent to $$ \hat{u}_n(-a,t)=-\left(\hat{u}_n\right)_r(-a,t). $$
Note that $t$ in the last three equations of $\hat{u}_n$ acts as an auxiliary parameter, and it suffices to deal with the following ordinary differential equation \begin{align} f''(r)+\frac{1}{r}f'(r)-\left(\frac{n^2}{r^2}+\frac{\nu}{c^2}\right)f(r)&=0,\\ f'(a)-f(a)&=0,\\ f'(-a)+f(-a)&=0, \end{align} by finding some appropriate $\nu$ that gives some non-zero $f$.
Keep in mind that $f$ is defined for $r\in\left[-a,a\right]$. Suppose $f$ is analytic, and it yields a power series expansion at $r=0$: $$ f(r)=\sum_{m=0}^{\infty}a_mr^m $$ for all $r\in\left(-a,a\right)$. Thanks to this expansion, the governing equation leads to $$ \sum_{m=0}^{\infty}a_mm\left(m-1\right)r^{m-2}+\sum_{m=0}^{\infty}a_mmr^{m-2}-n^2\sum_{m=0}^{\infty}a_mr^{m-2}-\frac{\nu}{c^2}\sum_{m=0}^{\infty}a_mr^m=0, $$ or equivalently, $$ \sum_{m=0}^{\infty}a_m\left(m^2-n^2\right)r^{m-2}-\frac{\nu}{c^2}\sum_{m=0}^{\infty}a_mr^m=0, $$ or equivalently, $$ \sum_{m=-2}^{-1}a_{m+2}\left[\left(m+2\right)^2-n^2\right]r^m+\sum_{m=0}^{\infty}\left\{\left[\left(m+2\right)^2-n^2\right]a_{m+2}-\frac{\nu}{c^2}a_m\right\}r^m=0. $$
- If $n=0$, the first sum implies that $a_0$ could be arbitrary and that $a_1=0$, while the second sum implies that $$ a_{m+2}=\frac{1}{\left(m+2\right)^2}\frac{\nu}{c^2}a_m $$ holds for all $m\ge 0$. This gives that $a_1=a_3=\cdots=0$ and that $$ a_{2k}=\frac{1}{\left(2^kk!\right)^2}\left(\frac{\nu}{c^2}\right)^ka_0 $$ for all $k=1,2,\cdots$. In this case, $$ f(r)=a_0\sum_{k=0}^{\infty}\frac{1}{\left(2^kk!\right)^2}\left(\frac{\nu}{c^2}\right)^kr^{2k}, $$ where $a_0$ and $\nu$ are the only unknowns. They could be uniquely determined by the boundary conditions $f'(a)-f(a)=0$ and $f'(-a)+f(-a)=0$, which, unfortunately, could not be expressed explicitly in an easy fashion.
- If $n=\pm 1$, the first sum implies that $a_0=0$ and that $a_1$ could be arbitrary, while the second sum implies that $$ a_{m+2}=\frac{1}{\left(m+2\right)^2-1}\frac{\nu}{c^2}a_m $$ holds for all $m\ge 0$. This gives that $a_0=a_2=\cdots=0$ and that each $a_{2k+1}$ ($k\ge 0$) is uniquely determined by $a_1$. Thus in this scenario, $a_1$ and $\nu$ are the only unknowns, can could be determined by the two boundary conditions.
- If $\left|n\right|\ge 2$, the first sum implies that $a_0=a_1=0$, while the second sum implies that $$ a_{m+2}=\frac{1}{\left(m+2\right)^2-n^2}\frac{\nu}{c^2}a_m $$ holds for all $m\ne\left|n\right|-2$, and that $a_{\left|n\right|}$ could be arbitrary. This gives that $a_0=a_1=\cdots=a_{\left|n\right|-1}=0$, and that $a_{\left|n\right|+1}=a_{\left|n\right|+3}=\cdots=0$, and that each $a_{\left|n\right|+2k}$ ($k\ge 1$) is uniquely determined by $a_{\left|n\right|}$. Hence in this case, $a_{\left|n\right|}$ and $\nu$ are the only unknowns, can could be determined by the two boundary conditions.
Recall that $$ u(r,\theta,t)=\sum_{n\in\mathbb{Z}}\hat{u}_n(r,t)e^{in\theta}, $$ where \begin{align} \left(\hat{u}_n\right)_{tt}-c^2\left(\left(\hat{u}_n\right)_{rr}+\frac{1}{r}\left(\hat{u}_n\right)_r-\frac{n^2}{r^2}\hat{u}_n\right)&=0,\\ \left(\hat{u}_n\right)_r(a,t)-\hat{u}_n(a,t)&=0. \end{align} The initial conditions yield $$ u(r,\theta,0)=0\iff\sum_{n\in\mathbb{Z}}\hat{u}_n(r,0)e^{in\theta}=0\iff\hat{u}_n(r,0)=0 $$ and $$ u_t(r,\theta,0)=\alpha(r)\sin 3\theta\iff\sum_{n\in\mathbb{Z}}\left(\hat{u}_n\right)_t(r,0)e^{in\theta}=\alpha(r)\sin 3\theta=\alpha(r)\frac{e^{i3\theta}-e^{-i3\theta}}{2i}, $$ the last of which is equivalent to \begin{align} \left(\hat{u}_3\right)_t(r,0)&=\frac{1}{2i}\alpha(r),\\ \left(\hat{u}_{-3}\right)_t(r,0)&=-\frac{1}{2i}\alpha(r),\\ \left(\hat{u}_n\right)_t(r,0)&=0,&&n\ne\pm 3, \end{align} or in a uniform way, $$ \left(\hat{u}_n\right)_t(r,0)=\frac{n}{6i}\alpha(r)\delta_{9,n^2}, $$ where $\delta$ denotes the Kronecker delta.
To sum up, the initial-boundary-value problem requires to solve \begin{align} \left(\hat{u}_n\right)_{tt}-c^2\left(\left(\hat{u}_n\right)_{rr}+\frac{1}{r}\left(\hat{u}_n\right)_r-\frac{n^2}{r^2}\hat{u}_n\right)&=0,\\ \left(\hat{u}_n\right)_r(a,t)-\hat{u}_n(a,t)&=0,\\ \left(\hat{u}_n\right)_r(-a,t)+\hat{u}_n(-a,t)&=0,\\ \hat{u}_n(r,0)&=0,\\ \left(\hat{u}_n\right)_t(r,0)&=\frac{n}{6i}\alpha(r)\delta_{9,n^2} \end{align} for all $n\in\mathbb{Z}$, $r\in\left[-a,a\right]$ and $t\ge 0$. Unfortunately, an easy expression for this seems impossible: the general solution witnesses a rather complicated form, while the parameters therein determined by the initial and boundary conditions are highly unlikely to be expressed in an explicit way.
$$ \left| \begin{array}{l} u_{tt}=c^2\Delta u\\ u(a,\theta,t)=u_r(a,\theta,t)\\ u(r,\theta,0)=0\\ u_t(r,\theta,0)=\alpha(r)\sin(3\theta) \end{array} \right. $$
I'm only going to focus on the initial/boundary value problem above. Letting some of the eigenvalues be squared is helpful, so I'm going to say that
$$h''(t)=-\lambda^2 c^2h(t),$$
is the time equation. This lets us write $h(t)=\sin(\lambda c t)$ (up to a scaling constant), which is the solution that vanishes at $t=0$. Similarly, we can see that $$ g(\theta) = A\cos(n\theta)+B\sin(n\theta) $$
meets the the continuity/periodicity conditions for the angular component and solves the equation.
The final piece is the bessel function / radial part, which you noted is solved by $$ f(r) = J_n(\lambda r). $$
To enforce the outward gradient condition on the boundary, we calculate $$ f(a) = f'(a) $$ $$ J_n(\lambda a) = \frac{J_{n-1}(\lambda a)-J_{n+1}(\lambda a)}{2} $$ I claim that for $a$ fixed, for every $n$, the above equation has a family of solutions I'll note as $\lambda_{nm}$ that satisfies this condition. There is no closed form to be had. Here, $m$ can be an integer, and $\lambda_{nm}$ is the $m$th solution of the $n$th equation above. Put another way, $n$ indexes the allowable angular solutions, and $m$ indexes the allowable radial ones, where the radial solutions can only have the discrete $\lambda$ values that satisfy the boundary condition. You can always numerically calculate the exact numbers, but take a look here to see what one example of the roots of this equation looks like:
http://www.wolframalpha.com/input/?i=BesselJ%5B1,x%5D-BesselJ%5B0,x%5D%2F2%2BBesselJ%5B2,x%5D%2F2
Taken together, we get a family of solutions of the form $$ u_{nm}(r,\theta,t) = A_{nm}\cos(n\theta)\sin(\lambda_{nm} c t)J_n(\lambda_{nm} r)+B_{nm}\sin(n\theta)\sin(\lambda_{nm} c t)J_n(\lambda_{nm} r) $$
The full general solution is then a sum over the angular and radial modes: $$ u(r,\theta,t) = \sum_{m=-\infty}^\infty\sum_{n=0}^\infty \left(A_{nm}\cos(n\theta)\sin(\lambda_{nm} c t)J_n(\lambda_{nm} r)+B_{nm}\sin(n\theta)\sin(\lambda_{nm} c t)J_n(\lambda_{nm} r)\right) $$ To finally solve your IVP, calculate $u_t\left.\right|_{t=0}$ $$ u_t(r,\theta,0) = \sum_{m=-\infty}^\infty\sum_{n=0}^\infty \left(A_{nm}\lambda_{nm} c\cos(n\theta)J_n(\lambda_{nm} r)+B_{nm}\lambda_{nm} c\sin(n\theta)J_n(\lambda_{nm} r)\right) $$ If this must equal something with only a $\sin(3\theta)$ dependency, that tells us all the $A_{nm}=0$ and only the $n=3$ term in the $n$ summation remains: $$ u_t(r,\theta,0) = \sin(3\theta)\sum_{m=-\infty}^\infty B_{m}\lambda_{3m} cJ_3(\lambda_{3m} r) = \alpha(r)\sin(3\theta) $$ Cancelling the $\theta$ dependency $$ \sum_{m=-\infty}^\infty B_{m}\lambda_{3m} cJ_3(\lambda_{3m} r) = \alpha(r) $$ For simplicity let $C_m = B_{m}\lambda_{3m} c$ so that $$ \sum_{m=-\infty}^\infty C_{m} J_3(\lambda_{3m} r) = \alpha(r) $$ Now the task is to find the $C_{m}$ coefficients that make this true, i.e., find a representation of an arbitrary $\alpha$ function in terms of $J_3$ bessel functions with the set of discrete frequencies determined by the radial boundary conditions. Again, this can all be done numerically to any precision desired, but there is no closed form to be had here. Once you find those coefficients, the final solution will be $$ u(r,\theta,t) = \sin(3\theta)\sum_{m=-\infty}^\infty B_{m}\sin(\lambda_{3m} c t)J_3(\lambda_{3m} r) $$