Abelian category from the category of Hopf algebras

$\DeclareMathOperator\Hker{Hker}\DeclareMathOperator\Hcoker{Hcoker}\DeclareMathOperator\Im{Im}\DeclareMathOperator\coIm{coIm}\DeclareMathOperator\Id{Id}$The category $\mathcal{H}$ of finite dimensional, commutative, cocommutative hopf algebras is an abelian category.
The set $\mathcal{H}(F,G)$ of all hopf algebra maps $F\to G$ is an abelian group with sum given by the convolution product. (The convolution inverse for $f\in F$ is: $f\!\circ\! S_F=S_G\!\circ\! f$ and the neutral element is $\eta_G\!\circ\!\varepsilon_F$).
We have also product of maps, given by composition (and composition distributes over convolution).

This is a classic result. I have been told it was first shown by Grothendieck. I know you can find details in Sweedler's book, ch. XVI, p. 314 (basically the whole chapter, p.303–315 presents a detailed proof of the above).

Remark 1: You are right to mention that in the general case the definitions of the kernels and the cokernels seem to pose a problem when one tries to figure out which hopf algebras would be good to make an abelian category from.
For $F$, $G$, f.d., commut, cocommut, hopf algebras and $\omega:F\to G$ a hopf algebra map, then in $\mathcal{H}$, kernels are defined through: $$\Hker\omega=\{f\in F|(I\otimes\omega)\Delta (f)=f\otimes 1\}$$ which -due to cocommutativity- is the same with $\{f\in F|(\omega\otimes I)\Delta (f)=1\otimes f\}$,
and cokernels through: $$\Hcoker\omega=G\big/\big(\omega (F^+)G\big)$$ where $F^+$ is the augmentation ideal and $\omega (F^+)G$ denotes the right ideal (which by commutativity is a two-sided ideal), of $G$ generated by $\omega(F^+)$.
Under the above defs, it can be shown that $\Hker\omega$ is a sub-hopf algebra of $F$, $\Hcoker\omega$ is a quotient hopf algebra of $G$ (i.e. $\omega (F^+)G$ is a hopf ideal) and any hopf algebra map $\omega\in\mathcal{H}(F,G)$ "factorizes" as: $$ \Hker\omega \overset{i_{\omega}}{\hookrightarrow} A \overset{\pi_{i_{\omega}}}{\twoheadrightarrow} \coIm\omega \cong \Im\omega \overset{i_{\pi_{\omega}}}{\hookrightarrow} B \overset{\pi_{\omega}}{\twoheadrightarrow} \Hcoker\omega $$ where:

  • $\Hker\omega \overset{i_{\omega}}{\hookrightarrow} A$ is the kernel of $A\overset{\omega}{\to}B$,
  • $B\overset{\pi_{\omega}}{\twoheadrightarrow} \Hcoker\omega$ is the cokernel of $A\overset{\omega}{\to}B$,
  • $A\overset{\pi_{i_{\omega}}}{\twoheadrightarrow} \coIm\omega$ is the cokernel of $\Hker\omega \overset{i_{\omega}}{\hookrightarrow} A$ and
  • $\Im\omega\overset{i_{\pi_{\omega}}}{\hookrightarrow} B$ is the kernel of $B \overset{\pi_{\omega}}{\twoheadrightarrow} \Hcoker\omega$.

Remark 2: There are even more detailed things to be said about the category $\mathcal{H}$ of finite dimensional, commutative, cocommutative hopf algebras over a field: We can construct a functor $\mathcal{G} : \mathcal{H} \Rrightarrow \mathcal{Ab}_{\text{fin}}$ from $\mathcal{H}$ to the category $\mathcal{Ab}_{\text{fin}}$ of finite, abelian groups (assigning to each hopf algebra of $\mathcal{H}$ the group of its grouplikes) and another functor $\mathcal{F} : \mathcal{Ab}_{\text{fin}} \Rrightarrow \mathcal{H}$ (assigning to each fin, abelian group its group hopf algebra). It is not difficult to show that these functors satisfy $$ \begin{array}{cccc} \mathcal{G} \mathcal{F} = \Id_{\mathcal{Ab}_{\text{fin}}} & & & \mathcal{F} \mathcal{G} \cong \Id_{\mathcal{H}}\\ \end{array} $$ constituting thus an equivalence of the categories $\mathcal{H}$, $\mathcal{Ab}_{\text{fin}}$.

This is a delicate matter, and there are in principle many possible answers depending on exactly what you wish to apply things to.

Frequently, one of two variations of the Hopf subalgebra mentioned by Konstantinos is used: Given $\pi\colon H\to K$ a morphism of Hopf algebras, we have the left coinvariants and right coinvariants (of $\pi$) defined respectively by: $$ {}^{\text{co}\,\pi}H = \{ h\in H \ | \ (\pi\otimes\operatorname{id})\Delta(h) = 1\otimes h\},\\ H^{\text{co}\,\pi} = \{ h \in H \ | \ (\operatorname{id}\otimes\pi)(\Delta(h))= h\otimes 1\}.$$ These need not be the same subobjects of $H$, however.

One could then define a short exact sequence of Hopf algebras (over $k$) $$ k\longrightarrow K \overset{i}{\longrightarrow} H \overset{\pi}{\longrightarrow} L\longrightarrow k$$ to be a sequence of morphisms of Hopf algebras such that

  1. $i$ is injective and $\pi$ is surjective;
  2. $\ker(\pi) = H\, i(K)^+$;
  3. $i(K) = {}^{\text{co}\,\pi}H$.

Note that the more classical notion of kernel is still involved: it will generally not be enough to conduct any of the usual arguments you'd like to do with a "short exact sequence" to assume only the first and third (or first and second) conditions. The first and third are enough when $H$ is faithfully coflat, and the first and second are enough when $H$ is faithfully flat.

This is a good definition for short exact sequences and results that normally rely on them, such as looking for Jordan-Holder analogues. As such, coinvariants-as-kernels will be prominent when working in a category of Hopf algebras. But as I said it's not the only answer, and in other categorical contexts (generalizing/lifting to functors between representation categories, namely) it is the wrong answer. The categorical kernels and cokernels of $\pi\colon H\to K$ are $$ \text{Hker}(\pi) = \{ h\in H \ | \ h_{(1)}\otimes \pi(h_{(2)})\otimes h_{(3)} = h_{(1)}\otimes 1 \otimes h_{(2)}\},\\ \text{Hcoker}(\pi) = K/(K \pi(H^+) K).$$ And this kernel (and cokernel) is not necessarily the same thing as the left or right coinvariants (and associated cokernel). By applying counits to the left/right of the defining relation, we see that $$\text{Hker}(\pi)\subseteq {}^{\text{co}\,\pi}H\cap H^{\text{co}\,\pi},$$ at least. Though equality (of all three subobjects) can happen, and assuring it does is usually the key to making sure attempts at defining exact sequences of tensor categories works out well.

Sonia Natale recently posted a review of these notions, and the related issues with tensor categories, that should be helpful.