Coherent sheaves vs sheaves locally of finite presentation
In order to get a sense of the differences between locally of finite presentation and coherent, it's first instructive to see when they're the same. On any locally noetherian scheme $X$, the following conditions on an $\mathcal{O}_X$-module $\mathcal{F}$ are equivalent (cf. Stacks 01XZ):
- $\mathcal{F}$ is coherent,
- $\mathcal{F}$ is quasi-coherent and of finite type (aka $J$ can be chosen to be finite in the exact sequence defining quasi-coherence)
- $\mathcal{F}$ is locally finitely presented (aka $I$ and $J$ can be chosen to be finite in the exact sequence defining quasi-coherence).
This is just fine for lots of algebraic geometry, and it's very easy to conflate "$\mathcal{F}$ is coherent" with "$\mathcal{F}$ is quasi-coherent and $I,J$ can be taken to be finite" because they're exactly the same in this relatively nice and extremely common context. The differences start showing up when we leave the realm of locally noetherian schemes.
In this setting, there can be a strict inclusion of coherent sheaves in to locally finitely presentable sheaves: simply take any scheme on which the structure sheaf is not coherent. One example is $\operatorname{Spec} R$, where $R$ is the ring of germs of $C^\infty$ functions on $\Bbb R$ at $0$ (details here). This example also shows that locally finitely presented sheaves don't form an abelian category in general: the kernel of a morphism of locally finitely presented sheaves can fail to be locally finitely presented.
Now that we've established where these concepts start separating a bit from each other, we can start to get a sense of when and why one might prefer one over the other. The key tradeoff with coherent sheaves is that the finiteness condition of $\ker(\mathcal{O}_X^n|_U\to \mathcal{F}|_U)$ being finite type for any $U$ and any morphism $\mathcal{O}_X^n|_U\to\mathcal{F}|_U$ is very strong - it's strong enough to make for a good theory on any ringed space ($Coh(X)$ is an abelian category for any ringed space $X$), but if your space $X$ is "big" enough, coherent sheaves can fail to include the structure sheaf as seen above. Coherent sheaves also don't necessarily play well with pullback - in general, if $f:Y\to X$ is a map of ringed spaces and $\mathcal{F}$ is a coherent sheaf on $X$, there's no reason for $f^*\mathcal{F}$ to be coherent on $Y$, which is a bit of a disappointment to folks who enjoy functors.
Sheaves locally of finite presentation (and quasi-coherent sheaves more generally), on the other hand, always transform properly under pullback, since the pullback of the structure sheaf is the structure sheaf and pullback is right exact. Combining this with the fact that a fair number of nice theorems we know for coherent sheaves on schemes in the locally noetherian case admit generalizations to locally finitely presented sheaves in the non-noetherian case (for instance, the sheaf hom result you link), it is useful to relax our finiteness condition a little bit from coherence to locally finite presentation.
It may be interesting to you to know a little bit of the history around coherent sheaves to understand why they're set up the way they are. Sheaves were introduced by Jean Leray in 1946, expositing work he did while imprisoned as a POW during World War II. Cartan and Oka were major players in introducing sheaves to complex-analytic geometry in the next few years, proving Cartan's theorem A and B and Oka's coherence theorem (circa 1950-51). In 1955, Serre published FAC, introducing sheaves in to algebraic geometry, and a scant few years later (~1957), Grothendieck introduced schemes. So coherence as a condition was originally developed in the analytic framework, where the extra strength of the finiteness condition was useful.
First of all, if $\mathscr{O}$ is coherent over itself, then a sheaf of $\mathscr{O}$-modules is coherent if and only if it is locally finitely presented. This is due to the fact, that cokernels of morphisms between coherent sheaves are always coherent themselves. Thus, for $\mathscr{O}$ the sheaf of regular functions on a noetherian scheme and for $\mathscr{O}$ the sheaf of holomorphic functions on a complex analytic variety both concepts agree.
Moreover, it is definitely not true in general, that the category of locally finitely presented $\mathscr{O}$-modules is abelian. Indeed, obviously $\mathscr{O}$ is always (locally) finitely presented itself, however the answer to this question provides an example of a morphism $\mathscr{O} \rightarrow \mathscr{O}$ (where $\mathscr{O}$ is the sheaf of smooth functions on $\mathbb{R}$) for which the kernel is not even of finite type.
Sadly, I do not really know anything more about cases in which $\mathscr{O}$ is not coherent over itself (I suppose you are rather interested in something algebraic).