How to publish superseding results without creating enemies
I disagree with the premise of your question. “Publishing superseding results” is basically the same as what’s known as “publishing”, since all papers build and improve on the existing literature in some way and push some older work slightly toward obsolescence or irrelevance. The extra twist in your situation that you are improving on unpublished work is of hardly any consequence and simply not worth worrying about, for the following reasons:
As others have said, the authors of the arxiv preprint still have precedence and will get credit for making their contribution at the time they did, before your subsequent improvement was discovered. Referees and journal editors should (and almost certainly will) take that into account, within reasonable limits.
In general, when you improve on earlier work you show that it is interesting and relevant enough for other people to follow up on. This is actually flattering to the authors of the earlier work, even if you imagine that it is unflattering (it’s also true that an improvement can sometimes portray earlier work in a slightly unflattering light, but my point is there would still be a separate flattering aspect that I think you are ignoring, and which in most cases will far outweigh any supposedly unflattering aspects).
The publication status of the earlier work is simply not your concern. It is up to individual researchers to submit their work for publication and otherwise promote it in a timely manner. If they fail to do so, they have only themselves to blame, and they should not expect others in the scientific community to delay their own follow up research, at the cost of hurting their own careers and slowing down scientific progress, out of pity or charity.
To summarize, I think the idea that you will make enemies by publishing honest work that you did in a STEM field, regardless of the specific circumstances, might make for a cute plot element in a satirical TV show or novel about academia, but is not a realistic thing to worry about in real life.
I agree with what Dan Romik writes in his answer. I write only to address a different aspect (implicitly addressed in his answer too), which are the notions, that seem to me specious, that ArXiv papers should be treated as "unpublished" and that "unpublished" materials should be treated differently from "published" materials.
An article on the ArXiv (or any other online repository) should be treated the same as an article published in a refereed journal for purposes of citation and judging priority. The word "publication" should be treated as a synonym for "dissemination", the only possible qualification being accessibility (and ArXiv papers are far more accessible than paywalled "published" articles!).
The purpose of citation is recognition of priority, acknowledgment of intellectual inspiration, and facilitation of consultation of related work by a reader. None of these objectives is much conditioned by the publication status of an accessible resource.
If a (correct) result is in a paper on the ArXiv, that establishes priority in the same way as if it had been published in a traditional journal. Likewise, a paper on the ArXiv can and should be cited if it would have been cited were it to have appeared in a published journal.
The distinction between prepublication and publication is archaic, a thing of the previous century, and the more so in mathematical areas where dissemination in traditional journals can take years.
Some authors do not cite work on the ArXiv for strange reasons - they haven't checked it, it hasn't been refereed, etc. - but this seems to me mostly just laziness or dishonesty - we should check what we use and never trust that the refereeing process eliminates errors.
As for what to do when your work improves on someone else's work - be glad you will probably have an interested reader and a competent referee for your work. Be generous in your acknowledgment of their work and they will likely react favorably. Don't worry about possible "damage" to them - journals are happy to publish work that has already been cited.
I agree with @Dan Romik's answer, but I have a few things to say about this part:
The publication status of the earlier work is simply not your concern. It is up to individual researchers to submit their work for publication and otherwise promote it in a timely manner. If they fail to do so, they have only themselves to blame, and they should not expect others in the scientific community to delay their own follow up research, at the cost of hurting their own careers and slowing down scientific progress, out of pity or charity.
Indeed the OP should not delay his own work because he built on work that others made public on the arxiv. (That it could be built upon as soon as possible is why they put it on the arxiv!) I hope though that no one will construe this to say that the authors of the original paper have some kind of moral or professional imperative to publish their results before the OP does. (In part I worry about this because of some other answers and comments on this question that seem to imply this.) That's simply not the way the publishing industry works in my experience: papers can be rejected, or can be delayed for years, due to reasons that are entirely out of the authors' control.
If you would not have known or thought to put X into your paper until you read Y's paper then you are not competing with Y, you are building on Y's work. I think the OP should write his paper to make absolutely clear that Y has priority on X and that X is in the paper because of Y's paper. This really should make it easier, not harder, for Y to publish their paper, and if it doesn't, then it is definitely not your fault.