Someone cited my paper in an irrelevant manner

Well, what can you do? Not much. I can confirm that it does indeed happen… sometime people even cite a paper of yours to justify a conclusion that you have not reached in the paper, and even one that you strongly disagree with.

One option is to let it slide. You are not responsible for the content of papers that refer to yours, or the accuracy of their citations for that matter. The paper author is responsible, and to some extent, the journal’s referees. (I tend to spend quite some time checking citations when I review papers, but that might just be me being overly sensitive to this particular issue.)

Another option is to contact the paper’s corresponding author, and ask him point blank. You have read his paper, and you are unclear as to the extent of the connection between his writing and yours. See what it gives.

Finally, in the current way academic research works, you do not really have any mean to call out their behaviour publicly. I do not believe you should, either.

There is a huge gulf between what would happen in an ideal world and what the norms are in this case.

"Ideally", you'd get in touch with the author(s), explain that you don't see what their paper has to do with yours; they'd explain why they think it is relevant or agree that it's not, and modify the paper accordingly. (Almost everything is online, so modification after publication is a possibility.)

In the actual world, citations are of benefit to you, even if they are stupid. Journals are mostly not set up to remove citations easily. No one will check, and if they do check, the detriment will be to the citer, not the cited. So you "shouldn't" do anything about it, and the author would probably be quite surprised if you did (especially if you weren't discreet about it). If you really feel like re-calculating your h-index with that paper removed, go for it. But this sort of thing happens all the time (I think all of my papers with over about 50 citations have been cited stupidly at least once), so you're free to just consider it part of the measurement error inherent in looking at citations.

Incidentally, the ideal isn't necessarily the pragmatic ideal. Doing anything important on the basis of small differences in numbers of citations is fraught with error even if all citations are sensible ones. There's a reasonable argument to be made that you shouldn't bother unless the paper is in your field and is citing you in support of something that your paper showed the converse of. Getting your work exactly backwards to advance their own idea isn't doing you or them any favors, so you should try to work that out.