Would it be rude to ask a famous professor who doesn't know me personally for a recommendation letter if I have published in a high ranking journal?
The benefit of recommendation letters is that they give insight into abilities that are not reflected in the rest of your application. Your grades and your publication are already in your application. A professor who doesn't know anything else about you has nothing to write that will help you.
To directly answer your question: it comes across more naïve/inexperienced than rude, but either way I do not see how it will help your case.
P.S. This answer assumes that the professor is not already familiar with your work before your email. To be more specific, I'm imagining you sending unsolicited emails to a bunch of the biggest names in your field (selected purely because they are "famous"), who have not previously heard of you or your work (as far as you know), asking them to write a recommendation letter for you entirely on the basis of your having published a paper in a journal.
If the professor is already familiar with your work (e.g. has cited it, or has had an email correspondence with you about it), then I agree with this answer that his/her letter can potentially be helpful. Similarly, I think Ben Webster offers some useful advice about attempting this in a more nuanced - and potentially more effective - way.
While I don't really like adding answers to questions that have too many already, I also think it's a little hard to cut through the arguing any other way.
It's not rude to ask for a letter of recommendation. Ever. (I mean, if you do it rudely, then yes, but the bare fact of asking isn't). It might be unlikely to succeed, or unrealistic in some cases, but it's not rude.
You should try to ascertain whether this is a normal thing to do in your field. In my field (mathematics), it borders on inconceivable that an undergrad would carry out and publish truly impressive work without an established researcher in the field supervising them and able to write a detailed letter about it. Maybe there are some other fields where that can be the case, though I can't say I know what they would be (perhaps some of the other answerers are). Similarly, JeffE's advice to try to create a dossier of the level that might be appropriate for a faculty position sounds completely preposterous to me, but I suppose it must not be in computer science.
Whenever you ask someone for a letter of recommendation, think about what you are hoping the person will say, and whether they are placed well to say it. So, I would only consider asking for a person to write a letter based purely on having read my research if I knew they were familiar with it, and could place it in a context that is not obvious to the people reading the letter. It can be very valuable to have a letter saying essentially "this paper is actually really important. here is why." I've read such letters, I've written such letters. But the paper has to be really important (in the view of the author) for such a letter to work. I suspect a lot of the argument here is based around whether the answerer really thinks this is an impressive publication or not (which we are all guessing about).
So, if I were you, I would ask some of your professors whether they really think this potential letter writer will be impressed by the publication. If they think maybe yes, then you can email her/him, and say:
Dear Prof. X,
I'm a student at the University of Y and am applying to graduate schools in country Z (or maybe be more specific) in underwater basket weaving, with a focus on the use of hemp. Prof. W thought you might be interested in this publication of mine, since it relates to your work on macramae at the bottom of the Mariana Trench. I was wondering if you thought you could write a strong recommendation for me on the basis of this publication. Thank you very much for your time.
Sincerely,
Joe Schmoe
Asking someone if they think they are able to write a strong letter for you is good way of leaving them an easy out, and not getting a terrible letter since someone finds it easier to write a terrible letter than to say no (it happens). I would almost certainly reject such a request since I really wouldn't feel like I could write a strong letter. But if the publication really is that good, maybe someone will feel like they can.
Let me disagree completely with all the other answers here, that I think have missed a crucial point of your question. In short: I believe it is a reasonable plan to ask for the letter.
Here's why: The OP has published a paper in a high ranking venue. Now, if this venue is really a top place, and the OP thinks that the professor knows about the result, which is supposedly a great result, then the fact that the professor doesn't know personally the OP is a huge advantage. This is the best recommendation that one can get: someone that doesn't know him/her but have heard of his/her result, based solely on its scientific merit!
P.S. do not add your grades. Ask for a reference based solely on the result, and explain your contribution if you're a coauthor.