How is the corresponding author on a (math) paper typically chosen?

My experience as a pure mathematician in the US is that there are essentially no benefits to being the corresponding author. When I was a postdoc and in grad school I was often the corresponding author, but I get the feeling it was mostly because I was often the most junior of the authors and the others didn't want to waste time navigating editorial websites, receiving emails from the journal, mailing people tex files, etc. I would say that if someone is a pure mathematician in the US, the number of times they've been a corresponding author for a paper will have absolutely no impact on their career.

(I should add that I believe that in other countries the situation may be markedly different.)

The only "benefit" that I can think of of being the corresponding author - if you can call it a benefit - is that it prevents underhanded actions by coauthors. Sometimes, people from different groups and motivations, who are possibly at odds with each other, end up co-authoring. Or - an estranged pair of advisor-advisee. Some of the authors might suspect the others of being willing to compromise the paper somehow (obviously I'm being vague since this is inspecific.) The corresponding author, however, has control over communications with the venue, so s/he can exercise effective veto power over steps s/he disapproves of.

As Ben Linowitz points out, the situation can be very different in other countries. In many places in Asia, for example, the significance of a publication on one's CV goes like this:

     publication as first author ≥ publication as corresponding author > publication as any other author.

The assumption here is that the first author contributed the most, while the corresponding author is the PI. Obviously, this should not apply to most mathematical publications where the authors are listed alphabetically, but university guidelines often do not take this into consideration...

Some countries will even put a quantitative value to this statement, explicitly weighting your publications based on the authorship order (in addition to other factors such as the impact factor). Here's an excerpt from Taipei Medical University's promotion standards (PDF):

        Rank of author                       weighted points (A)
----------------------------------------------------------------
first author or corresponding author               5.0
2nd author                                         3.0
3rd author                                         1.0
4th or lower rank author                           0.5 

In some cases things get even weirder — here's an excerpt from the regulations of Seoul National University (PDF):

Score of each research publication stated in Paragraph 1 of this article is as follows:
1. Single author: 100 points
2. Two authors: 70 points
3. Three authors: 50 points
4. Four authors or more: 30 points
If applicant, however, is the first author or the corresponding author in a publication with three or more authors, he/she is entitled to 70 points.

Curiously, sometimes a corresponding author is worth more than first author. The Academic Ranking of World Universities, compiled by an agency in China and arguably one of the most influential international university rankings, uses the following scaling:

To distinguish the order of author affiliation, a weight of 100% is assigned for corresponding author affiliation, 50% for first author affiliation (second author affiliation if the first author affiliation is the same as corresponding author affiliation), 25% for the next author affiliation, and 10% for other author affiliations.

As you can see, there are often advantages to being corresponding author in such cases. Universities can offer additional incentives (e.g. monetary) for being first or corresponding author on high-impact publications, which often leads to more arguments and occurrences such as six "joint first" authors or, similarly, multiple "co-corresponding" authors on a paper.

Anecdotally, I would say that in non-Asian countries, such weighting is also frequently done but on an implicit basis, and is significantly more field-dependent. (For instance, the Korean university regulations do not distinguish between different fields, and I am unsure if different departments can have different rules — if not, I encourage every mathematician named Aaronson to apply for faculty positions in South Korea immediately.)