Why does \dimexpr swallow \relax?

To add to what Hendrik says, I think the overall point was that \numexpr, \dimexpr, etc. can be used in a full expansion context without leaving a stray \relax or space:

\edef\example{\the\dimexpr 10 pt + 20 pt \relax}

gives \example defined as 30pt with no unexpected tokens. That is in many ways much 'neater' than the alternative of leaving the \relax in place. The same argument does not apply to TeX's setting of registers as that is never expandable, so the issue does not arise.

(Of course, for a definitive answer you would need to ask the members of the NTS team who actually wrote this code.)

I would say this makes it easy to delimit a \dimexpr in a fully expandable way. Putting a \relax is a usual way to delimit TeX dimensions, which I learned the hard way: It's always a good idea to put a \relax after something like \hskip1pt. Now it seems that eTeX just made a principle out of this.

(An alternative is to put a space to delimit a TeX dimension, but firstly a \relax is a lot more visible, and secondly it's sometimes not so easy to put a space, namely after a control sequence. As Bruno points out in his comment, spaces cannot be used to delimit \dimexpr and friends.)

For the same reason as TeX gobbles a trailing spaces from numbers. In the case of TeX, this is needed to stop a number cleanly and make sure that in all cases it will be terminated without expanding anything further nor inserting a blank space. In fact, TeX probably skips spaces after control words (multiletter control sequences) for the same reason (although that happens at a different stage of processing). The idea is that it should always be possible to "stop without inserting anything".

Example: \ifdigit below tests whether the argument is a digit. The space after #1 is required to stop TeX from expanding the first \expandafter prematurely. On the other hand, we don't want the space to end up in the output. So in that case at least, TeX's space gobbling is a feature.

\long\def\ifdigit#1{%
    \ifnum 9<1\string #1 %
        \expandafter\@firstoftwo
    \else
        \expandafter\@secondoftwo
    \fi}

eTeX's designers could decide that a space stops the \numexpr or \dimexpr, because they simply use TeX's procedures to read operands. And these procedures already remove a trailing space. eTeX does not see the space before + in \numexpr 1 + 2. It would have been technically possible to let \numexpr be stopped by two space tokens in a row, and gobble them, but that would be very difficult to input, since two space characters in a row are equivalent to one. Some other delimiter had to be chosen. It had to be non-expandable, preferably already there in the language, and it had better do nothing when used outside a \...expr situation... No need to add a new primitive: \relax was a natural choice.

Now consider a \compareint function defined as follows

\long\def\compareint#1#2#3{%
    \ifnum\numexpr#1\relax#2\numexpr#3\relax
        \expandafter\@firstoftwo
    \else
        \expandafter\@secondoftwo
    \fi}
\compareint{1+2}<{3-1}{Yes}{No}

In this particular case, leaving \relax behind would not be a problem. However, think of nesting this:

\def\X{345}\def\Y{534}
\compareint {132} < {\compareint{12}>{9}{\X}{\Y}+1} {Yes} {No}

If the \relax was not removed from the internal \compareint, it would stop expansion of the outside \compareint too early (actually, before expanding to \X or \Y, so that would be a problem).