Vim Regex Capture Groups [bau -> byau : ceu -> cyeu]

One way to fix this is by ensuring the pattern is enclosed by escaped parentheses:

:%s/\(\w\)\(\w\w\)/\1y\2/g

Slightly shorter (and more magic-al) is to use \v, meaning that in the pattern after it all ASCII characters except '0'-'9', 'a'-'z', 'A'-'Z' and '_' have a special meaning:

:%s/\v(\w)(\w\w)/\1y\2/g

See:

• :help \(
• :help \v

If you don't want to escape the capturing groups with backslashes (this is what you've missed), prepend \v to turn Vim's regular expression engine into very magic mode:

:%s/\v(\w)(\w\w)/\1y\2/g