using parentheses in xpath/xslt

See @Martin's answer for the key point: that valid patterns are only a subset of valid XPath expressions. In other words, there are many XPath expressions that cannot be used as patterns. (This is something about XSLT that it took me a long time to realize.)

As for valid alternatives:

//(X|Y|Z)/AABBCC

is a valid expression in XPath 2.0, but not in 1.0, because the parentheses cannot begin immediately after the // axis. But in 1.0,

(//X|//Y|//Z)/AABBCC

is a valid alternative expression (but still not a valid pattern). A valid but somewhat awkward pattern would be

*[contains('X Y Z', local-name())]/AABBCC

or

*[self::X | self::Y | self::Z]/AABBCC

As for

(book/author)[last()]

a valid pattern would be

(book/author)[not(following::author[parent::book])]

(But of course

(book/author)[not(following::book/author)]

would not be equivalent, because it would match all <author> children of the last <book> that had any.)

You have to understand that a match attribute of an xsl:template does not allow any XPath expression but rather only so called patterns: https://www.w3.org/TR/1999/REC-xslt-19991116#patterns, a subset of XPath expressions.

So while (book/author)[last()] is a syntactically correct XPath 1.0 expression I don't think it is a syntactically correct XSLT 1.0 pattern, the parentheses are not allowed.

I don't think //(X|Y|Z)/AABBCC is an allowed XPath 1.0 expression (nor a pattern of course) but match="X/AABBCC | Y/AABBCC | Z/AABBCC" should do.