parenthesis around underbrace too big

this is overkill for a 2-by-2 matrix, but it would work for larger matrices as well, even for ones where big versions of parentheses aren't available.

\documentclass{article}
\usepackage{amsmath}
\begin{document}
$\exp \left( {\vphantom{\begin{pmatrix} 1\\ 3 \end{pmatrix}}} \right. \!\!
\gamma \underbrace{\!\! \begin{pmatrix} 1 & 2\\
 3 & 4 \end{pmatrix} \!\!}_\text{some text} \!\!
\left. {\vphantom{\begin{pmatrix} 1\\ 3 \end{pmatrix}}} \right)$
\end{document}

observe the use of \! to get rid of some unwanted horizontal space.

Maybe I shouldn't have posted this but I'm reviewing a lot of papers involving huge matrices and I can't stand inline matrices. If needed use the smallmatrix environment or state the matrix exponential as $e^{\gamma A}$ and put A matrix as a display math.

\documentclass{article}
\usepackage{mathtools} %<- Builds on top of amsmath and fixes/improves some stuff
\begin{document}    
The sentences start like this then slowly they get longer and longer and then sentence
becomes like this 
$e^{\gamma\!\! 
\underbracket[0.4pt][1pt]{
\begin{psmallmatrix} 1 & 2\\ 3 & 4 \end{psmallmatrix}
}_{\text{\tiny some text}} }$ 
and maybe after that something like 
\raisebox{0.5ex}{%
$\exp{(\gamma\!\! \underbracket[0.4pt][1pt]{
\begin{psmallmatrix} 1 & 2\\ 3 & 4 \end{psmallmatrix}}_{\text{\tiny some text}} )}
$
}
then continues with more stuff that starts another line which still looks terrible.
However this one 
\[
e^{\left(\gamma\!\! \underbracket[0.4pt][1pt]{
\begin{psmallmatrix} 1 & 2\\ 3 & 4 \end{psmallmatrix}
}_{\text{\tiny some text}} \right)} = 
\exp{(\gamma\!\! \underbracket[0.4pt][1pt]{
\begin{psmallmatrix} 1 & 2\\ 3 & 4 \end{psmallmatrix}
}_{\text{\tiny some text}} )}
\]
has no such problems. But still none of them works properly
in my opinion. The best is to go with $e^{\gamma A}$ where 
\[
A =\begin{pmatrix}
     1&2\\3&4
\end{pmatrix}
\]
\end{document}

If you need it only once, I would go for a manual adjustment:

\documentclass{article}
\usepackage{amsmath}
\begin{document}    
$\exp
\mkern30mu
\underbrace{
\mkern-30mu
\left(
\gamma
\begin{pmatrix} 1 & 2\\ 3 & 4 \end{pmatrix}
\right)
\mkern-18mu
}_\text{some text}
\mkern18mu
$
\end{document}

I put the \underbrace outside, and using \mkern with positive and negative value I made it think that its contents is smaller. The values 30mu and 18mu were found by trial-and-error.