Combinatorial notation (\choose) not working with \frac

This is one of the reasons why using the \choose method is not the best.

What happens? When \frac is expanded, it looks for two arguments, which in your case are determined to be

{200 \choose 11}{ 200 \choose {N-189}}

and

200 \choose N

Then TeX will transform this into the primitive call

{\begingroup<first argument>\endgroup\over<second argument>}

which in your case is

{\begingroup{200 \choose 11}{ 200 \choose {N-189}}\endgroup\over 200 \choose N}

which is indeed ambiguous as the error message says: it's like

a \over b \over c

which is of course invalid.

Do yourself a favor and use

\documentclass{article}
\usepackage{amsmath}

\begin{document}

$p_N=\frac{\binom{200}{11} \binom{200}{N-189}}{\binom{200}{N}}$

\end{document}

which is clearer and doesn't require extra braces like

$p_N= \frac{{200 \choose 11}{ 200 \choose {N-189}}}{{200 \choose N}}$

By the way, the braces around N-189 are redundant. But, really, use the amsmath method that automatically supplies the internal braces when expanding things to the primitive calls.

I have seen often \choose in MathJaX and if I not remember not good also in Open Office or LibreOffice. For my humble opinion are important to have a full minimal working example to see the mistakes. I use often $\binom{n}{k}$.

\documentclass[a4paper,12pt]{article}
\usepackage{mathtools,amssymb,xcolor}


\begin{document}
\[p_N= \frac{\binom{200}{11}\binom{200}{N}}{\binom{200}{N-189}}\]
\end{document}   

---