How to configure additional modules to nginx after installation?

Yes, you have to uninstall nginx (installed via rpm) and re-install it according to the mentioned document that is from source file. There are some disadvantages of installing nginx using source, like you cannot use nginx as a service. Here, you can find instructions to do same thing with all the functionalities you get while installing nginx using OS-respective packages.

Unlike Apache, all modules, including the 3rd party modules, are going to be compiled into nginx. So every time you want to add a new module, you have to recompile nginx.

So yes, you have to install it as per the document. There is no much value of keeping 2 nginx runtimes on the same server any way. So you may also want to remove the previous nginx.

With nginx 1.9.11, it's not necessary to recompile the server, as they added support for dynamic modules. Take a look here: https://www.nginx.com/blog/dynamic-modules-nginx-1-9-11/

I had a similar problem where the auth-pam module broke after an upgrade. Here's what fixed it for me (debian stretch/sid, nginx 1.10.2):

apt install libnginx-mod-http-auth-pam
ln -s /usr/share/nginx/modules-available/mod-http-auth-pam.conf /etc/nginx/modules-enabled/50-mod-http-auth-pam.conf

The config file contains a single “load_module” directive which tells nginx to dynamically load the module on startup. As jekennedy mentioned, this would only apply to newer versions of nginx that support dynamic module loading.