setting the default gpu

OK, I figured it out.

First try creating an xorg config using the following command:

sudo nvidia-xconfig

this will create a new xorg config at /etc/X11/xorg.conf.

then change the device section to look something like below:

Section "Device"
    Identifier     "Device1"
    Driver         "nvidia"
    VendorName     "NVIDIA Corporation"
    BusID          "PCI:2:0:0"
EndSection

Section "Screen"
    Identifier     "Screen0"
    Device         "Device1"
    Monitor        "Monitor0"
    DefaultDepth    24
    SubSection     "Display"
        Depth       24
    EndSubSection
EndSection

IMPORTANT: make sure your monitor is connected to the second GPU. reboot and voila!

I have two GPUs. Here is what I did to set the GPU2 for display:

To get PCI in hex format, use:

lspci | grep VGA

For me, it returns:

05:00.0 VGA compatible controller: NVIDIA Corporation Device 1b02 (rev a1)
09:00.0 VGA compatible controller: NVIDIA Corporation GM200 [GeForce GTX TITAN X] (rev a1)

So, in my case it is 09:00.0 for the GPU I wish to use for display (GPU2). I then generated the /etc/X11/xorg.conf using sudo nvidia-xconfig.

this will create a new xorg config at /etc/X11/xorg.conf.

Then I only added the BusID line in the following part of the file /etc/X11/xorg.conf:

Section "Device"
    Identifier     "Device0"
    Driver         "nvidia"
    VendorName     "NVIDIA Corporation"
    BusID          "PCI:9:0:0"
EndSection

I then restarted the computer and connected the monitors to the GPU2. After this few of the USB ports stopped working but others were fine to be used for keyboard and mouse.