Where did CUDA get installed in my computer?

Usually, it is /usr/local/cuda. If this is not the case, you can try to locate cuda. If you want to find directories only, run

locate cuda | grep /cuda$

or

find / -type d -name cuda 2>/dev/null

For me, it turned out to be in /opt/cuda-7.5

I solved (ditched actually) the problem by using 'Runfile method' for installing. I could get the latest nvidia driver installed with the package method explained above, but the problem seemed to be the cuda toolkit.

Installing the driver through Runfile is pain. So, at the prompt I chose to install just 'cuda toolkit 7.5' and all the files got copied to /usr/local/cuda* directory properly.

I tracked the CUDA installation folder to /usr/lib/nvidia-cuda-toolkit. How? I used locate nvcc.

I had installed NVIDIA driver using the Software and Updates --> Additional Drivers followed by CUDA Toolkit installation using sudo apt install nvidia-cuda-toolkit on Ubuntu 20.04. nvcc --version was working fine but when it came to verifying cuDNN installation (https://docs.nvidia.com/deeplearning/cudnn/install-guide/index.html#verify), it was looking for the usr/local/cuda folder and failed.

I, therefore, created a symlink usr/local/cuda to /usr/lib/nvidia-cuda-toolkit as so: ln -s /usr/lib/nvidia-cuda-toolkit/ /usr/local/cuda.

Also added /usr/lib/nvidia-cuda-toolkit/libdevice to $LD_LIBRARY_PATH and /usr/lib/nvidia-cuda-toolkit/bin to$PATH variables.

All is working fine now.