Vim is still present and executable even after running apt remove and purge on its package

You uninstalled vim not vim. vim-tiny, vim-nox, vim-gtk, and vim-athena all provide vim compiled with different options.

To find the location of a program, use the whereis command

whereis vim

It should show you if and where it is installed.

Please also note: $VIMRUNTIME is only a directory holding mainly configuration files. Have a closer look here.

Based on answers by hildred/Fiximan, I used the following command and it worked:

sudo  apt-get purge --auto-remove vim-gtk3

I am using Ubuntu 20.04 LTS.