How do I remove a second remote git repo from an Xcode project?

So since the correct answer was never added here I'll do it myself: you have to delete the Xcode's Derived Data for it to forget about the link once you remove the references to other git repo's source files in the projects.

The easiest way to do so is to go to Xcode -> Preferences enter the Locations tab and press the little arrow beside Derived Data location (in the red box):

Just delete the entire folder (it's good to do so every once in a while to free up the space) and you are good to go!

Try in the Terminal:

$ cd /my/project/folder
$ git remote

Now you know the names of the remotes.

$ git remote remove myUnwantedRemote

Of course all this is easy without Terminal if you use SourceTree.