Can't remove file from git commit

Thanks to Chris I was able to fix this by running the following:

git filter-branch -f --index-filter 'git rm --cached --ignore-unmatch pi/data/node-login.0'

git filter-branch -f --index-filter 'git rm --cached --ignore-unmatch pi/data/node-login.1'

git filter-branch -f --index-filter 'git rm --cached --ignore-unmatch pi/data/local.0'

git filter-branch -f --index-filter "git rm -rf --cached --ignore-unmatch FOLDERNAME" -- --all

replace FOLDERNAME with the file or folder you wish to remove from the given git repository.

This worked for me:

git rm --cached name_of_a_giant_file
git rm --cached name_of_another_giant_file
git commit --amend -CHEAD
git push

Source: Github Help: Working with large files