Cache entry deserialization failed, entry ignored
Regarding the error/warning message in the question's title:
Cache entry deserialization failed, entry ignored
You can fix it by removing the pip cache, e.g. on Ubuntu:
rm -rf ~/.cache/pip
It was a network issue. It worked totally fine when I moved to a different network.