Google Colab is very slow compared to my PC

It's very slow to read file from google drives.

For example, I have one big file(39GB).

It cost more than 10min when I exec '!cp drive/big.file /content/'.

After I shared my file, and got the url from google drive. It cost 5 min when I exec '! wget -c -O big.file http://share.url.from.drive'. Download speed can up to 130MB/s.


Reading files from google drive slow down your training process. The solution is to upload zip file to colab and unzip there. Hope it is clear for you.


As @Feng has already noted, reading files from drive is very slow. This tutorial suggests using some sort of a memory mapped file like hdf5 or lmdb in order to overcome this issue. This way the I\O Operations are much faster (for a complete explanation on the speed gain of hdf5 format see this).