Why is it faster to transfer data from CPU to GPU rather than GPU to CPU?
This is a CW for anybody interested in posting benchmarks from their machine. Contributors are encouraged to leave their details in case some future question arises regarding their results.
System: Win10, 32GB DDR4-2400Mhz RAM, i7 6700K. MATLAB: R2018a.
Using a GeForce GTX 660 GPU.
Achieved peak send speed of 7.04747 GB/s
Achieved peak gather speed of 3.11048 GB/s
Warning: The measured time for F may be inaccurate because it is running too fast. Try measuring something that takes
longer.
Contributor: Dev-iL
System: Win7, 32GB RAM, i7 4790K. MATLAB: R2018a.
Using a Quadro P6000 GPU.
Achieved peak send speed of 1.43346 GB/s
Achieved peak gather speed of 1.32355 GB/s
Contributor: Dev-iL
I am not familiar with Matlab GPU toolboxes, but I suspect that the second transfer (that gets data back from GPU) starts before the first has ended.
% Time sending to GPU
sendFcn = @() gpuArray(hostData);
sendTimes(ii) = gputimeit(sendFcn);
%
%No synchronization here
%
% Time gathering back from GPU
gatherFcn = @() gather(gpuData);
gatherTimes(ii) = gputimeit(gatherFcn);
A similar question, for a C program, was posted here:
copy from GPU to CPU is slower than copying CPU to GPU
In that case, there is no explicit sync after launching a thread on the GPU and getting results data back from the GPU. So the function that gets data back, in C cudaMemcpy(), has to wait for the GPU to end the previous launched thread, before transferring data, thus inflating the time measured for the data transfer.
With the Cuda C API, it is possible to force the CPU to wait for the GPU to end the previously launched thread(s), with:
cudaDeviceSynchronize();
And only then start measuring the time to transfer data back.
Maybe in Matlab there is also some synchronization primitive.
Also in the same answer, it is recommended to measure time with (Cuda) Events.
In this POST on optimizing data transfers, also in C sorry, Events are used to measure data transfer times:
https://devblogs.nvidia.com/how-optimize-data-transfers-cuda-cc/
The time for transferring data is the same in both directions.