Transferring millions of files from one server to another

As you say, use rsync:

rsync -azP /var/www/html/txt/ username@ip-address:/var/www/html/txt

The options are:

-a : enables archive mode, which preserves symbolic links and works recursively
-z : compress the data transfer to minimise network usage
-P : to display a progress bar and enables you to resume partial transfers

As @aim says in his answer, make sure you have a trailing / on the source directory (on both is fine too).

More info from the man page


Just use rsync over ssh!

rsync -av username@ip:/var/www/html/txt /var/www/html/

From the man page:

-a, --archive : This is equivalent to -rlptgoD. It is a quick way of saying you want recursion and want to preserve almost everything (with -H being a notable omission). The only exception to the above equivalence is when --files-from is specified, in which case -r is not implied.

-v, --verbose : This option increases the amount of information you are given during the transfer. By default, rsync works silently. A single -v will give you information about what files are being transferred and a brief summary at the end. Two -v options will give you information on what files are being skipped and slightly more information at the end. More than two -v options should only be used if you are debugging rsync.

Note how did I use slashes at the end of the folders - it's important.