copying one file's contents to another in python

Please note that the shutil module also contains copyfileobj(), basically implemented like Barmar's answer.

Or, to answer your question:

from shutil import copyfileobj

with open('file2.txt', 'wb') as output, open('file.txt', 'rb') as input:
    copyfileobj(input, output)

would be my suggestion. It avoids re-implementing the buffering mechanism and, should the implementation of the standard library improve, your code wins as well.


On Unix, there also is a non-standardised syscall called sendfile. It is used mostly for sending data from an open file to a socket (serving HTTP requests, etc.).

Linux allows using it for copying data between regular files as well though. Other platforms don't, check the Python doc and your man pages. By using a syscall the kernel copies the content without the need of copying buffers to and from userland.

The os module offers os.sendfile() since Python 3.3. You could use it like:

import io
import os

with open('file2.txt', 'wb') as output, open('file.txt', 'rb') as input:
    offset = 0 # instructs sendfile to start reading at start of input
    input_size = input.seek(0, io.SEEK_END)
    os.sendfile(output.fileno(), input.fileno(), offset, input_size)

Otherwise, there is a package on PyPi, pysendfile, implementing the syscall. It works exactly as above, just replace os.sendfile with sendfile.sendfile (and import sendfile).


The only potential problem with your output.write(input.read()) version is if the size of the file is too large to hold all of it in memory. You can use a loop that reads smaller batches.

with open('file2.txt', 'wb+') as output, open('file.txt', 'rb') as input:
    while True:
        data = input.read(100000)
        if data == '':  # end of file reached
            break
        output.write(data)

This will work for both text and binary files. But you need to add the b modifier to the modes for portable operation on binary files.


While this may not completely answer your question, but for plain copying without any other processing of file contents, you should consider other means, e.g. the shutil module:

shutil.copy('file.txt', 'file2.txt')

Tags:

Python