How to run a subprocess with Python, wait for it to exit and get the full stdout as a string?

I am using the following construct, although you might want to avoid shell=True. This gives you the output and error message for any command, and the error code as well:

process = subprocess.Popen(cmd, shell=True,
                           stdout=subprocess.PIPE, 
                           stderr=subprocess.PIPE)

# wait for the process to terminate
out, err = process.communicate()
errcode = process.returncode

subprocess.check_output(...)

calls the process, raises if its error code is nonzero, and otherwise returns its stdout. It's just a quick shorthand so you don't have to worry about PIPEs and things.

If your process gives a huge stdout and no stderr, communicate() might be the wrong way to go due to memory restrictions.

Instead,

process = subprocess.Popen(cmd, shell=True,
                           stdout=subprocess.PIPE, 
                           stderr=subprocess.PIPE)

# wait for the process to terminate
for line in process.stdout: do_something(line)
errcode = process.returncode

might be the way to go.

process.stdout is a file-like object which you can treat as any other such object, mainly:

• you can read() from it
• you can readline() from it and
• you can iterate over it.

The latter is what I do above in order to get its contents line by line.