Python Subprocess usage summary

Preamble
import subprocess
cmd = ['command', 'option1', 'option2', '...']
Run a command, block until it finishes, and get the return code
returncode = subprocess.call(cmd)
Run a command, block until it finishes, and get its output to stdout and stderr and the return code
proc = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
stdout, stderr = proc.communicate()
returncode = proc.returncode
Run a command, do not block but read the output from the process, and get the return code when it finishes
proc = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)

import sys
output_text = ''
while True:
	output = proc.stdout.read(1)
	if output == '': break
	sys.stdout.write(output)
	output_text += output

returncode = proc.wait()
Run a command, and send the input to the process
input_text = 'some input to the process'
proc = subprocess.Popen(cmd, stdin=PIPE)
proc.communicate(input=input_text)
Advertisements
Python Subprocess usage summary

install matplotlib on mac

If it fails to install matplotlib on mac because the compiler cannot locate some freetype header files, and if you don’t have freetype, then first install it using brew for instance:

brew install freetype

Odds are the compiler still complains when you try the installation second time.

The reason is that many libraries are installed under /usr/X11 on mac, and they are not correctly located at compile time due to the non-canonical location. If you’re using pkg-config, an easy way to use them is to include the package database there in /usr/X11 when locating headers and libraries using pkg-config. Export the following variable before compiling matplotlib:

export PKG_CONFIG_PATH=/usr/X11/lib/pkgconfig

then

pip install matplotlib
install matplotlib on mac

Some Python subprocess.Popen tips

subprocess.Popen non-blocking pipe I/O using an additional thread and an asynchronous queue:
http://stackoverflow.com/questions/3076542/how-can-i-read-all-availably-data-from-subprocess-popen-stdout-non-blocking/3078292#3078292

Why subprocess.Popen fails on Windows with a command having a Unicode filename within it: it internally uses CreateProcessA, which is the single-byte character version of CreateProcess, meaning that it won’t handle Unicode characters correctly. For this to be correctly handled, the command line, and hence the filename contained, needs to be encoded with the file system encoding, that is usually MBCS.
http://stackoverflow.com/questions/1910275/unicode-filenames-on-windows-with-python-subprocess-popen/9113914#9113914
This thread and the pep below describe exactly this problem as well as the workaround above mentioned.
http://stackoverflow.com/questions/2595448/unicode-filename-to-python-subprocess-call/11442604#11442604
http://www.python.org/dev/peps/pep-0277/

You will need to specify the encoding of the Python file if you use Unicode characters there: most simply add # encoding=utf-8 as the first or the second line of your source code although there are some more degrees of freedom, which can be found here.
http://www.python.org/dev/peps/pep-0263/

Some Python subprocess.Popen tips