Pass many pieces of data from Python to C program

Pipes are the obvious way to go; if your c program accepts input from stdin, you can use Popen. This doesn't create a "thread" as you say in your edit; it creates an entirely new process with separate memory:

from subprocess import Popen, PIPE

input = "some input"
cproc = Popen("c_prog", stdin=PIPE, stdout=PIPE)
out, err = cproc.communicate(input)

Here's a more detailed example. First, a simple c program that echoes stdin:

#include<stdio.h>
#include<stdlib.h>
#define BUFMAX 100

int main() {
    char buffer[BUFMAX + 1];
    char *bp = buffer;
    int c;
    FILE *in;
    while (EOF != (c = fgetc(stdin)) && (bp - buffer) < BUFMAX) {
        *bp++ = c;
    }
    *bp = 0;    // Null-terminate the string
    printf("%s", buffer);
}

Then a python program that pipes input (from argv in this case) to the above:

from subprocess import Popen, PIPE
from sys import argv

input = ' '.join(argv[1:])
if not input: input = "no arguments given"
cproc = Popen("./c_prog", stdin=PIPE, stdout=PIPE)
out, err = cproc.communicate(input)
print "output:", out
print "errors:", err

If you don't plan to use the c program without the python frontend, though, you might be better off inlining a c function, perhaps using instant.

from instant import inline
c_code = """
    [ ... some c code ... ] //see the below page for a more complete example.
"""
c_func = inline(c_code)

As Joe points out, you could also write a python module in c: Extending Python with C or C++

This answer discusses other ways to combine c and python: How do I connect a Python and a C program?

EDIT: Based on your edit, it sounds like you really should create a cpython extension. If you want some example code, let me know; but a full explanation would make for a unreasonably long answer. See the link above (Extending Python...) for everything you need to know.


If your operating system supports it, named pipes are a drop in replacement for files.