Using Python and C Together

Kategorilenmemiş

If we have an algorithm that goes really hard on the cpu, or we want to make some low level things we can implement the related area with C, and then access it from Python.

The first part of the article will cover how to call a C function from Python with the help of ctypes. In the second part, we will cover  how to run Python code from C side -sea side :P-.

First, we need to prepare out C files. Assume that we have a C function that does some sophisticated things. Related file:

heavyweight.c:

#include "heavyweight.h"

int do_some_work(Command command)
{
    fprintf(stderr, "Command: %s\n", command.cmd);
    /* Some heavy weight job done */
    return 2; // 2 is good
}

Also we have an header file which defines the structure that our C file uses.

heavyweight.h:

#ifndef HEAVYWEIGHTH
#define HEAVYWEIGHTH
#include <stdio.h>

typedef struct {
    char * cmd;
    int some_num;
} Command;

int do_some_work(Command command);
#endif

As you can see, there is a struct named Command, and a function which takes it as a parameter.

Now let’s prepare a simple make file to compile this files into a shared object

Makefile:

cc = gcc -g
all: heavyweight.o
    $(cc) -shared -o libhw.so heavyweight.o
sort.o: sort.c
    $(cc) -fPIC -c -o heavyweight.o heavyweight.c
clean:
    rm *.o *.so*

When we run ‘make’, we will have a so file. Our aim is to cover accessing C code from Python so C related topics are not our issue, we won’t cover them in this article

Now let’s call the do_some_work function that resides in the so file.

do_some_work.py:

#!/usr/bin/env python
from ctypes import *

class Command(Structure):
    _fields_ = [
        ('command', c_char_p),
        ('some_num', c_int)
    ]

command = Command('some command', c_int(6))
hw = CDLL('./libhw.so')
print hw.do_some_work(command)

Our Command class that we inherited from Structure class imitates to be the Command structure in our header file. As we can see, it has two variables named command and some_num. We use c_char_p for of char *, c_int for int type of C. You can see the other type mappings in the documentation of ctypes. Now we let hw object to access our so file with the help of CDLL, and call do_some_work function with the object that is an instance of Command.

Now let’s see how to call Python code from C code. To bring some fun into this part of the article, we will try to trace open and exit functions from executable files.

We will write a simple class, this class will add files that are being opened to a dictionary. This way, we can see which file is opened how many times. This codes are run under Max OS X by the way, under Linux you may change the function to be traced and the DYLD_INSERT_LIBRARIES with LD_PRELOAD. For Windows search the keyword AppInit_DLLs.

tracer.py:

#!/usr/bin/env python

class Tracer(object):
    def __init__(self):
        self.opened = {}
    def add_to_opened(self, path):
        self.opened[path] = self.opened.get(path, 0) + 1
    def get_stats(self):
        print self.opened

tracer = Tracer()

Now we should prepare a C file that contains same arguments and return type with open function that resides in libc, and make our library override the original one.

fake_fop.c:

#include <stdio.h>
#include <dlfcn.h>
#include <python2.7/Python.h>

void prepare()
{
    Py_Initialize();
    char *path = "tracer.py";
    FILE *fp = fopen(path, "r");
    PyRun_SimpleFile(fp, path);
}

void add_to_path(const char *path)
{
    char buf[256];
    snprintf(buf, sizeof buf, "%s%s%s", "tracer.add_to_opened(\'", path, "\')");
    PyRun_SimpleString(buf);
}
int open(const char *path, int oflag, ...)
{
    static int (*real_open) (const char *, int, ...) = NULL;
    if (!real_open){
        real_open = dlsym(RTLD_NEXT, "open");
        prepare();
    }
    va_list ap; 
    va_start (ap, oflag); 
    int resp = real_open(path, oflag, ap);
    va_end (ap);
    add_to_path(path);
    fprintf(stderr, "opened (\'%s\') \n", path);
    return resp;
}

void at_exit()
{
    PyRun_SimpleString("tracer.get_stats()");
    Py_Finalize();
}

void exit(int status)
{
    void (*real_exit)(int) = NULL;
    real_exit = dlsym(RTLD_NEXT, "exit");
    at_exit();
    real_exit(status);
}

First let’s describe the open function. We define same open function that resides in the original libc, this way the program that we are tracing will call our function instead. If the function is called the first time, we take the original open function ant assign to a function pointer. Then run tracer.py from prepare function. Each time open function is called, we call add_to_path. This function adds the file path to our Tracer object’s dictionary.

Our statistics should be shown when the program finishes, but we can not add code into the program because it is an executable. So we override exit function this time. In the at_exit function we run get_stats method of our tracer object and show stats.

Lastly, let’s create a sh file that converts the C file that we wrote into a so file, makes that so file override the libc, then run the program that we want to trace. In this example we run file command that shows types of the files that are given as a parameter to the program.

try.sh:

gcc -dynamiclib -o fake_fop.so fake_fop.c -lpython2.7
export DYLD_INSERT_LIBRARIES=./fake_fop.so
export DYLD_FORCE_FLAT_NAMESPACE=1
file fake_fop.c fake_fop.c dene.sh

When we run this file, fake_fop.c is compiled and fake_fop.so is created. Then fake_fop.so overrides libc. Then we run file command and see which files it opens.

Output:

opened ('/usr/share/file/magic.mgc') 
opened ('fake_fop.c') 
fake_fop.c: ASCII c program text
opened ('fake_fop.c') 
fake_fop.c: ASCII c program text
opened ('dene.sh') 
dene.sh: ASCII text
{'/usr/share/file/magic.mgc': 1, 'dene.sh': 1, 'fake_fop.c': 2}

Bu yazı toplamda 2517, bugün ise 1 kez görüntülenmiş

4 Comments
  • Rustam

    helal olsun, Huseyin!)

    • admin

      sagol :))

    • huseyinalb

      sagol :))

  • http://twitter.com/emresavas Emre Savas

    Bu yazı toplamda 31, bugün ise 31 kez görüntülenmiş.