How to wrap a C pointer and length in a new-style buffer object in Cython?
You can define an extension type that implements the buffer protocol by defining the
__releasebuffer__ special methods. For example:
from cpython.buffer cimport PyBuffer_FillInfo from libc.stdlib cimport free, malloc from libc.string cimport memcpy cdef void dummy_function(const void **p, size_t *l): cdef void *tmp = malloc(17) memcpy(tmp, "some test\0 bytes", 17) p = tmp l = 17 cdef void free_dummy_data(const void *p, size_t l, void *arg): free(<void *>p) cpdef getbuf(): cdef const void *p cdef size_t l dummy_function(&p, &l) return MemBuf_init(p, l, &free_dummy_data, NULL) ctypedef void dealloc_callback(const void *p, size_t l, void *arg) cdef class MemBuf: cdef const void *p cdef size_t l cdef dealloc_callback *dealloc_cb_p cdef void *dealloc_cb_arg def __getbuffer__(self, Py_buffer *view, int flags): PyBuffer_FillInfo(view, self, <void *>self.p, self.l, 1, flags) def __releasebuffer__(self, Py_buffer *view): pass def __dealloc__(self): if self.dealloc_cb_p != NULL: self.dealloc_cb_p(self.p, self.l, self.dealloc_cb_arg) # Call this instead of constructing a MemBuf directly. The __cinit__ # and __init__ methods can only take Python objects, so the real # constructor is here. See: # https://mail.python.org/pipermail/cython-devel/2012-June/002734.html cdef MemBuf MemBuf_init(const void *p, size_t l, dealloc_callback *dealloc_cb_p, void *dealloc_cb_arg): cdef MemBuf ret = MemBuf() ret.p = p ret.l = l ret.dealloc_cb_p = dealloc_cb_p ret.dealloc_cb_arg = dealloc_cb_arg return ret
With the above (named
test.pyx) you get the following behavior:
$ python -c 'import test; print repr(memoryview(test.getbuf()).tobytes())' 'some test\x00 bytes\x00'
I don't know if there's an easier way.
Python 3.3 has
PyMemoryView_FromMemory C-API function, which creates a
memoryview Python object from supplied C buffer.
memoryview objects indeed implement new-style buffer interface.
If you look into its sources, you'll notice that they're rather simple.
It does that same thing as
PyMemoryView_FromBuffer does, except the former
Since the latter one exists in Python 2.7, so why we can't just call
from libc.stdlib cimport malloc from libc.string cimport memcpy cdef extern from "Python.h": ctypedef struct PyObject object PyMemoryView_FromBuffer(Py_buffer *view) int PyBuffer_FillInfo(Py_buffer *view, PyObject *obj, void *buf, Py_ssize_t len, int readonly, int infoflags) enum: PyBUF_FULL_RO cdef void dummy_function(const void **p, size_t *l): cdef void *tmp = malloc(17) memcpy(tmp, "some test\0 bytes", 17) p = tmp l = 17 cpdef getbuf(): cdef const void *cstr cdef size_t l cdef Py_buffer buf_info cdef char[:] ret cdef int readonly dummy_function(&cstr, &l) readonly = 1 PyBuffer_FillInfo(&buf_info, NULL, <void*>cstr, l, readonly, PyBUF_FULL_RO) ret = PyMemoryView_FromBuffer(&buf_info) return ret
Note that, however, that the returned value will have a repr that looks like this:
<MemoryView of 'memoryview' at 0x7f216fc70ad0>. This is because Cython seems to wrap bare
memoryview objects implement buffer interface already, you should probably simply return the result of
PyMemoryView_FromBuffer call instead.
Additionally, you're responsible for managing the lifetime of your buffer.
memoryview objects created this way will not free memory automatically. You must do it yourself, ensuring that you only do that once no
memorybuffer references it. In this regard, answer by Richard Hansen is much better alternative.
As @RichardHansen correctly observes in his self-answer, what you want is a class that implements the buffer protocol, and has a suitable destructor that manages the memory.
Cython actually provides a fairly lightweight class built into it in the form of
cython.view.array so there's no need to create your own. It's actually documented in the page you linked but for the sake of providing a quick example that fits your case:
# at the top of your file from cython.view cimport array # ... # after the call to dummy_function my_array = array(shape=(l,), itemsize=sizeof(char), format='b', # or capital B depending on if it's signed allocate_buffer=False) my_array.data = cstr my_array.callback_free_data = free cdef char[:] ret = my_array
Just to draw attention to a couple of bits:
allocate_buffer is set to
False since you're allocating your own in
callback_free_data ensures that the standard library
free function is used.