Function callbacks in Python?

[neitsa]

Hello Adrien :)

This is rather tough issue (I think), so I'm obviously not asking for a quick resolution.

In DragonFFI, and AFAIK (correct me if I'm wrong), functions that have a callback (function pointer of some sort) need the callback to be in C.

Question

The question is simple, but I doubt the implementation would be (it seems rather complicated, I tried to understand how CPython is doing is but it looks complicated).

• Would it be technically possible to have python callbacks in DragonFFI, rather than compiled C callbacks?

---

DragonFFI

Below is a DragonFFI example with the CreateThread Windows API, which takes a function pointer as its 3rd parameter:

(note that I'm using some kind of wrapper around dragonFFI to make my life easier, but it's not hard to get).

thread_proc = """
#include <errhandlingapi.h>

DWORD WINAPI MyThreadProc(LPVOID lpParameter) 
{   
    SetLastError(0x1337);
    return 0;
}
"""

def main():
    # init stuff for dragonFFI wrapper
    include_base = pathlib.Path(r"C:\Program Files (x86)\Windows Kits\10\Include\10.0.19041.0")
    ffi = DragonFFI()
    ffi.include_bases = [include_base, ]
    ffi.set_includes(["windef.h", "processthreadsapi.h"])
    ffi.cdef(additional_text=thread_proc)

    nullptr = ffi.types.LPVOID()

    # pointer to thread function.
    pthreadproc = pydffi.ptr(ffi.funcs.MyThreadProc)

    ffi.funcs.SetLastError(0x42)

    input("press <enter> to call CreateThread.")
    err = ffi.funcs.GetLastError()
    print(f"LastErr (windows): {err.value:#x}")

    # call thread
    ffi.funcs.CreateThread(nullptr, 0, pthreadproc, nullptr, 0, nullptr)
    dffi_err = pydffi.getLastError()
    print(f"LastErr (pydffi): {dffi_err:#x}")

Debugged it with a native debugger, and everything's working as expected.

---

Python ctypes

Python ctypes has the possibility to wrap a python function so it can be called by a ctypes function. Below is a slightly different example, but still with CreateThread (see the wrapped function pointer called LPTHREAD_START_ROUTINE):

def my_thread(parameter):
    k32 = ctypes.WinDLL("kernel32", use_last_error=True)
    get_current_thread_id = k32.GetCurrentThreadId
    get_current_thread_id.argtypes = ()
    get_current_thread_id.restype = ctypes.c_uint32

    tid = get_current_thread_id()
    print(f"[THREAD] I'm a thread! My TID is: {tid:#x}")

    # wait a bit.
    sleep(1.0)

    print("[THREAD] Exiting.")

    return 0


def test_ctypes():
    k32 = ctypes.WinDLL("kernel32", use_last_error=True)

    # LPTHREAD_START_ROUTINE passed as 3rd arg to CreateThread.
    LPTHREAD_START_ROUTINE = ctypes.WINFUNCTYPE(
        ctypes.c_uint32,  # return type
        ctypes.c_void_p  # param type.
    )

    # CreateThread
    create_thread = k32.CreateThread
    create_thread.argtypes = (
        ctypes.c_void_p,
        ctypes.c_size_t,
        LPTHREAD_START_ROUTINE,  # callback
        ctypes.c_void_p,
        ctypes.c_uint32,
        ctypes.POINTER(ctypes.c_uint32)
    )
    create_thread.restype = ctypes.c_void_p

    # call CreateThread
    callback_func = LPTHREAD_START_ROUTINE(my_thread)
    tid = ctypes.c_uint32()
    create_thread(None, 0, callback_func, None, 0, ctypes.byref(tid))

    print(f"[MAIN] created TID: {tid.value:#x}")

    sleep(2.0)

    print("[MAIN] Exiting")

    return 0

---

Some pointers

• Python implementation of WINFUNCTYPE ; seems to be a wrapper around _CFuncPtr.
• _CFuncPtr defined as from _ctypes import CFuncPtr as _CFuncPtr at the top of ctypes.py.
• CFuncPtr defined here in ctypes.c.

I'm not sure of the whole inner workings machinery behind CFuncPtr...

---

Sorry for the looooong issue, Thanks a lot!