Prettier Struct Definitions for Python CTypes



I was writing some Python code that uses [ctypes][] for interfacing with the [Windows ToolHelp API](https://docs.microsoft.com/en-us/windows/win32/api/_toolhelp/). Specifically, I had to [define a Python equivalent][structs] for the [PROCESSENTRY32](https://docs.microsoft.com/en-us/windows/win32/api/tlhelp32/ns-tlhelp32-processentry32) struct. Now, the [ctypes][] [struct definitions][structs] are a little annoying because they do not give you type hints. You can add the type hints _on top_ of the _fields_ attribute but that looks a little silly because you _literally_ [write everything twice][wet]. In Python 3.7+ ((Starting in Python 3.7, dictionaries keep the insertion order.)), you can solve this using a metaclass, and one stigma of metaclasses is that they have very few applications, so I decided to blog about this one:
class FieldsFromTypeHints(type(ctypes.Structure)):
    def __new__(cls, name, bases, namespace):
        from typing import get_type_hints
        class AnnotationDummy:
            __annotations__ = namespace.get('__annotations__', {})
        annotations = get_type_hints(AnnotationDummy)
        namespace['_fields_'] = list(annotations.items())
        return type(ctypes.Structure).__new__(cls, name, bases, namespace)
and now you can write:
class PROCESSENTRY32(ctypes.Structure, metaclass=FieldsFromTypeHints):
    dwSize              : ctypes.c_uint32
    cntUsage            : ctypes.c_uint32
    th32ProcessID       : ctypes.c_uint32
    th32DefaultHeapID   : ctypes.POINTER(ctypes.c_ulong)
    th32ModuleID        : ctypes.c_uint32
    cntThreads          : ctypes.c_uint32
    th32ParentProcessID : ctypes.c_uint32
    pcPriClassBase      : ctypes.c_long
    dwFlags             : ctypes.c_uint32
    szExeFile           : ctypes.c_char * 260
The metaclass simply deduces the _fields_ attribute of the new class for you before the class is _even created_. It might make sense to think of metaclasses as "class creation hooks", i.e. you get to modify what you have written before the class is actually being defined. The following bit is just to be compatible with [PEP-563](https://www.python.org/dev/peps/pep-0563/):
        from typing import get_type_hints
        class AnnotationDummy:
            __annotations__ = namespace.get('__annotations__', {})
        annotations = get_type_hints(AnnotationDummy)
And then, the following is the actual magic line:
        namespace['_fields_'] = list(annotations.items())
This adds the attribute _fields_ before the class is created. [ctypes]: https://docs.python.org/3/library/ctypes.html [structs]: https://docs.python.org/3/library/ctypes.html#structures-and-unions [wet]: https://en.wikipedia.org/wiki/Don%27t_repeat_yourself ### The Code Since you clicked, I'll share with you the code that comes next:
def get_parent_processes():
    k32 = ctypes.windll.kernel32
    entry = PROCESSENTRY32()
    entry.dwSize = ctypes.sizeof(PROCESSENTRY32)
    snap = k32.CreateToolhelp32Snapshot(2, 0)
    if not snap:
        raise RuntimeError('could not create snapshot')
    try:
        if not k32.Process32First(snap, ctypes.byref(entry)):
            raise RuntimeError('could not iterate processes')
        processes = {
            entry.th32ProcessID: (
                entry.th32ParentProcessID,
                bytes(entry.szExeFile).decode('latin1')
            ) for _ in iter(
                functools.partial(k32.Process32Next, snap, ctypes.byref(entry)), 0)
        }
    finally:
        k32.CloseHandle(snap)
    pid = os.getpid()
    while pid in processes:
        pid, path = processes[pid]
        yield path
This code gives me a list of the executable names of the process ancestors of the current Python process. I needed this because I had to figure out whether I am running inside PowerShell or inside a Command Prompt. Why, you ask? In essence, because [#1908](https://github.com/PowerShell/PowerShell/issues/1908) and [#559](https://github.com/PowerShell/PowerShell/issues/559) are turning my [binary refinery](https://github.com/binref/refinery/) toolkit into a complete shit show under PowerShell, and I am currently looking for a workaround that I can implement on my end.

3 Replies to “Prettier Struct Definitions for Python CTypes”

  1. I just realized that it is very important for this whole thing that I am in Python 3.7+, where [dictionaries keep insertion order](https://mail.python.org/pipermail/python-dev/2017-December/151283.html). Otherwise, using annotations could mess up the order of the struct fields.
    1. Hi! I've written more or less similar implementation of such metaclass, but a bit simpler: https://github.com/dfint/dfrus/blob/develop/dfrus/ctypes_annotated_structure.py Not sure if it is better, or I am missing some important details (I'm a newbie in metaclasses). As for insertion order, Python 3.6 too keeps insertion order of dictionaries (see https://stackoverflow.com/a/39980744/4752653). For earlier versions, I think it's possible to add __prepare__ class method to the metaclass that would return some object as an initial value of the namespace, that converts any dict inserted into the namespace into an OrderedDict.
  2. That whole process list thing, or any Win32 API situation where you have to do CloseHandle type cleanup should probably be a [context manager](https://docs.python.org/3/reference/datamodel.html#context-managers). My application is still very specific, so I think I'll keep the code as-is, though.

Leave a Reply to insolor Cancel reply

Your email address will not be published. Required fields are marked *