Prettier Struct Definitions for Python CTypes



I was writing some Python code that uses [ctypes][] for interfacing with the [Windows ToolHelp API](https://docs.microsoft.com/en-us/windows/win32/api/_toolhelp/). Specifically, I had to [define a Python equivalent][structs] for the [PROCESSENTRY32](https://docs.microsoft.com/en-us/windows/win32/api/tlhelp32/ns-tlhelp32-processentry32) struct. Now, the [ctypes][] [struct definitions][structs] are a little annoying because they do not give you type hints. You can add the type hints _on top_ of the `_fields_` attribute but that looks a little silly because you _literally_ [write everything twice][wet]. In Python 3.7+ ((Starting in Python 3.7, dictionaries keep the insertion order.)), you can solve this using a metaclass, and one stigma of metaclasses is that they have very few applications, so I decided to blog about this one: ```python class FieldsFromTypeHints(type(ctypes.Structure)): def __new__(cls, name, bases, namespace): from typing import get_type_hints class AnnotationDummy: __annotations__ = namespace.get('__annotations__', {}) annotations = get_type_hints(AnnotationDummy) namespace['_fields_'] = list(annotations.items()) return type(ctypes.Structure).__new__(cls, name, bases, namespace) ``` and now you can write: ```python class PROCESSENTRY32(ctypes.Structure, metaclass=FieldsFromTypeHints): dwSize : ctypes.c_uint32 cntUsage : ctypes.c_uint32 th32ProcessID : ctypes.c_uint32 th32DefaultHeapID : ctypes.POINTER(ctypes.c_ulong) th32ModuleID : ctypes.c_uint32 cntThreads : ctypes.c_uint32 th32ParentProcessID : ctypes.c_uint32 pcPriClassBase : ctypes.c_long dwFlags : ctypes.c_uint32 szExeFile : ctypes.c_char * 260 ``` The metaclass simply deduces the `_fields_` attribute of the new class for you before the class is _even created_. It might make sense to think of metaclasses as "class creation hooks", i.e. you get to modify what you have written before the class is actually being defined. The following bit is just to be compatible with [PEP-563](https://www.python.org/dev/peps/pep-0563/): ```python from typing import get_type_hints class AnnotationDummy: __annotations__ = namespace.get('__annotations__', {}) annotations = get_type_hints(AnnotationDummy) ``` And then, the following is the actual magic line: ```python namespace['_fields_'] = list(annotations.items()) ``` This adds the attribute `_fields_` before the class is created. [ctypes]: https://docs.python.org/3/library/ctypes.html [structs]: https://docs.python.org/3/library/ctypes.html#structures-and-unions [wet]: https://en.wikipedia.org/wiki/Don%27t_repeat_yourself <span id="more-5700"></span> ### The Code Since you clicked, I'll share with you the code that comes next: ```python def get_parent_processes(): k32 = ctypes.windll.kernel32 entry = PROCESSENTRY32() entry.dwSize = ctypes.sizeof(PROCESSENTRY32) snap = k32.CreateToolhelp32Snapshot(2, 0) if not snap: raise RuntimeError('could not create snapshot') try: if not k32.Process32First(snap, ctypes.byref(entry)): raise RuntimeError('could not iterate processes') processes = { entry.th32ProcessID: ( entry.th32ParentProcessID, bytes(entry.szExeFile).decode('latin1') ) for _ in iter( functools.partial(k32.Process32Next, snap, ctypes.byref(entry)), 0) } finally: k32.CloseHandle(snap) pid = os.getpid() while pid in processes: pid, path = processes[pid] yield path ``` This code gives me a list of the executable names of the process ancestors of the current Python process. I needed this because I had to figure out whether I am running inside PowerShell or inside a Command Prompt. Why, you ask? In essence, because [#1908](https://github.com/PowerShell/PowerShell/issues/1908) and [#559](https://github.com/PowerShell/PowerShell/issues/559) are turning my [binary refinery](https://github.com/binref/refinery/) toolkit into a complete shit show under PowerShell, and I am currently looking for a workaround that I can implement on my end.

3 Replies to “Prettier Struct Definitions for Python CTypes”

  1. I just realized that it is very important for this whole thing that I am in Python 3.7+, where [dictionaries keep insertion order](https://mail.python.org/pipermail/python-dev/2017-December/151283.html). Otherwise, using annotations could mess up the order of the struct fields.
    1. Hi! I've written more or less similar implementation of such metaclass, but a bit simpler: https://github.com/dfint/dfrus/blob/develop/dfrus/ctypes_annotated_structure.py Not sure if it is better, or I am missing some important details (I'm a newbie in metaclasses). As for insertion order, Python 3.6 too keeps insertion order of dictionaries (see https://stackoverflow.com/a/39980744/4752653). For earlier versions, I think it's possible to add __prepare__ class method to the metaclass that would return some object as an initial value of the namespace, that converts any dict inserted into the namespace into an OrderedDict.
  2. That whole process list thing, or any Win32 API situation where you have to do `CloseHandle` type cleanup should probably be a [context manager](https://docs.python.org/3/reference/datamodel.html#context-managers). My application is still very specific, so I think I'll keep the code as-is, though.

Leave a Reply

Your email address will not be published. Required fields are marked *