Can I use mypy plugins to type-check functions created at runtime?

[RedRoserade]

First, an explanation: I have two kinds of types in my project, for representing a data model:

• Types for validation and data representation (e.g., JSON), implemented as Pydantic models
• Types used at runtime, implemented either as bare Python classes, or with Python dataclasses

It allows decoupling between the two. However, I need functions that map between the two. Usually the types are the same, but sometimes some conversions are required.

I can write the mappings myself, but creating mappings for ~100 types, back and forth, means ~200 functions, mostly with code like this:

def map_a_to_b(a: A) -> B:
    return B(...)

Which is tedious, so, because most of the mappings are simple (or simple enough to compose), I started by creating something like this:

def map_to(value: T1, to_type: T2) -> T2:
  kwargs = {}

  for prop_name, value in get_values(value).items():
    kwargs[prop_name] = value

  return to_type(**kwargs)

This works fine, but it has non-negligible overhead, performance-wise (about 20-40% vs. a hand-written function). So, I decided to take a code generation approach, where I would inspect the types T1 and T2 at runtime, build the source code for the function at runtime, and then exec() it to get the function object out. The function is then created something like this:

map_a_to_b = create_mapper(A, B)

# map_a_to_b looks like (value: A) -> B

This works well, and it doesn't have the overhead of the dynamic version shown above, but I would like to have mypy type-check this generated code. I think I can use a mypy plugin for this, but I'm not sure if it is possible, or where I should start.

Ideally, what I want is to have the mypy plugin generate the same source code that the create_mapper function does, pass it to mypy, and propagate such errors to the overall error report, for every call to create_mapper.

[Daverball]

I don't see why you would need a plugin for this, unless you want to type check the generated code internally, I don't see the point. You can infer what the signature of the mapper function looks like from the types being passed into the generator function statically i.e. def create_mapper(type[T1], type[T2]) -> Callable[[T1], T2]: ...

Or is the problem that you loop through a list of type names to generate the mapper function, rather than having explicit calls to create_mapper? mypy can only really infer what is statically known at import time anyways, so your code generation would have to entirely happen at import time as well and at that point you might want to consider just adding a build script to the project that does the code generation based on the current source tree, and spits out a .py file with type annotations. That way you also don't add overhead to the application startup.

Sure it adds a little mental overhead where you have to now run that build script every time you change code that requires new mappers, but you could just add a pre-commit step to do it for you, before the type checker is run.

[RedRoserade]

My goal is to type check the source code (implementation) of the generated functions, to catch invalid assignments (e.g., assigning a str to an int).

If I generate the source code via a separate command, yes, mypy checks come "for free", though it's another step that needs doing, and then there's questions like:

• how do I find the places where I declare such mapping functions? because they can be anywhere. I can recurse though a module/directory, but reliably detecting and parsing such calls, unless I import the modules, might be tricky
• how can I ensure the code is in sync? ok, I can run the command again, or mypy might fail, but it's a step I'd like to avoid, and it could introduce sneaky bugs (e.g., adding an optional field on both sides, but not updating the mappers, mypy will pass but tests may fail with unexpected results)
• how/where do I put the generated function code, ensuring that I don't break other code, in case I decide to do it in-place?

Doing it on-demand, at runtime, happens pretty much transparently (despite the startup penalty), however, that's why I decided to go with this approach.

Mind you, I am not at all familiar with code generation in Python, this is something I'm exploring as I go, so suggestions and/or state-of-the-art are welcome.

[Daverball]

That's fair enough. I have some limited experience with mypy plugins and it's probably possible you can get this done with a mypy plugin, but it would be pretty annoying to write and would probably end up being quite fragile, on top of the challenges of reporting the errors in a way where they would be easy to trace back to their origin. I would personally recommend against it.

How about a pragmatic compromise? You could add ascript to the module which exports all the generated code into a file/string that can then be passed into mypy and type checked against the full module/package. You could add comments to the file to indicate where each function was generated, so you get more granular error reporting, i.e. you can see exactly which generated statement is faulty, but from the comment you can still trace back the function to its origin.

[hmc-cs-mdrissi]

If I had to do something like this with mypy I would not recommend a mypy plugin and instead recommend calling mypy in a subprocess after code is generated and just pass generated code ast/string to mypy dumped to some temp file. Mypy plugins are more for enhancing mypy when used statically not for using mypy at runtime.

Alternatively I'd recommend explore runtime type checking libraries like typeguard/beartype to add type assertions.