Chained/Cloned containers

[jamie-at-arkifi]

Per fedi, https://tacobelllabs.net/@astraluma/111904242641573392

I think it would be useful to have chained/cloned/inherited containers: basically, being able to have a container access the instances of another container without taking over it's cleanup.

This would allow more flexibility in using svcs in a way that reflects the lifecycle structure of the application. By which I mean, many applications have several lifecycles/scopes, and it would be good to have more options for integrating DI/IoC into that.

• A web app might have the app scope (eg config, database connection pool) and the request/job scope (eg, a specific db transaction, the current user)
• pytest has many nested scopes, reflecting the structure of the test suite

Basically, more scopes more good?

These could also replace singleton factories or registry-based value, since instead you could have a global root container, and could be a way to handle #26.

[jamie-at-arkifi]

I also like this for advanced features, like reloading settings by clearing/reloading values in the App-bound container.

[hynek]

I’ve been thinking about this quite a bit and if I read you correctly, the best, yet nontrivial move, would be to move away from registries as a first-class concept to containers all the way?

IOW the moment svcs is initialized, it creates its first root container that is then passed into the app/web integrations. This container could carry system-wide dependencies like configs or engines (fixing #34) and each request would create a child container with the lifetime of the request.

The full cache of a container could be a ChainMap of all parents’s caches and if the service is missing, it’s added to the local one.

When a parent container is closed, it also closes all of its child containers.

Child containers therefore need to know both about their parents for cache lookups and their children for cleanups.

I think, it would also make testing easier, since you can clone the real container and overwrite just the factory you want to change. No need to flush upstream caches.

Open questions

• it would be necessary to create certain services in the root container, otherwise your app ends up with countless Engines and load configuration on ever request. What is an unconfusing API for that?
• relatedly: is there a need for a special-cased root container?
• should it be possible to circumvent a parent’s service cache? Maybe inherit=True argument to register_factory?

I think this whole proposal is basically what @elfjes has been asking for from day 1? /o\

[hynek]

happy new year everyone 🙈

How would such an API work for y'all:

# Create parent container
parent = Container(registry)

# Create child container
child = parent.create_child()

# Child can access parent's services
parent.register_local_value(str, "parent")
assert child.get(str) == "parent"

# Child can override parent's services
child.register_local_value(str, "child")
assert child.get(str) == "child"
assert parent.get(str) == "parent"

# Closing parent closes children
parent.close()  # This also closes child

# Child can be closed independently
child2 = parent.create_child()
child2.close()  # Only closes child2

There is a bit of deviation in using a method to create a child compared to just passing a registry into containers (vs. registry.create_container()) but since we need a two-way relationship between parents and children, this is more idiomatic than a child container modifying the innards of its parents within its __init__.

Pls lmk what y'all think, I've procrastinated on this long enough, hope everyone hasn't given up already.

[hynek]

Side-note: going down this route will make it necessary to deal with race conditions, I'm afraid.

I think the safest way would be to store lock_s_ (one threading, one asyncio) on the root container since a child can go away but a parent cannot. As I write this it occurs to me that this also means that the lock can be passed into child containers.

[tuukkamustonen]

So is the idea that there is:

• There's one registry
• Nested containers

What if I call child.get(str):

1. If parent.get(str) was called beforehand, it would retrieve the value stored in the parent container?
2. If if it wasn't, then a new value would be instantiated and stored to the child container?
   • If after this I would call parent.get(str) would it instantiate a new value into parent container?

Pretty obvious (I guess) that register_local_value() would register a container-local value. But let's forget about that, what if you only use the usual Registry.register_factory as above, what would happen?

[hynek]

I mean that's exactly how scopes work, no? If you create a child, it means that:

1. you want to share something with the parent container (otherwise you could just create another unrelated container)
2. you want the new scope cleaned separately

Your example is good, because depending on whether or not the parent creates + caches a value, the child will reuse or create/cleanup which sounds weird – but exactly how scoping would work? And exactly what you'd want if you, for example have an http pool/database connection that is supposed to be used throughout a request and then build on that in smaller scopes?

If they wouldn't share the cache, what would be the point except inheriting local factories? Is there some authoritative prior art we could orient ourselves at?

[tuukkamustonen]

Yeah, just thought it would be confusing if there's no predefined way to know at what "level" the value was instantiated(?) That's why my previous example showed two different registries.

Wouldn't registering "local" values go away with inheritable containers? Ie. if you need an override, create a nested container and instead?

Maybe it would be best think this through an example use case. My assumption was that this could be used in web apps like:

def some_endpoint(svcs: Container):
    request = svcs.get(Request)

But then, for that example it would be a bit silly because Request is nothing to lazy load / cache, instead better just inject it and pass along:

def some_endpoint(request: Request):
    pass_it_somewhere_that_needs_it(request)

So... uh, what would be a good use case?