-
Notifications
You must be signed in to change notification settings - Fork 13
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Interpolation and Accessing environment variables and other resolvers inside the config.yaml similar to OmegaConf #243
Comments
Just thinking out loud here which might help you grok some of the internals of spock so you can contribute if you're up for it... BasicsMost of the config values get set in this file here These field handlers reconcile values set in the class definition or those specified in the config files by building a dictionary and then passing that to the actual instance constructor to instantiate the actual object (this is where attrs is handling type checking, nested types, etc. via the validators functionality). There are different 'types' of resolvers as some fundamental `types' need to be handled differently (e.g. list or tuple is very different than int from a python type perspective) to make sure nesting, class references, etc. work correctly These field handlers are called by iterating through the dependency graph of classes here as there can be references to other spock classes, thus using a graph makes sure parent-child relationships (via a directed graph) are respected in the correct order. So here are my thoughts for the requested features (fyi: we should probably break these into at least 2 features/PRs since I think they will be pretty independent of each other) Environment Resolver (Easier to implement)Within the field handlers we simply need to regex for the syntax we devise (probably something similar to OmegaConf but shorter) such as All this should be able to be done within the field handlers (I think at the base level i.e. not type specific) right before the assembled dictionary gets passed to the class object instantiation. Variable Resolver (Harder to implement)Called 'config node interpolation' in OmegaConf... First thought here is that we will need to build another dependency graph between value references (fyi there is already a graph class to handle this here) to make sure we can instantiate objects in the correct order (again via a directed graph). Then when we discover a reference in the dictionary before object instantiation we can just go 'look-up' the requested value in the already built object (the Spockspace). Syntactically this is similar to above where we would regex for the syntax we want to use (probably just like OmegaConf) such as: Again I think this should be able to be handled right before class instantiation in the field handler. The graph would have to be built before then though, so we can have the correct iteration order. This is where it will be a bit tricky because we need to reconcile the order of the class dependencies and the value dependencies into one order. Not too hard but just need to think it through... Nick |
Hi Nick, thanks for your detailed thoughts and explanation on this. I will start on the Environment Resolver in the first PR, "easier" to implement sounds good to me :) |
@svenstehle if you are still interested #254 should implement the Environment Resolver feature above. Apologies that you couldn't ever get the unit tests to run for #246... You might have just needed to make sure the src dir of |
Is your feature request related to a problem? Please describe.
Cannot access e.g., environment variables in config (can I access and compose from other config variables and append to them?). This would be the oc.env resolver.
Maybe a separate feature request but Spock already offers some (or all, haven't checked?) of the oc.decode functionality.
Describe the solution you'd like
save_path: ${oc.env:PWD}/runs
and
More examples found in OmegaConf.
Describe alternatives you've considered
Nick mentioned this might be done with the underlying library
attrs
(?)Additional context
Started discussing the idea in this issue
Thanks for considering to work on this :)
As I already mentioned in the other thread/issue: happy to help, but no idea where to begin and how to move into a worthwhile direction without guidance. Would love to work on this together though.
The text was updated successfully, but these errors were encountered: