Layered variables

[jayvdb]

Currently all variables are merged immediately.

Each set of data should be kept in separate dictionaries and layered on top of each other, so that dict read of the top level merges the values at runtime (and has an lru cache).

Then it is possible to read each config , and combined them in different groups as needed for #126

i.e. two targets should be able to use the same config files and combined them in different merge-order.

This also allows disabling env-vars for some targets, but keeping them for others.

[jayvdb]

I am quite certain we can find an existing PyPI library which dynamically merges layered dictionaries like this.

[jayvdb]

https://github.com/EvgeniyMakhmudov/laminated looks quite good conceptually, but might need a bit of work to be compatible with moban's merge strategy. Their class system should even allow different merge strategies for each layer. It appears each layer doesnt have a link to the original dict, but it does have a name field which could be a key to mobans list of datasources.

https://github.com/lee-b/dictstack andhttps://github.com/eagafonov/overlay_dict/blob/master/overlay_dict/init.py doesnt have IDs for the layers.

https://github.com/staffanm/layeredconfig and https://github.com/scruffystuffs/clac are higher level - could be useful, but might be too heavy.

[chfw]

I could see the idea here. what moban does now: is to merge the data at once. And this PR says: hold on! we want to configure which of all layers will be pulled out before rendering. So moban needs to know which layers are involved and pull out un-wanted layers for some rendering group but keep some for others.

[chfw]

but it seems the ecosystem is not ready yet hence will postpone this work.