Support for database backends

[kwatsen]

Looking into Sysrepo’s support for database backends, one can find Sysrepo#774, in which Radek makes this comment:

This will need support in libyang, which expect this way of enahncement, but not directly in a form of plugins. It will still need some (small) changes in sysrepo code then, but the first step is definitely support for storing and loading data in such format in libyang.

This issue is to track the support for this feature.

[kwatsen]

What intersection and/or kind of libyang modification would be needed?

For the most part, libyang seems to be independent of external data access layer (DAL) concerns. That said, I could imagine callbacks being registered into libyang to fetch parts of the schema and/or data tree from a DAL. Is this the idea?

[rkrejci]

What I was talking about in the sysrepo issue, was a support for another data format in libyang next to XML, JSON and LYB format currently supported. Probably it can be in a form of some plugins that would implement a data printer and parser to get data out and into the internal data tree structures.

That was my idea. However, even what you wrote make sense. libyang is able to write data info FILE, file descriptor, memory or filepath. So here could be a place for plugin to actually convert XML string from memory into a database (or something else) and vice versa (from DB into XML string). The difference is in efficiency (the second approach will be worse) and simplicity (the second approach will be probably more simple).

What do you think about efficiency vs simplicity in this case. Probably it would be possible to allow both types of plugins, but then libyang could seem over-pluginned :)

[kwatsen]

I'm unsure, but I think that the goals are two-fold:

1. for apps with known-small data-models, having an in-memory instance all the time is reasonable and the only desire to use database is for ACID compliance.

2. for apps with potentially huge data-model, having an the entire instance in-memory at any time is undesirable. Some way to incrementally read/write parts to a persistence layer would be desired. There would obviously be a need to define how the model is partitioned into persistable chunks, but that seems outside libyang's scope.