nlohmann::json instance and JSON Schema for MemorySourceAccessor

[Ericson2314]

Motivation

As documented, this for file system objects themselves, since MemorySourceAccessor is an implementation detail.

Context

Depends on #14598

---

Add 👍 to pull requests you find important.

The Nix maintainer team uses a GitHub project board to schedule and track reviews.

[roberth]

This seemed so simple, but bytes strike again.

[roberth]

Maybe these should point the other way around?
Being a documentation example is more significant than being tested as part of some test suite.
Also it's helpful for the test suite to have an explicit reference to path that's in the manual, to distinguish it from test cases that only exist for the purpose of testing, i.e. corner cases that aren't relevant enough for the manual.

[Ericson2314]

I switched it from your version because of how _NIX_TEST_ACCEPT works, especially when making new files.

[roberth]

Can it not write through a symlink? IIRC I had something like tests/data/examples point into the manual. Wouldn't it just write to data/examples/new-file if you passed examples/new-file as the stem?

Otherwise, I guess we could use a naming convention to highlight which examples are for docs.

[Ericson2314]

It will write through the symlink if the symlink already exists, but I like to use _NIX_TEST_ACCEPT=1 for making the initial versions of those files too.

[roberth]

There's probably semantics we want to document here, like we don't allow /?
Probably best to make a link, so that we don't duplicate all prose here.

I guess we could encode a no / restriction with draft 6 propertyNames if we want.

[roberth]

Names may not be valid UTF-8 and therefore not be valid JSON (sequence of unicode characters, https://www.rfc-editor.org/rfc/rfc8259#section-1)

One possible solution, if this is something we've tested that it even works:

"It is commonly accepted that file names should be UTF-8, but not guaranteed.
Nix will forward invalid names, so if you wish to support all possible NARs in your application, you will need to use a parser that forwards invalid UTF-8 into your application, and allows you to produce invalid UTF-8 in the JSON you may have to output."

Alternatively, we could require base 64 here too.

Same applies to the symlink targets.

[roberth]

Or we could have a LenientString type that's either a valid JSON string, or a { "base64": ... }, but that only helps for values, not field names. I guess we could steal syntax and have "/aGVsbG8=": be an equivalent, non-canonical field name to "hello":.

So LenientString would be a type for representing values that are typically valid strings, but may not always be, and base 64 would be an unwarranted hindrance in practice. (Not sure about the latter; maybe we should just use it for file contents too?)

[Ericson2314]

I was wrong about null bytes (I used the wrong kind of string literal), I think nlohmann::json will happy do escape sequences for pretty arbitrary stuff. (I am not sure we even give it enough unicode support at runtime to not do that, fwiw.)

Not it will complain about invalid UTF-8. I would prefer to fix this in a follow-up PR, however.

[roberth]

Another lack of semantics. Should probably just be a link.
What is the meaning of an absolute path? Or a relative path? Can a relative path be allowed to ..-leave the FSO? I don't think the manual has good answers yet, but this should link to what we have, @docroot@/store/file-system-object.md

[Ericson2314]

The manual is now updated a bit, and linked.

[edolstra]

Approved pending a check that this is a superset of the format produced by listNar().

[dpulls]

🎉 All dependencies have been resolved !