Surrender to the dark side: formalize metadata structure?

[hollasch]

My prior plan was to rely on an ad hoc, use-case defined waterfall of element string "recognizers". However, that's an unformalized, error-prone approach, and can lead to erroneous assignment of types.

This recently was in the news, as some DNA sequences look like dates to Excel when importing CSVs: https://www.theverge.com/2020/8/6/21355674/human-genes-rename-microsoft-excel-misreading-dates

One could easily imagine similar cases in a world of unlimited types.

I have been very resistant to the idea of formalizing a metadata logic in LSON, but I the above example forces me to throw in the towel.

I'll need to think about this much more, but here's a representative sketch:

{!
    aliases: {
        foo: bar
        baz: qux
        number: (real:)
        infinity: (real:infinity)
        empty: ()
    }
    auto-type: [ null, bool, real, css-color ]
    ...
    body: ...
!}

Of course, this "solution" may well be insufficient. For example, if the auto-type waterfall has date, dna, then all we did was just formalized the weakness that led to the problem referenced above.

Further, how could we avoid the bloat associated with all this formalism up front? This is reminiscent of the old Direct3D .x file format, or the XML boilerplate bloat associated with many applications.

An alternative is to eliminate the concept of a waterfall altogether (or perhaps that's a recommendation?), and instead, require explicit types everywhere, and alleviate the pain with aliases.

[hollasch]

For maximum JSON compatibility, would there be standard implicit aliases? Something like aliases: { null: (null:null), n: (real:), true: (bool:true), false: (bool:false) }.

Also note that aliases would include different kinds, such as types, values, and possibly keys.

[hollasch]

If we require explicit typing, what's a good shorthand for unitary values, like null, empty, or undefined? Some ideas:

• (null:0)
• (null::)
• (/null)
• (!null)
• @null
• (null)

The last option is pretty tempting, as it's brief, and possible if multi-valued types require explicit typing.

[hollasch]

Also, the alias structure might allow implicit typing for specific values. For example:

values: { true: bool, false: bool }