Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Codegen of unions, and their keyed representations #60

Merged
merged 12 commits into from
Jul 8, 2020
Merged

Commits on Jul 2, 2020

  1. Configuration menu
    Copy the full SHA
    036a0a1 View commit details
    Browse the repository at this point in the history
  2. Outline of union gen type and first thoughts.

    Expect this to evolve slowly over and the course of *many* commits.
    Unions are exceptionally tricky, and their various representations
    are by far the most complicated parts of IPLD Schemas.
    
    In particular, the fact that how to interpret data *inside* a union
    may vary, whilst some representations may not actually tell us which
    union member we have until later in the stream... is going to produce
    a lot of absolutely fabulous complexity.  Look forward to it.
    warpfork committed Jul 2, 2020
    Configuration menu
    Copy the full SHA
    a667b68 View commit details
    Browse the repository at this point in the history
  3. Notes while thinking ahead on codec%union performance.

    It's kind of dark.  I've been leaving remarks strewn around github
    for a while now to the general effect of "please please please
    prefer keyed unions whenever you can; the others can be Not Fast".
    But it's even more nasty in terms of details like error handling
    than I think I had yet realized.  Sigh.
    
    Oh well.  Remember: we exist to describe data that people *have made*,
    not just the data that we want to encourage them to make in the future.
    warpfork committed Jul 2, 2020
    Configuration menu
    Copy the full SHA
    df85307 View commit details
    Browse the repository at this point in the history
  4. Rename s/anyType/typeBase/. Internal.

    The appearance of the word "any" there has started to perturb me;
    "any" is a concept that we also need to describe in schemas, and that
    type has nothing to do with it.  It's more of a base mixin.
    warpfork committed Jul 2, 2020
    Configuration menu
    Copy the full SHA
    486a5dc View commit details
    Browse the repository at this point in the history
  5. Fix up the type structures for unions.

    Also make some of the placeholder construction functions.
    (Not all of them.  Just the ones I intend to use first.)
    
    There was some very unfinished placeholder stuff going on there.
    (And ironically, it wasn't written in a very clear union-like way;
    it gets a lot less icky when it's rewritten so that it's uniony.)
    warpfork committed Jul 2, 2020
    Configuration menu
    Copy the full SHA
    b60e5f7 View commit details
    Browse the repository at this point in the history
  6. I just needed the top object again. Make helper.

    Golang templates are mostly pretty smooth to get started with,
    but this particular issue is really a bizarre tripwire, imo.
    
    Will clean up the existing "range modifies dot unhelpfully" messes
    sometime over the rainbow; I just refuse to make more of them today.
    warpfork committed Jul 2, 2020
    Configuration menu
    Copy the full SHA
    390d3fe View commit details
    Browse the repository at this point in the history
  7. More steps in union codegen.

    Generates the type structure itself, and a marker interface.
    Not much else yet.
    
    Wired test harnesses and plugged out one example.
    
    It's a bummer we don't really have a great way to poke part of it into
    running until the whole thing satisfies the generator interface.
    But it'll come soon enough.
    warpfork committed Jul 2, 2020
    Configuration menu
    Copy the full SHA
    a9b5a19 View commit details
    Browse the repository at this point in the history
  8. Most of union type-level read implemented, for both style of memory l…

    …ayouts.
    
    Outline for keyed representation, but only enough to get some more
    things compilable and start getting all the templates exercised.
    
    Still can't really e2e test the things we've got without getting
    the rest of it and its builders finished.  It's coming along pretty
    reasonably, though, it seems.
    warpfork committed Jul 2, 2020
    Configuration menu
    Copy the full SHA
    0ddeba4 View commit details
    Browse the repository at this point in the history

Commits on Jul 4, 2020

  1. First partial draft of union assemblers.

    These are turning out to be *very* fun.
    
    In the process of detailing how child assemblers work for unions,
    it has become time to consider how separately-allocated child
    assemblers are handled, and what their existence implies about
    the rest of our logical rules how all the assemblers behave in concert.
    Like so many things in this endeavor... the far-reachingness of
    the implications of even seemingly very mild choices is... high.
    
    I think it's quite likely that the current claims about union
    assemblers not needing extra state for set/focus might turn out to be
    redacted in upcoming commits, and for the strangest reason: we might
    actually need that state to be persisted (outside of 'w') so that
    the *reset* mechanism, of all things, can be efficient.
    We'll see; still sort of tumbling that thought around in my mind.
    There are various options.
    warpfork committed Jul 4, 2020
    Configuration menu
    Copy the full SHA
    3997fc8 View commit details
    Browse the repository at this point in the history
  2. I believe this is the rest of union assemblers.

    ... for the type-level assemblers, anyway.
    
    We *still* can't do e2e tests on this that include compiling the
    generated code; not until we get at least one representation strategy
    implemented.  All the templates exercise, though (even though you
    might have to uncomment some of the test "skip" lines to do it).
    
    I'll move on to doing the keyed representation next, and that will
    be the thing we use to prove out all of this stuff.  It should be
    the easiest one to do now, too, since it's semantically very very
    similar to how we made the type-level behaviors work.
    warpfork committed Jul 4, 2020
    Configuration menu
    Copy the full SHA
    15b5a48 View commit details
    Browse the repository at this point in the history
  3. Union generation complete, and keyed repr; tests; and they pass.

    I can't quite claim tests passed on the *first* shot... but,
    the first shot after mostly syntactical (rather than semantic) fixes?
    Yeah, actually.  That was pretty fun.
    
    Snuck in a bit of DRY'ing up.  The repetition in BeginMap methods
    got to me, and was low hanging fruit, so I extract that from unions
    and also backported it to structs.
    
    Errors got some work in this commit, because it turns out I've
    straightjacketed myself by not allowing "fmt.Errorf" due to imports.
    There's a lot to do there, and I only tackled what was directly
    critical to get this commit about unions across the finish line,
    but there's a few remarks in comments about where to go next.
    
    Some more comments about future work on the type info holder types
    also appears; I'm starting to skid on those placeholders and their
    issues more and more.  I really hope we can get to replace those
    sooner than later.
    
    And... also, yes, the idea of not having a "focus" state field in
    assemblers really bit it, hard, as speculated in the previous
    commit message.  I ended up using 'ca' in more switches than I
    expected, simply because it's easier to use that than have the
    conditonal templating branches that would be necessary to use the other
    tagging mechanisms that do also have sufficient info.
    
    One big fixme in the core interfaces for nodebuilders (wince):
    the ValuePrototype method can error sometimes, and that hasn't been
    accounted for.  Need to make a decision about what to do there.
    It's not really an exercised path in practice, but it shouldn't
    contain caltrops, regardless of how frequently used it is.
    (This probably would've come up earlier, except there's a bunch of
    stubs about ValuePrototype in other parts of codegen already; all of
    them need backfill, but haven't yet made it to top of the todo heap.)
    
    But despite all the fixmes accumulated, this does bring unions
    to be a usable thing, and definitively proves out that the design
    still works, even for what turns out to be one of the most complicated
    parts of the schema system!  It's very, *very* exciting to add the
    checkmarks to this part of the feature table -- it's one of the places
    I most feared "unknown unknowns", now it's put to bed.  Woot!
    warpfork committed Jul 4, 2020
    Configuration menu
    Copy the full SHA
    f62d944 View commit details
    Browse the repository at this point in the history

Commits on Jul 8, 2020

  1. ValuePrototype may return nil when asked about invalid keys.

    This is a clear and simple solution to the problem.
    
    (I think I may have gotten so optimistic about type systems for a while
    that I forgot that returning nils is an option.  mmmm.)
    
    The docs on NodeBuilder already even nodded towards this possibility,
    so I've just further clarified that.
    
    After staring at this method for a while, I begin to wonder if it even
    is all that useful.  To use it sensibly on structs or unions (for any
    purpose other than checking if something is *not* a field/member),
    you have to figure out which keys you can ask it about... which...
    means you'd need the type info, to be able to enumerate that!  At which
    point you'd just be able to look at the type info, and wouldn't really
    need to ask the builder about ValuePrototype in this way at all,
    rendering the whole thing moot.
    
    But removing it seems a little drastic, too.  And would leave questions
    about how to do those inspections on untyped things.  So.  Despite the
    nibbles of oddity around this, perhaps this is still the least-bad
    design that can make these situations legible.
    warpfork committed Jul 8, 2020
    Configuration menu
    Copy the full SHA
    ce2ef76 View commit details
    Browse the repository at this point in the history