I chose the name Eon fairly hastily and ended up kind of disliking it. I wanted to change it almost immediately, but decided to hold off until I felt sure about its replacement. Memo is one character longer but just sounds better to me and reflects the system's ability to record every keystroke. It's kinda silly to worry this much about a name, but I just needed to change it. Now it's done. Moving on.
The bigger news is that we've finally achieved convergence in our randomized tests of replicated directory trees. The problem ended up being way harder than we imagined. The final challenge was to fully simulate the possibility for the file system to change at any time, including during a directory scan.
We are cautiously optimistic that the worst of the algorithmic challenges could be behind us. Weeks of wading through randomized test failures has been a bit monotonous, but hopefully we can pick up some momentum building on top of this abstraction.
The next step is to add support for files to the directory tree, which we think should be easier. Much of what we learned dealing with pure directories can be applied to files, and since files are always leaf nodes we shouldn't need to deal with cycles. We do need to deal with hard links, however, which should add some complexity.
Supporting files also means we need to figure out the relationship between the CRDT that maintains the state of the directory tree and the CRDTs that contain the contents of individual text files.
This week seems like the right time to zoom out and get a bit more clarity on the system's higher level design. Until we had a working CRDT for directory trees that felt premature, but now it seems like understanding the big picture a bit better might inform the relationship between the directory tree and individual text files.
We've gone back and forth on whether we should try to decouple them, but for now we think we're going to try a more integrated approach where the directory tree CRDT has explicit knowledge of the file CRDTs. For now, we've decided to wrap both concerns in a single type called a Timeline, which will represent the full state of a working tree as it evolves forward in time. A Repository will contain multiple timelines which can evolve in parallel, fork, and eventually merge.
There's still quite a bit to figure out though. How will we route operations to and from buffers? What will the ownership structure look like? How can we ensure that performing I/O doesn't interfere with the responsiveness of the system? We'll hopefully have some conclusions about those questions and more to share in the next update.