Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
56 changes: 56 additions & 0 deletions docs/BACKLOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -94,6 +94,62 @@ within each priority tier.
side (Window.fs wiring pending). Target: measured numbers in
`docs/BENCHMARKS.md` by end of round 20.

## P1 β€” SQL frontend + query surface (round-33 vision, v1 scope)

- [ ] **Shared query IR that compiles to the DBSP operator
algebra.** The pattern `../SQLSharp/openspec/specs/query-
frontends/spec.md` establishes: SQL-text parsing and
integrated-query (LINQ) both lower to one logical
planning pipeline. Zeta needs the same convergence point.
Design-doc round before code.
- [ ] **LINQ integration (`IQueryable<T>` roots).** Mapped-
table query roots + property-to-column descriptors per
SQLSharp's shape. Lowers to the shared query IR.
First v1 surface consumers will touch.
- [ ] **SQL parser with multi-dialect support.** Dialects to
target: T-SQL, PostgreSQL, MySQL, SQLite, DuckDB. Shared
parser front-end + dialect overlays per
`../SQLSharp/openspec/specs/language-and-extension-model/`
pattern. Aaron round 33: "we also want to target many
different SQL dialects."
- [ ] **Entity Framework Core provider.** Zeta ships an EF
Core provider so EF consumers get DBSP incremental query
plans for free. Aaron: "work tightly with entity framework,
then branch out to other ORMs." First-class v1.
- [ ] **F# DSL reimagining SQL for the modern era.** Extends
the existing `circuit { ... }` computational-expression
seed. Natively retraction-aware, bitemporal-ready,
incremental-by-default. Needs a design round; Aaron: "a
start with our computational workflow."
- [ ] **Additional ORM providers (post-EF).** Dapper,
NHibernate, LLBLGen, etc. After the EF provider lands
and the pattern is understood.

## P2 β€” Post-v1 query-surface research

- [ ] **Reaqtor-inspired durable-Rx "stored procedures"
design doc.** Microsoft's Reaqtor (MIT, dormant-but-stable,
reaqtive/reaqtor on GitHub) ships serializable expression
trees + a query engine that persists operator state across
restarts β€” durable Rx. Reaqtor is push/event-at-a-time
with no native retraction; the Zeta niche is "Rx +
durability + retraction-native" (a Reaqtor-shaped hole
nobody has filled). Research questions: (a) does the
`IReactiveQbservable<T>` surface shape translate to DBSP
Z-set deltas cleanly? (b) what does the URI-keyed
operator vocabulary look like against Zeta's operator
algebra? (c) `OnError/OnCompleted` vs
`Result<_, DbspError>` β€” keep our error model? (d) is
this a Zeta feature or a separate library on top?
Output: `docs/research/reaqtor-shape-for-zeta.md`.
- [ ] **Bitemporal + time-travel queries as a first-class
v2 surface.** Append-dated history with retraction-aware
point-in-time queries. Paper-worthy. Aaron round 33:
"yes I want this haha." Needs design round on storage
shape (append-dated rows vs versioned Spine), query
syntax (`AS OF TIMESTAMP` vs F# DSL primitives), and
retraction semantics under time-travel.

## P1 β€” Factory / static-analysis / tooling (round-33 surface)

- [ ] **`openspec-gap-finder` skill** (round 32 ask). Viktor
Expand Down
122 changes: 94 additions & 28 deletions docs/VISION.md
Original file line number Diff line number Diff line change
@@ -1,14 +1,14 @@
# Zeta β€” Long-Term Vision

> **Status:** round 33 v2 after Aaron's first pass of edits.
> **Status:** round 33 v3 after Aaron's second pass of edits.
> Aaron is the source of truth; this document changes freely.
> The `product-visionary` role (to be spawned, see
> `docs/BACKLOG.md`) will steward it once it exists.
>
> **Lore:** Aaron almost named the project "Database." The
> codename landed on Zeta, but the ambition stayed: all DB
> technologies in one big playground, built retraction-native
> from the ground up.
> codename landed on Zeta (locked as of round 33), but the
> ambition stayed: all DB technologies in one big playground,
> built retraction-native from the ground up.

## The project has two products

Expand Down Expand Up @@ -107,6 +107,25 @@ What makes `Zeta.Core 1.0.0` on NuGet:
- FsCheck LawRunner β€” `checkBilinear`,
`checkSinkTerminal`, `checkRetractionCompleteness`
already landed.
- **SQL frontend (v1).** Multiple dialect targets (T-SQL,
PostgreSQL, MySQL, SQLite, DuckDB) via a shared query
IR that compiles to the DBSP operator algebra.
`../SQLSharp/openspec/specs/query-frontends/` is the
pattern to study; that project has started the
LINQ-first frontend + planning convergence we want.
- **Tight LINQ integration (v1).** `IQueryable<T>` roots
on mapped tables; LINQ lowers to the same IR the SQL
parser targets. This is the primary surface for F# +
C# consumers.
- **Entity Framework provider (v1).** Zeta ships an EF
Core provider so EF consumers get DBSP incremental
query plans for free; downstream ORMs follow after EF.
- **F# DSL reimagining SQL for the modern era (v1).** The
existing computational-expression sketch (`DSL.fs`,
`circuit { ... }`) is the seed. Long-term ambition: a
natively F# relational DSL that treats retractions,
bitemporality, and incremental plans as first-class β€”
not bolted on. Inspired by LINQ but shaped for DBSP.
- Formal-method coverage β€” TLA+ / Alloy specs running
green in CI.
- CI parity + security posture β€” see Product 2 below.
Expand All @@ -117,11 +136,27 @@ What makes `Zeta.Core 1.0.0` on NuGet:
- Retraction-aware analytic sketches (HyperBitBit,
retraction-native quantiles; publication target).
- Info-theoretic sharder (Alloy-verified).
- Multi-node deployment (control-plane shape is an open
research question in-scope for this vision, not
deferred out).
- SQL frontend on top of the algebra.
- Bitemporal query surface.
- **Multi-node deployment** (control-plane shape open β€”
NATS / gRPC / Arrow Flight / bespoke; sharding,
replication, consensus, info-theoretic sharder;
firmly IN scope).
- **Bitemporal + time-travel queries (first-class v2).**
Append-dated history with retraction-aware point-in-
time queries. Paper-worthy and native to DBSP's
retraction model.
- **Additional ORM providers.** Dapper, NHibernate, LLBLGen,
etc. β€” follow the EF Core provider as the pattern.
- **"C# stored procedures" via Reaqtor-style durable Rx
queries.** Microsoft's Reaqtor (MIT, dormant-but-stable)
ships `IReactiveQbservable<T>` + a query engine that
persists operator state across restarts β€” "durable
Rx." Reaqtor is push/event-at-a-time with no native
retraction; Zeta would take the serializable-expression-
tree + durable-subscription shape and swap in DBSP
Z-set deltas so retractions are free. This is the
research-worthy niche: "Rx + durability + retraction-
native" is a Reaqtor-shaped hole nobody has filled.
See `docs/BACKLOG.md` for the design-doc task.

## Product 2 β€” The software factory

Expand Down Expand Up @@ -215,20 +250,26 @@ degrading the factory is a net-negative round.
## Commercial posture

Pure research / open-source is the first-class experience.
Commercial ambitions are on the table β€” not decided, not
required, not refused. The factory and the library are
designed to support either trajectory:
Commercial ambitions are on the table.

- If Zeta stays pure-OSS: research papers + reference
implementation + NuGet distribution.
**Trigger event.** Aaron: "[Commercial moves to decided]
when I'm able to use this in a real project at some point
for its database." That's the milestone β€” when Zeta is
load-bearing in a real Aaron-run project and could be
shipped under a commercial license or hosted offering
without degrading the OSS core.

- If Zeta stays pure-OSS through that milestone: research
papers + reference implementation + NuGet distribution.
- If commercial emerges: enterprise-grade features (tenancy,
compliance attestations, hosted offering, etc.) layer on
top of the OSS core; the OSS core never degrades for the
commercial story.

When commercial moves from "on the table" to "decided,"
the `product-visionary` + `branding-specialist` pair updates
this section.
the `product-visionary` + `branding-specialist` pair
updates this section and spawns the corresponding roles
(sales, enterprise-support, hosted-ops, etc.).

## How we decide what to build

Expand Down Expand Up @@ -301,15 +342,40 @@ Things Aaron has now stated directly and Kenji is confident about:
- NuGet library shipping at v1.0 (the first slice of the
full database).

Remaining gaps for the product-visionary's first audit
with Aaron:

- Naming: Aaron almost named it "Database" β€” is "Zeta"
locked in, or is a rename still possible at v1.0?
- SQL frontend: is it P1 post-v1 or much later?
- Commercial trigger: what event / milestone would move
commercial from "on the table" to "decided"?
- Bitemporal / time-travel queries: first-class v2 target
or speculative?
- Query IR: is Zeta aiming at "multiple frontends target
one IR" or "the F# DSL is the surface" long-term?
Things Aaron resolved this round (round 33 v3):

- **Zeta name is locked.**
- **SQL frontend is v1**, not post-v1. With tight LINQ
integration + multiple SQL dialect targets + Entity
Framework provider first, then other ORMs.
- **F# DSL reimagining SQL is v1** β€” extension of the
existing computational-expression sketch.
- **Commercial trigger**: when Aaron uses Zeta in a real
project for its database.
- **Bitemporal / time-travel: first-class v2.** ("yes
I want this haha" β€” framing distinction was noise.)
- **Query IR**: multiple frontends (SQL dialects + LINQ +
native F# DSL) target one IR that compiles to the DBSP
operator algebra. Inspiration pattern: SQLSharp's
"SQL-text and integrated-query flows converge on one
logical planning pipeline."
- **Reaqtor-shaped niche** ("C# stored procedures" as
durable Rx queries) is post-v1 research; genuinely
open territory because no upstream has Rx, durability,
and retraction-native semantics in one box.

Remaining gaps the product-visionary walks on first
audit:

- Which SQL dialect lands first in v1? T-SQL, PostgreSQL,
SQLite (all three? incremental?)?
- What's the license shape β€” Apache-2 / MIT / LGPL / dual-
licensed AGPL+commercial? Licensing is a commercial-
trajectory lever worth deciding before the commercial
trigger fires.
- Entity Framework provider surface β€” full LINQ provider
or incremental rollout (query first, then save-changes,
then migrations)?
- F# DSL name β€” does it need a name distinct from the
computational-expression builder syntax, or is
`circuit { ... }` the permanent brand?
Loading