-
Notifications
You must be signed in to change notification settings - Fork 1
memory + backlog: dependency-priority + Microsoft-Research preferred + metrics-are-our-eyes (Helen Keller framing) + B-0147/B-0148/B-0149 (Aaron 2026-05-01) #1117
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
848c36c
4a2c93f
3c0d8b3
c5ea70a
c8b1aa6
4723a61
c4dbe5b
daa974c
45acf4b
1521b6a
0b994d8
8912a3e
dc3c3c1
4c35cb7
c5c5a1d
0686ba9
7fa7eba
5965be1
21c7600
ede777a
e584903
d75f6f0
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Large diffs are not rendered by default.
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,213 @@ | ||
| --- | ||
| id: B-0148 | ||
| priority: P2 | ||
| status: open | ||
| title: MDX as meta-DSL framing for multi-DSL Zset substrate + F# MDX DSL implementation | ||
| created: 2026-05-01 | ||
| last_updated: 2026-05-01 | ||
| --- | ||
|
|
||
| # B-0148 — MDX as meta-DSL framing + F# MDX DSL implementation | ||
|
|
||
| ## What | ||
|
|
||
| Evaluate **MDX (Multidimensional Expressions)** — the | ||
| Microsoft-published OLAP query language used in SQL Server | ||
| Analysis Services and many BI tools — as the **meta-DSL | ||
| framing** for the multi-DSL Zset substrate (per Aaron's | ||
| multi-algebra-DB vision: graph + hierarchy + filesystem + | ||
| timeseries + ... unified through a single meta-DSL). | ||
|
|
||
| If MDX fits as the meta-DSL shape, design and implement an | ||
| **F# MDX DSL** that natively hosts MDX-style queries against | ||
| the Zset algebra, with the existing types (graph, hierarchy, | ||
| filesystem, timeseries) appearing as MDX dimensions / | ||
| hierarchies / measures. | ||
|
|
||
| ## Why now | ||
|
|
||
| Aaron 2026-05-01 (composing two messages): | ||
|
|
||
| > *"plus promethius as a sick MCP and promtool and you'll love | ||
| > the query language its like simplifed multidimensonal query | ||
| > language MDX, oh shit backlog f# mdx dsl"* | ||
| > | ||
| > *"that's might be meta dsl framing"* | ||
|
|
||
| Aaron's recognition: **PromQL** (Prometheus Query Language) is | ||
| **MDX-shaped** — both are multidimensional-first query | ||
| languages with dimensions / hierarchies / measures / tuples / | ||
| sets. If PromQL composes naturally from MDX primitives, then | ||
| **MDX may be the right shape for the meta-DSL** that unifies | ||
| graph + hierarchy + filesystem + timeseries + future types | ||
| under the Zset substrate. | ||
|
|
||
| This composes directly with B-0147 (timeseries-DB | ||
| native-in-Zsets) — that row asks *what is the timeseries | ||
| algebra?*; this row asks *what is the meta-DSL that hosts the | ||
| timeseries algebra alongside the others?*. Both questions | ||
| need answers; they may share a candidate. | ||
|
|
||
| ## MDX background — why it might fit | ||
|
|
||
| **MDX core concepts**: | ||
|
|
||
| - **Cubes** — multidimensional data containers (≈ Zset of | ||
| tuples) | ||
| - **Dimensions** — axes of categorization (graph nodes, | ||
| hierarchy levels, filesystem paths, time) | ||
| - **Hierarchies** — ordered nested levels within a dimension | ||
| (filesystem trees, organizational charts, time periods) | ||
| - **Members** — elements within a hierarchy level (specific | ||
| nodes, specific paths, specific timestamps) | ||
| - **Measures** — numeric quantities computed over the cube | ||
| (counts, sums, ratios) | ||
| - **Tuples** — coordinates in multidimensional space | ||
| - **Sets** — collections of tuples | ||
| - **Calculated members** — derived measures | ||
|
|
||
| **MDX strengths for the meta-DSL role**: | ||
|
|
||
| - **Microsoft-published spec** (per | ||
| `feedback_dependency_source_priority_open_source_microsoft_cncf_apache_mit_research_microsoft_research_metrics_are_our_eyes_aaron_2026_05_01.md` | ||
| Tier 2 priority); not proprietary | ||
| - **First-class hierarchies** — directly maps to Aaron's named | ||
| types | ||
| - **Multidimensional from the start** — graph + hierarchy + | ||
| filesystem + timeseries are all dimensions; queries naturally | ||
| span dimensions | ||
| - **Mature semantics** — 25+ years of OLAP usage; well- | ||
| understood evaluation | ||
| - **Already PromQL-shaped** — per Aaron's recognition, | ||
| observability queries already fit | ||
| - **Compositional** — measures can be calculated from other | ||
| measures; queries can be parameterized | ||
|
|
||
| **MDX weaknesses to investigate**: | ||
|
|
||
| - **OLAP-cube-shaped** — designed for fact-and-dimension data; | ||
| may need adaptation for graph traversal | ||
| - **Read-only history** — MDX is query, not update; the | ||
| Zset retraction-native semantics need to compose with | ||
| MDX-as-query rather than be expressed in MDX itself | ||
| - **String-heavy syntax** — MDX is very string-y; | ||
| F# host should produce well-typed AST not stringly-typed | ||
| query | ||
| - **Possible verbosity** — MDX queries can be long; an F# DSL | ||
| embedding might be more concise than literal MDX | ||
|
|
||
| ## Acceptance criteria | ||
|
|
||
| 1. **Fit-analysis design doc** at | ||
| `docs/research/2026-XX-mdx-as-meta-dsl-fit-analysis.md` | ||
| answering: | ||
| - Does MDX's dimension/hierarchy/measure shape match | ||
| Aaron's named types (graph / hierarchy / filesystem / | ||
| timeseries)? Worked example for each. | ||
| - How does MDX compose with the Zset retraction-native | ||
| semantics? (Mathematical analysis; possibly involves | ||
| extending MDX with retraction operators.) | ||
| - How does PromQL specifically map to MDX? (Worked | ||
| example: a real PromQL query translated to MDX form.) | ||
| - Are there alternative meta-DSL candidates that fit | ||
| better? (e.g., Datalog, GraphQL, SPARQL, custom F# | ||
| DSL.) Evaluate at least 3. | ||
|
|
||
| 2. **Recommendation**: | ||
| - **Adopt MDX as meta-DSL** — proceed to F# DSL design | ||
| - **Adopt MDX-with-extensions** — proceed with documented | ||
| extensions | ||
| - **Reject MDX, pick alternative** — document why; pivot | ||
| to alternative | ||
| - **Defer** — the question is premature; revisit after | ||
| B-0147 lands | ||
|
|
||
| 3. **If adopt or adopt-with-extensions**: F# MDX DSL design | ||
| sketch at | ||
| `docs/research/2026-XX-fsharp-mdx-dsl-design.md` covering: | ||
| - AST shape (well-typed, not stringly-typed) | ||
| - Embedding style — quotation-based vs computation- | ||
| expression-based vs combinator-library-based | ||
| - Query-evaluation strategy — translate to underlying | ||
| algebras vs unified evaluation engine | ||
| - Type-system mapping — how MDX dimensions/measures get | ||
| F# types | ||
| - Worked examples — at least 3 queries spanning multiple | ||
| types (graph + timeseries + filesystem) | ||
|
|
||
| 4. **Implementation follow-up rows filed** for each major | ||
| step of the F# MDX DSL build (parser, AST, type-checker, | ||
| evaluator, integration tests). | ||
|
|
||
| ## Research-cadence inputs | ||
|
|
||
| Per the dependency-source priority + Microsoft-Research-as- | ||
| preferred-research-source memory: | ||
|
|
||
| 1. **Microsoft Research** (https://www.microsoft.com/en-us/research/) — search | ||
| for MDX evaluation semantics, OLAP query optimization, | ||
| F# DSL design papers (Don Syme + collaborators) | ||
| 2. **MDX official spec** (Microsoft docs) — the canonical | ||
| reference | ||
| 3. **PromQL docs** (CNCF Prometheus) — the worked-example | ||
| target | ||
| 4. **F# DSL design literature** — `Computation Expressions`, | ||
| `Quotations`, FSharp.Charting / Deedle / Math.NET as | ||
| examples of mature F# DSL embedding | ||
| 5. **Datalog research** — alternative meta-DSL candidate; | ||
| has rich academic literature | ||
| 6. **GraphQL** — alternative; CNCF-adjacent ecosystem | ||
|
|
||
| Per Otto-364 search-first: verify every load-bearing claim | ||
| against current docs/papers, not training data. | ||
|
|
||
| ## Out of scope (defer) | ||
|
|
||
| - **Implementation.** This row is research + design. | ||
| Implementation lands in follow-up rows. | ||
| - **Performance benchmarks.** Comes after design lands. | ||
| - **Backwards-compatibility with existing Zset query API.** | ||
| Whatever exists today; whether the F# MDX DSL replaces it | ||
| or composes with it is a design decision in the analysis. | ||
|
|
||
| ## Composes with | ||
|
|
||
| - `feedback_dependency_source_priority_open_source_microsoft_cncf_apache_mit_research_microsoft_research_metrics_are_our_eyes_aaron_2026_05_01.md` | ||
| — MDX is Microsoft-published (Tier 2); the dependency-priority | ||
| hierarchy applies | ||
| - `project_zeta_multi_algebra_database_one_algebra_to_rule_them_all_sequenced_after_frontier_and_demo_2026_04_23.md` | ||
| — the multi-algebra DB vision MDX is being evaluated as | ||
| meta-DSL for | ||
| - B-0147 — the timeseries-DB row that motivates this row; | ||
| PromQL/MDX-shape is the bridge | ||
| - B-0149 (Prometheus MCP) — sibling research lane; informs | ||
| the PromQL-as-MDX worked example | ||
| - B-0146 (formal architecture ladder) — when the design lands, | ||
| declare layer (likely Layer 3: class taxonomy, since | ||
| meta-DSL is a pattern catalog instantiating type-theoretic | ||
| primitives) | ||
| - The 4-axis tightness rule (ZSet-backed + first-class event + | ||
| retractable + columnar storage) per the indexed graph- | ||
| substrate-tight memory entry in `memory/MEMORY.md`; MDX must | ||
| compose with retraction-native to satisfy | ||
| - F# DSL design lineage — Don Syme's research (Microsoft Research, | ||
| Tier 2 + Microsoft-Research-preferred citation per the | ||
| dependency-priority memory) | ||
|
|
||
| ## Effort | ||
|
|
||
| **L (large, 3+ days, research-grade)** for the fit-analysis | ||
| doc + design sketch. F# MDX DSL implementation is open-ended | ||
| (multiple follow-up rows). | ||
|
|
||
| ## Why P2 | ||
|
|
||
| - **Not P0/P1** because the meta-DSL design isn't blocking | ||
| current factory operation; queries use whatever ad-hoc | ||
| shape exists today. | ||
| - **Not P3** because if MDX IS the right meta-DSL framing, | ||
| the cost of operating without it scales — every additional | ||
| type added under the multi-algebra vision will face | ||
| meta-DSL friction until it lands. | ||
| - **P2** sits where the research is important-but-not-urgent; | ||
| lands when bandwidth permits. |
| Original file line number | Diff line number | Diff line change | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| @@ -0,0 +1,169 @@ | ||||||||||
| --- | ||||||||||
| id: B-0149 | ||||||||||
| priority: P2 | ||||||||||
| status: open | ||||||||||
| title: Prometheus MCP integration + promtool — factory agents direct-query observability | ||||||||||
| created: 2026-05-01 | ||||||||||
| last_updated: 2026-05-01 | ||||||||||
| --- | ||||||||||
|
|
||||||||||
| # B-0149 — Prometheus MCP integration + promtool | ||||||||||
|
|
||||||||||
| ## What | ||||||||||
|
|
||||||||||
| Wire **Prometheus** into the factory as an MCP (Model Context | ||||||||||
| Protocol) server so factory agents can directly query | ||||||||||
| observability data via PromQL. Also adopt **promtool** (the | ||||||||||
| Prometheus CLI) for ad-hoc query / config-validation use. | ||||||||||
|
|
||||||||||
| ## Why now | ||||||||||
|
|
||||||||||
| Aaron 2026-05-01: | ||||||||||
|
|
||||||||||
| > *"plus promethius as a sick MCP and promtool and you'll love | ||||||||||
| > the query language its like simplifed multidimensonal query | ||||||||||
| > language MDX"* | ||||||||||
|
|
||||||||||
| This is the **operational counterpart** to B-0147 (timeseries-DB | ||||||||||
| research) and B-0148 (MDX as meta-DSL). While B-0147 / B-0148 | ||||||||||
| research the *long-term* substrate question (which timeseries | ||||||||||
| DB? which meta-DSL?), this row makes Prometheus *immediately | ||||||||||
| usable* as a factory observability surface. | ||||||||||
|
|
||||||||||
| Per the metrics-are-our-eyes framing (per | ||||||||||
| `feedback_dependency_source_priority_open_source_microsoft_cncf_apache_mit_research_microsoft_research_metrics_are_our_eyes_aaron_2026_05_01.md`), | ||||||||||
| the timeseries-channel is additive sensory capacity. | ||||||||||
| Prometheus plus MCP is the lowest-friction path to *getting | ||||||||||
| eyes operational NOW*, while the research about the optimal | ||||||||||
| long-term substrate proceeds in parallel. | ||||||||||
|
|
||||||||||
| ## Acceptance criteria | ||||||||||
|
|
||||||||||
| 1. **Prometheus deployment** — local Prometheus instance | ||||||||||
| scraping factory metrics (initial scope: tick-history | ||||||||||
| aggregations + PR-pipeline metrics + per-persona dispatch | ||||||||||
| counts). Configuration in `tools/observability/prometheus/`. | ||||||||||
|
|
||||||||||
| 2. **MCP server integration** — Prometheus exposed as an MCP | ||||||||||
| server consumable by Claude Code agents. Configuration in | ||||||||||
| `.mcp.json` (or wherever harness MCP config lives). | ||||||||||
|
|
||||||||||
| 3. **promtool wired into factory tooling** — `tools/observability/ | ||||||||||
| promtool/` wraps `promtool` for: | ||||||||||
| - Query validation (check PromQL syntax before storing | ||||||||||
| queries) | ||||||||||
| - Rule-file linting (ensure recording-rules / alerting- | ||||||||||
| rules are well-formed) | ||||||||||
|
AceHack marked this conversation as resolved.
|
||||||||||
| - Ad-hoc query execution from CLI | ||||||||||
|
|
||||||||||
| 4. **Initial query catalog** at | ||||||||||
| `tools/observability/queries/factory.promql` covering at | ||||||||||
| minimum: | ||||||||||
| - Tick rate over time (DORA-style deployment frequency) | ||||||||||
| - PR-cycle latency p50 / p95 (RED-style duration) | ||||||||||
| - Per-persona dispatch counts (USE-style utilization | ||||||||||
| proxy) | ||||||||||
| - Aaron-correction rate (Four Golden Signals | ||||||||||
|
AceHack marked this conversation as resolved.
|
||||||||||
| errors-style) | ||||||||||
|
|
||||||||||
| 5. **Documentation** at `tools/observability/README.md` | ||||||||||
| covering: | ||||||||||
| - How to start local Prometheus | ||||||||||
| - How agents query via MCP | ||||||||||
| - How to add new metrics (factory-side instrumentation) | ||||||||||
| - How promtool is used in the loop | ||||||||||
|
AceHack marked this conversation as resolved.
|
||||||||||
|
|
||||||||||
| 6. **Initial dashboard** (optional, ergonomics-only) — a | ||||||||||
| simple Prometheus-native UI / Grafana dashboard exposing | ||||||||||
| the SRE metric framework views (DORA / USE / RED / Four | ||||||||||
| Golden Signals) per | ||||||||||
| `feedback_reproducible_accuracy_before_quality_fitness_function_harness_first_aaron_2026_05_01.md` | ||||||||||
| (forward-ref to PR #1116). | ||||||||||
|
|
||||||||||
| ## Why Prometheus first (per Aaron's "good citizen" framing) | ||||||||||
|
|
||||||||||
| Aaron 2026-05-01 (earlier message): | ||||||||||
|
|
||||||||||
| > *"i know prometheus, that's our good citizen dependency | ||||||||||
| > candidate"* | ||||||||||
|
|
||||||||||
| Prometheus is Aaron's known-quantity dependency: | ||||||||||
|
|
||||||||||
| - **CNCF graduated** (Tier 3 per the dependency-source priority | ||||||||||
| hierarchy) | ||||||||||
| - **Apache 2.0 licensed** | ||||||||||
| - **Mature ecosystem** — promtool, alertmanager, exporters, | ||||||||||
| Grafana integration | ||||||||||
| - **Well-understood operational characteristics** — pull-based | ||||||||||
| scrape, time-series-native, label-cardinality-careful | ||||||||||
| - **PromQL is MDX-shaped** — composes with the meta-DSL | ||||||||||
| research line (B-0148) | ||||||||||
|
|
||||||||||
| Even if B-0147's research recommends a *different* long-term | ||||||||||
| timeseries DB, Prometheus is the right *starting point* | ||||||||||
| because: | ||||||||||
|
|
||||||||||
| 1. It exists today, deployable in minutes | ||||||||||
| 2. The dependency-priority hierarchy passes it (Tier 3) | ||||||||||
| 3. Its query language is already MDX-shaped (informs B-0148) | ||||||||||
| 4. Migration to a different backend later is well-understood | ||||||||||
| (OpenTelemetry-style portable metrics protocol; many | ||||||||||
| Prometheus-compatible backends) | ||||||||||
|
|
||||||||||
| ## Out of scope (defer) | ||||||||||
|
|
||||||||||
| - **Long-term backend choice.** B-0147 owns that question. | ||||||||||
| This row instantiates Prometheus *now*; substrate-level | ||||||||||
| decisions can revise later. | ||||||||||
| - **Production deployment.** Initial scope is local-dev / | ||||||||||
| loop-runner consumption. Production observability stack | ||||||||||
| (HA Prometheus, persistent storage, alerting routes) is | ||||||||||
| follow-up. | ||||||||||
| - **Custom exporters.** Use existing exporters (Node, GitHub, | ||||||||||
| PR-board) where possible. Custom exporter for factory- | ||||||||||
| specific metrics is follow-up if the standard ones don't | ||||||||||
| cover the needs. | ||||||||||
| - **PromQL → MDX translation.** B-0148's worked-example | ||||||||||
| exercise; this row only consumes PromQL natively. | ||||||||||
|
|
||||||||||
| ## Composes with | ||||||||||
|
|
||||||||||
| - `feedback_dependency_source_priority_open_source_microsoft_cncf_apache_mit_research_microsoft_research_metrics_are_our_eyes_aaron_2026_05_01.md` | ||||||||||
| — the substrate this row instantiates; Prometheus = Tier 3 | ||||||||||
| (CNCF graduated) | ||||||||||
| - `feedback_reproducible_accuracy_before_quality_fitness_function_harness_first_aaron_2026_05_01.md` | ||||||||||
| (forward-ref to PR #1116) | ||||||||||
| (PR #1116) — SRE metric frameworks (DORA/USE/RED/FGS) the | ||||||||||
|
Comment on lines
+134
to
+136
|
||||||||||
| - `feedback_reproducible_accuracy_before_quality_fitness_function_harness_first_aaron_2026_05_01.md` | |
| (forward-ref to PR #1116) | |
| (PR #1116) — SRE metric frameworks (DORA/USE/RED/FGS) the | |
| - PR #1116 — SRE metric frameworks (DORA/USE/RED/FGS) the |
Uh oh!
There was an error while loading. Please reload this page.