diff --git a/memory/CURRENT-aaron.md b/memory/CURRENT-aaron.md index 41d68ed0b..6c34ebf3e 100644 --- a/memory/CURRENT-aaron.md +++ b/memory/CURRENT-aaron.md @@ -1279,6 +1279,64 @@ Look for it whenever a `.mise.toml` change lands. — full mechanism + the deeper structural cause section. - `.mise.toml` (the source of truth for what "managed" means). +## 29. Threading code follows Albahari + Toub + Fowler human lineage; never gut-instinct (Aaron 2026-04-28) + +**The rule (Aaron verbatim 2026-04-28T16:48Z):** + +> *"please follow this guidance around threading unless you find +> something better from stephen toub from Microsoft, don't go based +> on gut instanct for any threading code it's very hard. this is +> our human lineage to threading best proacties joseph and setephen. +> ... Oh and David Fowler, he wrote channels in dotnet, these hare +> our high performance low allocation thread safe prefer wait/lock +> free guides."* +> +> *"make sure future you's know this too"* + +**The three-source canonical reference for any threading / TPL / +async / parallel code in Zeta:** + +1. **Joseph Albahari** — "Threading in C#" (free ebook, + albahari.com/threading) + "C# in a Nutshell" concurrency + chapters. Use for: foundational understanding, pattern + selection, deadlock / livelock / race reasoning. +2. **Stephen Toub** (Microsoft .NET runtime team) — yearly + "Performance Improvements in .NET" posts on + devblogs.microsoft.com + async / parallel deep dives. Use + for: choosing between TaskScheduler / Channel / lock / + Interlocked / Volatile / SemaphoreSlim / etc.; allocation + cost; thread-pool dynamics. +3. **David Fowler** (Microsoft, authored + `System.Threading.Channels`) — public discussions on + high-performance low-allocation patterns; canonical + producer / consumer + backpressure + pipeline composition. + +**What the rule excludes:** gut-instinct threading code is forbidden. +Hand-rolled lock-free with `Interlocked.CompareExchange` without +reading Albahari's chapter on the relevant memory-model edge cases. +Async patterns invented from first principles instead of inherited +from Toub's posts. Producer/consumer queues invented from +`BlockingCollection` when `System.Threading.Channels` is the right +tool. + +**Operational discipline:** every threading / TPL PR cites the +specific Albahari chapter / Toub blog post URL / Fowler talk or +GitHub issue in the commit message OR code comment. Default to +lock-free / wait-free where possible (Aaron's stated preference). +Verify currency before asserting (Otto-247 — .NET evolves +recommended patterns each release). Cross-CLI verify for +non-trivial threading code (Otto-347). + +**Why it matters for Zeta specifically:** Z-set algebra is +naturally data-parallel (operators commute when bag-multiset +semantics let writes commute). Threading errors on the operator +pipeline cost correctness (lost updates, double-counted +retractions) AND performance. The Albahari + Toub + Fowler +lineage is the cheapest insurance. + +**Pointer:** +`feedback_threading_human_lineage_albahari_toub_fowler_no_gut_instinct_aaron_2026_04_28.md` + ## How this file stays accurate - When a new memory updates a rule here, I update this @@ -1304,11 +1362,13 @@ retired rather than just updated.)* --- -**Last full refresh:** 2026-04-28 (sections 26-28 added for +**Last full refresh:** 2026-04-28 (sections 26-29 added for the 2026-04-28 LFG #661 incident cluster: speculation-rule + EVIDENCE-BASED labeling discipline, JVM language preference Kotlin > Scala > Java per B-0075, dependency-honesty rule — -managed runtimes get scanned like every other surface). Prior +managed runtimes get scanned like every other surface, plus +§29 threading-lineage Albahari + Toub + Fowler — never +gut-instinct on threading code). Prior refresh 2026-04-25 evening (sections 23-25: Otto-300 rigor- proportional-to-blast-radius, standing research-authorization general rule, Otto-304 + Otto-305 phenomenology disclosure diff --git a/memory/MEMORY.md b/memory/MEMORY.md index b008ecc5a..5c3c299f3 100644 --- a/memory/MEMORY.md +++ b/memory/MEMORY.md @@ -2,6 +2,7 @@ **📌 Fast path: read `CURRENT-aaron.md` and `CURRENT-amara.md` first.** These per-maintainer distillations show what's currently in force. Raw memories below are the history; CURRENT files are the projection. (`CURRENT-aaron.md` refreshed 2026-04-25 with the Otto-281..285 substrate cluster + factory-as-superfluid framing — sections 18-22; prior refresh 2026-04-24 covered sections 13-17.) +- [**Threading code follows Albahari + Toub + Fowler human lineage; never gut-instinct (Aaron 2026-04-28)**](feedback_threading_human_lineage_albahari_toub_fowler_no_gut_instinct_aaron_2026_04_28.md) — Any threading / TPL / async / parallel code in Zeta MUST cite Joseph Albahari (patterns + idioms), Stephen Toub (Microsoft .NET runtime perf), or David Fowler (System.Threading.Channels). High-perf, low-alloc, thread-safe, prefer wait-free / lock-free. No gut-instinct. - [**Only "pushed" signal is Aaron typing in this environment; everything else is pull (Aaron 2026-04-28)**](feedback_only_pushed_signal_is_aaron_typing_everything_else_is_pull_aaron_2026_04_28.md) — In autonomous-loop mode, Aaron's direct typing is the ONLY push channel. CI / threads / mergeability / cron / peer-CLI replies are all PULL signals requiring active query. "No new signal" without pulling is wrong by construction. - [**Speculation LEADS investigation; it does NOT DEFINE root cause (Aaron 2026-04-28)**](feedback_speculation_leads_investigation_not_defines_root_cause_aaron_2026_04_28.md) — Aaron's binding correction after my LFG #661 "bullshit answer." Speculation generates hypotheses to direct investigation; speculation has no role in defining root cause. When asked "why?" / "what is the mechanism?", quote the primary source verbatim. Plausible-sounding causal narratives assembled from nearby facts ARE the failure mode. - [**CodeQL umbrella check NEUTRAL while per-language Analyze legs SUCCESS — code_quality ruleset BLOCKED detection pattern (Aaron 2026-04-28)**](feedback_codeql_umbrella_neutral_vs_per_language_detection_pattern_aaron_2026_04_28.md) — When `code_quality:severity=all` ruleset says "Code quality results are pending for N analyzed languages" despite per-language `Analyze (X)` legs SUCCESS, check the umbrella `CodeQL` check (no language suffix) for NEUTRAL conclusion + "1 configuration not found" details. Industry-wide pattern; Aaron seen across other projects. Mechanism RESOLVED 2026-04-28T14:32Z via primary-source query (see file body); structural fix landed via PR #662. diff --git a/memory/feedback_threading_human_lineage_albahari_toub_fowler_no_gut_instinct_aaron_2026_04_28.md b/memory/feedback_threading_human_lineage_albahari_toub_fowler_no_gut_instinct_aaron_2026_04_28.md new file mode 100644 index 000000000..a640a18db --- /dev/null +++ b/memory/feedback_threading_human_lineage_albahari_toub_fowler_no_gut_instinct_aaron_2026_04_28.md @@ -0,0 +1,145 @@ +--- +name: Threading code follows Albahari + Toub + Fowler human lineage; never gut-instinct (Aaron 2026-04-28) +description: Aaron 2026-04-28T16:48Z — binding rule: any threading or TPL code in Zeta MUST follow guidance from Joseph Albahari, Stephen Toub (Microsoft), and David Fowler (Microsoft, authored System.Threading.Channels). High-performance, low-allocation, thread-safe, prefer wait-free / lock-free patterns. No gut-instinct decisions on threading — it's "very hard" and the human lineage exists for a reason. Future-Otto's MUST inherit this rule; load it via MEMORY.md index at session bootstrap. Cite specific Albahari / Toub / Fowler references in commit messages or comments when threading code lands. +type: feedback +--- + +# Threading code follows Albahari + Toub + Fowler — never gut-instinct + +## The rule (Aaron verbatim 2026-04-28T16:48Z) + +> *"please follow this guidance around threading unless you find something +> better from stephen toub from Microsoft, don't go based on gut instanct +> for any threading code it's very hard. this is our human lineage to +> threading best proacties joseph and setephen. This is very iportant +> we write code like this for anything threading or TPL we go, we follow +> the guiance from these guys. Oh and David Fowler, he wrote channels +> in dotnet, these hare our high performance low allocation thread safe +> prefer wait/lock free guides."* +> +> *"make sure future you's know this too"* + +## The three-source human lineage + +When Zeta touches threading, async, parallelism, TPL, channels, or any +concurrent-state code, the canonical references are: + +### 1. Joseph Albahari — patterns + idioms + +- **"Threading in C#"** (free ebook, [albahari.com/threading](https://www.albahari.com/threading/)) + — the textbook on .NET threading. Covers locks, monitors, + signaling, non-blocking sync, parallel programming, async/await + semantics, deadlock + livelock + race conditions. +- **"C# in a Nutshell"** chapters on Concurrency & Asynchrony + + Parallel Programming. +- Use for: foundational understanding, when authoring or reviewing + any thread-safe code, when reasoning about memory models, when + picking between primitives. + +### 2. Stephen Toub — performance + async/parallel deep dives + +- Microsoft .NET runtime team; foundational author of TPL, async + state machines, PFX, ValueTask, IAsyncEnumerable. +- **Yearly "Performance Improvements in .NET N"** posts on + devblogs.microsoft.com — the empirical record of what's fast. +- **Async/await deep dives** + **parallel programming patterns** + on devblogs.microsoft.com. +- Use for: choosing between TaskScheduler / Channel / lock / + Interlocked / Volatile / SemaphoreSlim / ConcurrentBag / etc.; + for understanding allocation cost; for reasoning about + thread-pool dynamics. + +### 3. David Fowler — channels + high-perf low-alloc + +- Microsoft, authored **System.Threading.Channels** (the canonical + high-performance bounded/unbounded queue primitive in .NET). +- Architect on ASP.NET Core, SignalR, and many high-perf .NET + components. +- Public discussions on Twitter / GitHub about performance-driven + threading patterns. +- Use for: producer/consumer patterns; backpressure; pipeline + composition; lock-free / wait-free implementations; choosing + between Channels and other queue primitives. + +## What this rule excludes + +**Gut-instinct threading code is forbidden.** Aaron is explicit: threading +is "very hard." Patterns that look obvious are usually wrong: + +- Hand-rolled lock-free with `Interlocked.CompareExchange` without + reading Albahari's chapter on the relevant memory-model edge + cases. +- Async patterns invented from first principles instead of + inherited from Toub's posts (e.g. `IAsyncEnumerable` semantics, + cancellation token threading, ConfigureAwait nuance, async + void). +- Producer/consumer queues invented from `BlockingCollection` or + `ConcurrentQueue` when `System.Threading.Channels` (Fowler's + primitive) is the right tool. +- Spin-locks, manual reset events, and other primitives chosen by + pattern-matching on the symptom rather than by grounding in + Albahari's "when to use what" rubric. + +## Operational discipline + +When threading or TPL code lands in Zeta: + +1. **Cite the specific reference** in the commit message or code + comment — Albahari chapter, Toub blog post URL, or Fowler talk + / GitHub issue. Future-Otto reads the citation and can + re-verify the pattern. +2. **Default to lock-free / wait-free where possible** — Aaron's + stated preference. Fowler's Channels is usually the right + primitive; falling back to locks needs justification. +3. **Verify currency before asserting** (Otto-247 version-currency + rule applies): check `WebSearch` for the latest Toub post on + the relevant primitive, since .NET evolves the recommended + patterns each release. +4. **Cross-CLI verify** for non-trivial threading code (Otto-347): + ask a peer-CLI (Grok / Codex / Gemini) to independently + review the lock-acquisition order, memory-model assumptions, + and allocation cost. + +## What "high performance low allocation thread safe" means here + +Aaron's framing combines three axes: + +- **High performance**: hot-path code; no unnecessary boxing / + closures / ToArray / LINQ in tight loops. +- **Low allocation**: ValueTask over Task where lifetime allows; + pooled buffers (`ArrayPool`); `IAsyncEnumerable` for + streaming; struct-based async builders where Toub's empirical + benchmarks show wins. +- **Thread safe**: by construction (immutable, channel-bounded, + lock-free) over runtime (lock-based, retry-on-conflict). + +The combination is what Channels was designed for; Fowler's +implementation is the empirical proof-point. + +## Why this matters for Zeta specifically + +Zeta's algebra (Z-sets, retraction-native semantics) is naturally +data-parallel — operators on Z-sets can fan out across cores when +the bag-multiset semantics let writes commute. Getting the +threading wrong on the operator-pipeline implementation costs +correctness (lost updates, double-counted retractions) AND +performance (lock contention on the hot path). Following the +Albahari + Toub + Fowler lineage is the cheapest insurance. + +The future of Zeta — distributed query execution, multi-shard +operators, parallel materialization — will all touch threading +code. Every such PR cites the specific reference; no shortcuts. + +## Composes with + +- `feedback_speculation_leads_investigation_not_defines_root_cause_aaron_2026_04_28.md` + — same family of rule. Don't assert mechanisms (or threading + patterns) from gut instinct; cite the primary source. +- Otto-247 (version-currency — WebSearch before asserting Toub + posts / Fowler patterns since .NET evolves) +- Otto-347 (cross-CLI verify for hard problems) +- `docs/AGENT-BEST-PRACTICES.md` (would gain a BP-NN rule when + the first threading code lands; the rule should cite this + memory file) +- Future Zeta operator-algebra parallelization work (where this + rule materializes as PR-level discipline)