diff --git a/docs/reference/21-lean-md.md b/docs/reference/21-lean-md.md new file mode 100644 index 0000000000..9df67ac24a --- /dev/null +++ b/docs/reference/21-lean-md.md @@ -0,0 +1,91 @@ +# Journey 21 — lean-md (Addon Integration) + +> lean-md is an **external lean-ctx addon** — a macro/directive Markdown renderer. +> It lives in its own repository (`dasTholo/lean-md`) with its own release cycle. +> This page documents how lean-ctx **integrates** the addon. The full `@directive` +> catalog, engine spec, and E-constructs live in the addon repo, not here. + +--- + +## 1. What lean-md is + +lean-md renders `.lmd.md` / `.lean-md` files: `@directive` calls plus a macro +engine (`@define`/`@call`), container gating (`@if`/`@consumer`), and pipes +(`@render`). Code-intel directives (`@read`/`@refactor`/`@search`/…) call lean-ctx +`ctx_*` tools **over the wire** (CLI/MCP); the renderer itself is standalone +(`rushdown` + `evalexpr`) with **no** lean-ctx crate dependency. + +Engine, full directive catalog, and spec: **https://github.com/dasTholo/lean-md**. + +## 2. Installation + +```bash +lean-ctx addon add @dasTholo/lean-md # hosted pack (ctxpkg.com) +lean-ctx addon add ./lean-ctx-addon.toml # local manifest (dev/test) +``` + +`addon add` resolves a local manifest first, then a hosted `ns/slug` pack, then the +bundled registry slug. The bundled `lean-md` entry is **listed** — it makes the addon +discoverable through `lean-ctx addon search`, it is not an install path. + +After install, restart the MCP client so the gateway catalog is re-read. The addon +is spawned as a stdio gateway child; its tools (`ctx_md_render`, `ctx_md_check`) +become reachable through the lean-ctx server. + +## 3. Integration points in lean-ctx + +lean-ctx keeps its lmd surface deliberately small: `.lmd.md` is read **raw** (§3.1), +the addon ships as a registry entry (§3.2), and the addon calls back through the +stable `ctx_*` surface (§3.3). Everything else is the addon's. + +### 3.1 Raw `.lmd.md` read (no in-tree rendering) + +`ctx_read` treats `.lmd.md` like any other file: it returns the **raw** bytes and +never renders (a half-rendered body would be worse than none). Rendering is the +addon's job, reached explicitly through its `ctx_md_render` / `ctx_md_check` tools +once installed. lean-ctx carries **no** `.lmd.md` special-casing in `ctx_read`; the +earlier auto-render delegation hook was reverse-cut before merge. + +Source: `rust/src/tools/registered/ctx_read.rs` (no lmd branch), +gate test `rust/tests/ctx_read_lmd_md_raw.rs`. + +### 3.2 Addon registry entry + +`rust/data/addon_registry.json` carries the **listed** `lean-md` entry (no runnable +`[mcp]` command, no `[install]` block), so `core::addons::manifest::is_installable` +reports `false` and the entry serves discovery only. The validator +(`core::addons::registry::validate_entries`) requires a homepage for a listed entry. + +### 3.3 ctx_* outbound surface = addon contract + +Every lean-md code-intel directive calls back into lean-ctx via +`backend.call("ctx_*", …)`. That tool set (`ctx_read`, `ctx_refactor`, +`ctx_search`, `ctx_outline`, `ctx_impact`, `ctx_repomap`, `ctx_review`, +`ctx_routes`, `ctx_smells`, `ctx_architecture`, `ctx_graph`, `ctx_callgraph`, +`ctx_knowledge`, `ctx_handoff`, `ctx_agent`, …) is a stable **outbound contract** +and must stay registered. Only `ctx_md_render` / `ctx_md_check` are addon-provided +and absent from lean-ctx. + +## 4. Decoupling rationale (vs. main) + +lean-md was developed in-tree (phases 1–9) and then **reverse-cut** before merge: +the in-tree engine never reaches `main`. The lmd-related deltas this branch lands +in lean-ctx are integration-only. + +| Class | Change (vs. main) | Why | +|---------|-------------------------------------------------------------------------|--------------------------------------------------| +| removed | `.lmd.md` auto-render delegation in `ctx_read.rs` → **raw read** | no in-tree engine renders; the addon renders on request | +| changed | `addon_registry.json`: `lmd` placeholder → **listed** `lean-md` entry | discoverability; install goes through the hosted pack | +| added | generic `extension_registry::RenderTransform` trait + registry | infra for `@render type=`, not lmd-exclusive | +| kept | ctx_* outbound tool surface | the addon calls them over the wire | +| added | gate tests `reverse_cut_gate.rs`, `ctx_read_lmd_md_raw.rs` | enforce the cut invariant + raw read | + +The engine, full `@directive` catalog, E-constructs, and spec now live in +`dasTholo/lean-md` and are **not** mirrored here. + +## 5. See also + +- Addon repo (engine + full directive reference): https://github.com/dasTholo/lean-md +- Addon manifest contract: `docs/contracts/addon-manifest-v1.md` (upstream) +- MCP tool catalog: [`appendix-mcp-tools.md`](appendix-mcp-tools.md) +- Decoupling design: https://github.com/dasTholo/lean-md (addon repo — hosts engine, spec & decoupling design) diff --git a/docs/reference/appendix-mcp-tools.md b/docs/reference/appendix-mcp-tools.md index 45e4d942a0..992ef77c66 100644 --- a/docs/reference/appendix-mcp-tools.md +++ b/docs/reference/appendix-mcp-tools.md @@ -146,3 +146,17 @@ shows the smallest tool profile that exposes the tool (`M` minimal, `S` standard `metrics`, `session`) independently of the static profile filter. 3. Lazy clients use `ctx_call` + `ctx_discover_tools` + `ctx_load_tools` to reach tools not in their active profile without listing all 79 upfront. + +--- + +## lean-md addon (`.lmd.md` render pipeline) + +`.lmd.md` / `.lean-md` rendering is provided by the **external lean-md addon** +(`dasTholo/lean-md`), not by lean-ctx itself. `ctx_md_render` / `ctx_md_check` are +exposed by the addon's MCP server once installed (`lean-ctx addon add @dasTholo/lean-md`); a +`.lmd.md` passed to `ctx_read` is returned **raw** — lean-ctx never renders it +(rendering is an explicit addon call). The `@directive` catalog and `@lean-md` header fields live in the +addon repo. + +- **Integration reference:** [`21-lean-md.md`](21-lean-md.md) +- **Addon repo:** https://github.com/dasTholo/lean-md diff --git a/rust/data/addon_registry.json b/rust/data/addon_registry.json index 93e028681a..b2d3446bc0 100644 --- a/rust/data/addon_registry.json +++ b/rust/data/addon_registry.json @@ -388,26 +388,27 @@ }, { "addon": { - "name": "letta", - "display_name": "Letta (MemGPT)", - "version": "", - "description": "Stateful agent runtime that manages its own memory like an operating system (core + archival tiers). Exposes MCP-compatible tools, but needs a running server + database, so it is listed rather than auto-installed.", - "author": "letta-ai", - "homepage": "https://github.com/letta-ai/letta", + "name": "lean-md", + "display_name": "Lean-MarkDown (LMD)", + "version": "0.2.0", + "description": "Directive-driven Markdown for agent plans. Reusable macros, phase-isolation and single-sourcing keep plans token-lean and cache-safe — written hard against lean-ctx, inspired by MarkdownAI.", + "author": "dasTholo", + "homepage": "https://github.com/dasTholo/lean-md", "license": "Apache-2.0", "categories": [ - "memory" + "plans", + "workflow" ], - "integration": "memory", + "integration": "mcp", "keywords": [ - "memory", - "agent-runtime", - "stateful", - "memgpt", - "competitor", - "mcp" + "markdown", + "plans", + "macros", + "directives", + "phase-isolation", + "subagents" ], - "min_lean_ctx": "3.8.0", + "min_lean_ctx": "3.9.6", "verified": false }, "mcp": { @@ -422,25 +423,24 @@ }, { "addon": { - "name": "lmd", - "display_name": "Lean-MarkDown (LMD)", + "name": "letta", + "display_name": "Letta (MemGPT)", "version": "", - "description": "Directive-driven Markdown for agent plans. Reusable macros, phase-isolation and single-sourcing keep plans token-lean and cache-safe — written hard against lean-ctx, inspired by MarkdownAI.", - "author": "dasTholo", - "homepage": "https://github.com/dasTholo/lean-ctx/tree/feat-lmd-v1", + "description": "Stateful agent runtime that manages its own memory like an operating system (core + archival tiers). Exposes MCP-compatible tools, but needs a running server + database, so it is listed rather than auto-installed.", + "author": "letta-ai", + "homepage": "https://github.com/letta-ai/letta", "license": "Apache-2.0", "categories": [ - "plans", - "workflow" + "memory" ], - "integration": "none", + "integration": "memory", "keywords": [ - "markdown", - "plans", - "macros", - "directives", - "phase-isolation", - "subagents" + "memory", + "agent-runtime", + "stateful", + "memgpt", + "competitor", + "mcp" ], "min_lean_ctx": "3.8.0", "verified": false diff --git a/rust/src/bin/gen_registry.rs b/rust/src/bin/gen_registry.rs index 06bd624aab..96a71f0725 100644 --- a/rust/src/bin/gen_registry.rs +++ b/rust/src/bin/gen_registry.rs @@ -31,7 +31,7 @@ fn main() { let data_dir = repo_data_dir(); let mut failed = false; - let targets: Vec<(&str, fn(&str) -> Result)> = vec![ + let targets: Vec<(&str, Canonicalize)> = vec![ ("addon_registry.json", canonical_addon), #[cfg(feature = "tree-sitter")] ("grammar_registry.json", canonical_grammar), @@ -85,6 +85,9 @@ fn main() { use lean_ctx::core::addons::registry_snapshot::{Snapshot, canonical_addon_registry}; +/// Canonicalizer for one registry file: raw JSON text -> validated `Snapshot`. +type Canonicalize = fn(&str) -> Result; + fn canonical_addon(text: &str) -> Result { canonical_addon_registry(text) } diff --git a/rust/src/core/addons/registry.rs b/rust/src/core/addons/registry.rs index 282b1883c8..dde31d6ba6 100644 --- a/rust/src/core/addons/registry.rs +++ b/rust/src/core/addons/registry.rs @@ -404,25 +404,26 @@ mod tests { } #[test] - fn flagship_lmd_is_listed() { - let lmd = get("lmd").expect("lmd in registry"); + fn flagship_lean_md_is_listed() { + let lmd = get("lean-md").expect("lean-md in registry"); assert_eq!(lmd.addon.author, "dasTholo"); assert!(!lmd.addon.homepage.is_empty()); - // Listed-only until it publishes an MCP endpoint — never fabricated. + // Listed-only: the addon ships as a hosted pack (`addon add @dasTholo/lean-md`), + // the bundled entry is discovery only — never a fabricated endpoint. assert!(!lmd.is_installable()); } #[test] fn search_matches_keywords_and_categories() { - assert!(search("markdown").iter().any(|m| m.addon.name == "lmd")); - assert!(search("plans").iter().any(|m| m.addon.name == "lmd")); - assert!(search("").iter().any(|m| m.addon.name == "lmd")); + assert!(search("markdown").iter().any(|m| m.addon.name == "lean-md")); + assert!(search("plans").iter().any(|m| m.addon.name == "lean-md")); + assert!(search("").iter().any(|m| m.addon.name == "lean-md")); assert!(search("definitely-no-such-term").is_empty()); } #[test] fn get_is_case_insensitive() { - assert!(get("LMD").is_some()); - assert!(get(" lmd ").is_some()); + assert!(get("LEAN-MD").is_some()); + assert!(get(" lean-md ").is_some()); } } diff --git a/rust/src/core/cache.rs b/rust/src/core/cache.rs index 20114a9c72..bbf4b6b066 100644 --- a/rust/src/core/cache.rs +++ b/rust/src/core/cache.rs @@ -1200,6 +1200,13 @@ mod tests { fn hebbian_eviction_bonus_is_wired() { // #3: files read together build a Hebbian association via store()'s // recording, and that association must feed the eviction bonus. + // + // Warm up tiktoken first: the very first count_tokens() in the process + // lazily loads the BPE tables (can exceed the 500ms co-access burst + // window). store() calls count_tokens() internally, so without warming + // up, the two store() calls below straddle that window and never + // associate — a flaky-empty bonus. Warming up keeps them in one burst. + let _ = count_tokens("warmup"); let mut cache = SessionCache::new(); cache.store("/a.rs", "fn a() {}"); cache.store("/b.rs", "fn b() {}"); diff --git a/rust/src/core/extension_registry.rs b/rust/src/core/extension_registry.rs index e186356d43..6119e37b1d 100644 --- a/rust/src/core/extension_registry.rs +++ b/rust/src/core/extension_registry.rs @@ -42,12 +42,20 @@ pub trait ReadMode: Send + Sync { fn render(&self, source: &str, path: &str) -> String; } +/// `@render type=` — a WASM-backed render transform. +/// `hint`: consumer hint (ai = 0, human = 1). +pub trait RenderTransform: Send + Sync { + fn name(&self) -> &str; + fn render(&self, input: &str, hint: i32) -> String; +} + /// Registry of pluggable read-modes, compressors, and chunkers. #[derive(Default)] pub struct ExtensionRegistry { read_modes: BTreeMap>, compressors: BTreeMap>, chunkers: BTreeMap>, + render_transforms: BTreeMap>, } impl ExtensionRegistry { @@ -134,6 +142,24 @@ impl ExtensionRegistry { pub fn chunker_names(&self) -> Vec { self.chunkers.keys().cloned().collect() } + + /// Register (or replace) a render transform by its name. + pub fn register_render_transform(&mut self, handler: Arc) { + self.render_transforms + .insert(handler.name().to_string(), handler); + } + + /// Look up a render transform by name. + #[must_use] + pub fn render_transform(&self, name: &str) -> Option> { + self.render_transforms.get(name).cloned() + } + + /// Registered render transform names (sorted). + #[must_use] + pub fn render_transform_names(&self) -> Vec { + self.render_transforms.keys().cloned().collect() + } } /// Process-global registry, seeded with built-ins on first access. @@ -319,6 +345,25 @@ mod tests { assert_eq!(c.compress("hi", None), "HI"); } + struct UpperRender; + impl RenderTransform for UpperRender { + fn name(&self) -> &str { + "upper" + } + fn render(&self, input: &str, hint: i32) -> String { + format!("{}:{}", hint, input.to_uppercase()) + } + } + + #[test] + fn render_transform_registers_and_resolves_with_hint() { + let mut reg = ExtensionRegistry::with_builtins(); + reg.register_render_transform(Arc::new(UpperRender)); + let r = reg.render_transform("upper").unwrap(); + assert_eq!(r.render("hi", 1), "1:HI"); + assert!(reg.render_transform_names().contains(&"upper".to_string())); + } + #[test] fn global_registry_seeds_builtins() { let reg = global().read().unwrap(); diff --git a/rust/src/lsp/format/mod.rs b/rust/src/lsp/format/mod.rs new file mode 100644 index 0000000000..e3689a8b96 --- /dev/null +++ b/rust/src/lsp/format/mod.rs @@ -0,0 +1,254 @@ +//! Formatter routing for `ctx_refactor action=reformat`: pick a formatter by +//! file extension, using built-in routing per extension. + +/// The formatter selected for a file: either the IDE HTTP backend or an external +/// shell command (template with a `{file}` placeholder). +#[derive(Debug, Clone, PartialEq, Eq)] +pub enum Formatter { + Jetbrains, + Command(String), +} + +/// Pick the formatter for `abs_path` using built-in defaults per extension. +/// Extension match is case-insensitive; no extension or an unknown extension → `Jetbrains`. +pub fn resolve_formatter(abs_path: &str) -> Formatter { + let ext = std::path::Path::new(abs_path) + .extension() + .and_then(|e| e.to_str()) + .map(str::to_ascii_lowercase) + .unwrap_or_default(); + builtin_default(&ext) +} + +/// Built-in routing when the config has no entry for this extension. +fn builtin_default(ext: &str) -> Formatter { + match ext { + "rs" => Formatter::Command("rustfmt {file}".to_string()), + _ => Formatter::Jetbrains, + } +} + +/// The binary name of a command template, for the `via ` output label. +pub fn command_label(template: &str) -> &str { + template.split_whitespace().next().unwrap_or("formatter") +} + +/// Split a command template into argv, substituting the `{file}` placeholder with +/// `abs_path`. `{file}` may be a standalone token or embedded in a token. If no +/// placeholder is present, `abs_path` is appended as the final argument. The path +/// is always a single argv element (spaces in the path are preserved). +pub fn build_argv(template: &str, abs_path: &str) -> Vec { + let mut argv: Vec = Vec::new(); + let mut saw_placeholder = false; + for tok in template.split_whitespace() { + if tok == "{file}" { + argv.push(abs_path.to_string()); + saw_placeholder = true; + } else if tok.contains("{file}") { + argv.push(tok.replace("{file}", abs_path)); + saw_placeholder = true; + } else { + argv.push(tok.to_string()); + } + } + if !saw_placeholder { + argv.push(abs_path.to_string()); + } + argv +} + +/// Run an external formatter command on `abs_path` with cwd `project_root` (so +/// tool config like `rustfmt.toml` is discovered). Returns `Err` with a clear +/// message if the binary is missing or the command exits non-zero. +pub fn run_command_formatter( + template: &str, + abs_path: &str, + project_root: &str, +) -> Result<(), String> { + let argv = build_argv(template, abs_path); + let (bin, rest) = argv + .split_first() + .ok_or_else(|| "INVALID_TARGET: empty formatter template".to_string())?; + let output = std::process::Command::new(bin) + .args(rest) + .current_dir(project_root) + .output() + .map_err(|e| { + if e.kind() == std::io::ErrorKind::NotFound { + format!("formatter '{bin}' not found in PATH") + } else { + format!("failed to run '{bin}': {e}") + } + })?; + if !output.status.success() { + let code = output + .status + .code() + .map_or_else(|| "signal".to_string(), |c| c.to_string()); + let stderr = String::from_utf8_lossy(&output.stderr); + return Err(format!("{bin} exited {code}: {}", stderr.trim())); + } + Ok(()) +} + +/// Hex BLAKE3 of the file content, for honest before/after change detection. +pub fn blake3_of(abs_path: &str) -> Result { + let bytes = std::fs::read(abs_path).map_err(|e| format!("FILE_NOT_FOUND: {abs_path}: {e}"))?; + Ok(crate::core::hasher::hash_hex(&bytes)) +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn rs_defaults_to_rustfmt() { + let f = resolve_formatter("/x/a.rs"); + assert!(matches!(f, Formatter::Command(ref t) if t == "rustfmt {file}")); + } + + #[test] + fn md_and_unknown_and_no_ext_default_to_jetbrains() { + assert!(matches!(resolve_formatter("/x/a.md"), Formatter::Jetbrains)); + assert!(matches!( + resolve_formatter("/x/a.txt"), + Formatter::Jetbrains + )); + assert!(matches!( + resolve_formatter("/x/README"), + Formatter::Jetbrains + )); + } + + #[test] + fn extension_is_case_insensitive() { + assert!(matches!( + resolve_formatter("/x/A.RS"), + Formatter::Command(_) + )); + } + + #[test] + fn command_label_is_first_token() { + assert_eq!(command_label("rustfmt {file}"), "rustfmt"); + assert_eq!(command_label("ruff format {file}"), "ruff"); + assert_eq!(command_label(""), "formatter"); + } + + #[test] + fn argv_substitutes_placeholder() { + assert_eq!( + build_argv("rustfmt {file}", "/x/a.rs"), + vec!["rustfmt".to_string(), "/x/a.rs".to_string()] + ); + assert_eq!( + build_argv("ruff format {file}", "/x/a.py"), + vec![ + "ruff".to_string(), + "format".to_string(), + "/x/a.py".to_string() + ] + ); + } + + #[test] + fn argv_appends_path_when_no_placeholder() { + assert_eq!( + build_argv("gofmt -w", "/x/a.go"), + vec!["gofmt".to_string(), "-w".to_string(), "/x/a.go".to_string()] + ); + } + + #[test] + fn argv_path_with_spaces_stays_one_arg() { + let argv = build_argv("rustfmt {file}", "/x/my dir/a.rs"); + assert_eq!( + argv, + vec!["rustfmt".to_string(), "/x/my dir/a.rs".to_string()] + ); + } + + #[test] + fn blake3_detects_change() { + let dir = tempfile::tempdir().unwrap(); + let f = dir.path().join("a.txt"); + std::fs::write(&f, "one").unwrap(); + let p = f.to_str().unwrap(); + let h1 = blake3_of(p).unwrap(); + let h2 = blake3_of(p).unwrap(); + assert_eq!(h1, h2, "same content → same hash"); + std::fs::write(&f, "two").unwrap(); + assert_ne!( + h1, + blake3_of(p).unwrap(), + "changed content → different hash" + ); + } + + #[test] + fn blake3_missing_file_errors() { + assert!(blake3_of("/no/such/file.xyz").is_err()); + } + + #[test] + fn run_command_missing_binary_errors() { + let dir = tempfile::tempdir().unwrap(); + let f = dir.path().join("a.rs"); + std::fs::write(&f, "fn x(){}\n").unwrap(); + let err = run_command_formatter( + "definitely-not-a-formatter-binary {file}", + f.to_str().unwrap(), + dir.path().to_str().unwrap(), + ) + .unwrap_err(); + assert!(err.contains("not found"), "got: {err}"); + } + + #[test] + fn run_command_nonzero_exit_errors() { + let dir = tempfile::tempdir().unwrap(); + let f = dir.path().join("a.rs"); + std::fs::write(&f, "fn x(){}\n").unwrap(); + // `false` exits 1 and ignores its args. + let err = run_command_formatter( + "false {file}", + f.to_str().unwrap(), + dir.path().to_str().unwrap(), + ) + .unwrap_err(); + assert!(err.contains("exited"), "got: {err}"); + } + + #[test] + fn run_rustfmt_formats_and_reports_change() { + // Gated: only runs when rustfmt is installed. + if std::process::Command::new("rustfmt") + .arg("--version") + .output() + .is_err() + { + eprintln!("SKIP: rustfmt not in PATH"); + return; + } + let dir = tempfile::tempdir().unwrap(); + let f = dir.path().join("a.rs"); + std::fs::write(&f, "fn x( ){let y=1;}\n").unwrap(); // deliberate drift + let p = f.to_str().unwrap(); + let before = blake3_of(p).unwrap(); + run_command_formatter("rustfmt {file}", p, dir.path().to_str().unwrap()).unwrap(); + let after = blake3_of(p).unwrap(); + assert_ne!( + before, after, + "rustfmt should have changed the drifted file" + ); + + // A second run is a no-op (already conformant). + let before2 = blake3_of(p).unwrap(); + run_command_formatter("rustfmt {file}", p, dir.path().to_str().unwrap()).unwrap(); + assert_eq!( + before2, + blake3_of(p).unwrap(), + "second run should be unchanged" + ); + } +} diff --git a/rust/src/lsp/mod.rs b/rust/src/lsp/mod.rs index 1883286e66..f8229fd70b 100644 --- a/rust/src/lsp/mod.rs +++ b/rust/src/lsp/mod.rs @@ -2,6 +2,7 @@ pub mod backend; pub mod client; pub mod config; pub mod edit_apply; +pub mod format; pub mod jetbrains_backend; pub mod port_discovery; pub mod router; diff --git a/rust/src/tools/ctx_refactor/ops.rs b/rust/src/tools/ctx_refactor/ops.rs index 0f29bdebff..9e3487b417 100644 --- a/rust/src/tools/ctx_refactor/ops.rs +++ b/rust/src/tools/ctx_refactor/ops.rs @@ -879,9 +879,10 @@ pub(super) fn resolve_reformat_scope( } } -/// Single-Phase reformat: resolve address → scope, jail, require live IDE, run the -/// IDE reformat, then Single-File evict (spec §5.3). No plan_hash, no preview. -fn render_reformat( +/// Jetbrains-arm renderer: run the IDE reformat and Single-File/Multi-File evict +/// EVERY changed path (spec §5.3). No plan_hash, no preview. Keeping the full +/// `changed_paths` list — and invalidating all of them — is the B2 contract. +pub(super) fn render_reformat( backend: &mut dyn crate::lsp::backend::LspBackend, project_root: &str, query: &crate::lsp::backend::ReformatQuery, @@ -903,6 +904,32 @@ fn render_reformat( ) } +/// Command-arm renderer (e.g. rustfmt, single file): hash the resolved single +/// file before/after so the report is honest, and only invalidate when the bytes +/// actually changed. blake3 runs on the resolved file path — NEVER a directory — +/// so a no-op run reports "unchanged", not a false "changed" (B2). +fn render_reformat_command( + template: &str, + abs_path: &str, + rel_path: &str, + project_root: &str, +) -> String { + let before = crate::lsp::format::blake3_of(abs_path).ok(); + if let Err(e) = crate::lsp::format::run_command_formatter(template, abs_path, project_root) { + return format!("ERROR: {e}"); + } + let after = crate::lsp::format::blake3_of(abs_path).ok(); + let changed = before.is_some() && before != after; + if changed { + crate::core::cli_cache::invalidate(abs_path); + } + let label = crate::lsp::format::command_label(template); + format!( + "reformat: '{rel_path}' via {label} — {}\n", + if changed { "changed" } else { "unchanged" } + ) +} + pub(super) fn handle_reformat_refactor(args: &Value, project_root: &str) -> String { let (abs_path, rel_path, scope) = match resolve_reformat_scope(args, project_root) { Ok(t) => t, @@ -912,19 +939,28 @@ pub(super) fn handle_reformat_refactor(args: &Value, project_root: &str) -> Stri if let Some(e) = deny_if_read_only(&abs_path) { return e; } - let mut backend = match live_jetbrains_backend(project_root) { - Ok(b) => b, - Err(e) => return format!("ERROR: {e}"), - }; - let optimize_imports = args - .get("optimize_imports") - .and_then(Value::as_bool) - .unwrap_or(false); - let query = crate::lsp::backend::ReformatQuery { - abs_path, - rel_path, - scope, - optimize_imports, - }; - render_reformat(backend.as_mut(), project_root, &query) + // T4 formatter routing: an external command (e.g. rustfmt) formats the single + // file directly (no IDE needed); otherwise defer to the live JetBrains backend. + match crate::lsp::format::resolve_formatter(&abs_path) { + crate::lsp::format::Formatter::Command(template) => { + render_reformat_command(&template, &abs_path, &rel_path, project_root) + } + crate::lsp::format::Formatter::Jetbrains => { + let mut backend = match live_jetbrains_backend(project_root) { + Ok(b) => b, + Err(e) => return format!("ERROR: {e}"), + }; + let optimize_imports = args + .get("optimize_imports") + .and_then(Value::as_bool) + .unwrap_or(false); + let query = crate::lsp::backend::ReformatQuery { + abs_path, + rel_path, + scope, + optimize_imports, + }; + render_reformat(backend.as_mut(), project_root, &query) + } + } } diff --git a/rust/src/tools/ctx_refactor/tests_ops.rs b/rust/src/tools/ctx_refactor/tests_ops.rs index 14a04412ca..1f0b9c8627 100644 --- a/rust/src/tools/ctx_refactor/tests_ops.rs +++ b/rust/src/tools/ctx_refactor/tests_ops.rs @@ -868,3 +868,160 @@ fn reformat_without_ide_is_backend_required() { "got: {out}" ); } + +/// Minimal backend whose `reformat` returns a canned `changed_paths` list. +struct ReformatStub { + changed: Vec, +} +impl crate::lsp::backend::LspBackend for ReformatStub { + fn open_file(&mut self, _u: &lsp_types::Uri, _l: &str, _t: &str) -> Result<(), String> { + Ok(()) + } + fn references( + &mut self, + _u: &lsp_types::Uri, + _p: lsp_types::Position, + _s: &str, + ) -> Result, String> { + Ok(vec![]) + } + fn definition( + &mut self, + _u: &lsp_types::Uri, + _p: lsp_types::Position, + ) -> Result { + Ok(lsp_types::GotoDefinitionResponse::Array(vec![])) + } + fn implementations( + &mut self, + _u: &lsp_types::Uri, + _p: lsp_types::Position, + _s: &str, + ) -> Result, String> { + Ok(vec![]) + } + fn rename( + &mut self, + _u: &lsp_types::Uri, + _p: lsp_types::Position, + _n: &str, + ) -> Result, String> { + Ok(None) + } + fn reformat( + &mut self, + _q: &crate::lsp::backend::ReformatQuery, + ) -> Result { + Ok(crate::lsp::backend::ReformatResult { + applied: true, + changed_paths: self.changed.clone(), + }) + } +} + +#[test] +fn reformat_command_path_reports_changed_and_invalidates_single_file() { + // rustfmt-gated: only runs when rustfmt is installed. + if std::process::Command::new("rustfmt") + .arg("--version") + .output() + .is_err() + { + eprintln!("SKIP: rustfmt not in PATH"); + return; + } + let dir = tempfile::tempdir().unwrap(); + std::fs::write(dir.path().join("drift.rs"), "fn x( ){let y=1;}\n").unwrap(); // drift + let root = dir.path().to_str().unwrap(); + let args = serde_json::json!({ "action": "reformat", "path": "drift.rs" }); + + // First run: rustfmt rewrites the drifted file → "changed". + let out = super::handle_reformat_refactor(&args, root); + assert!(out.contains("via rustfmt"), "got: {out}"); + assert!( + out.contains("— changed"), + "first run must report changed; got: {out}" + ); + + // Second run: already conformant → honest "unchanged" (B2: blake3 on the + // single file, never a directory). + let out2 = super::handle_reformat_refactor(&args, root); + assert!( + out2.contains("— unchanged"), + "second run must report unchanged; got: {out2}" + ); +} + +#[test] +fn reformat_jetbrains_scope_invalidates_all_changed_paths() { + // B2: the Jetbrains arm must keep every changed path, invalidate ALL of them, + // and report the true count. This test observes the REAL cache effect rather + // than the output string alone: it warms `cli_cache` for both changed paths, + // runs `render_reformat`, then asserts both entries were evicted. A B-regression + // that skips the invalidate loop (e.g. `.map(|_| ())`) leaves the — unchanged — + // entries cached, so the post-run reads stay `Hit` and this test goes red. + // + // The private data dir isolates the on-disk `cli_cache` store and serializes + // via the global env-lock, so warming/eviction is deterministic here. + let _data = crate::core::data_dir::isolated_data_dir(); + + let dir = tempfile::tempdir().unwrap(); + std::fs::write(dir.path().join("a.kt"), "val a = 1\n").unwrap(); + std::fs::write(dir.path().join("b.kt"), "val b = 1\n").unwrap(); + let root = dir.path().to_str().unwrap(); + + // Resolve exactly as `render_reformat` does for each changed path, so the cache + // keys line up: `invalidate` and `check_and_read` both funnel the abs path + // through `normalize_tool_path` (same key convention). + let abs_a = crate::core::path_resolve::resolve_tool_path(Some(root), None, "a.kt").unwrap(); + let abs_b = crate::core::path_resolve::resolve_tool_path(Some(root), None, "b.kt").unwrap(); + + // `check_and_read` re-inserts on a Miss, so it is a single-shot probe: read it + // exactly once per assertion. A `Hit` proves the entry is currently present. + let is_cached = |abs: &str| { + matches!( + crate::core::cli_cache::check_and_read(abs), + crate::core::cli_cache::CacheResult::Hit { .. } + ) + }; + + // Warm: the first read is a Miss that inserts the entry; the second must Hit. + let _ = crate::core::cli_cache::check_and_read(&abs_a); + let _ = crate::core::cli_cache::check_and_read(&abs_b); + assert!( + is_cached(&abs_a), + "premise: a.kt must be cached after warming" + ); + assert!( + is_cached(&abs_b), + "premise: b.kt must be cached after warming" + ); + + let mut be = ReformatStub { + changed: vec!["a.kt".into(), "b.kt".into()], + }; + let query = crate::lsp::backend::ReformatQuery { + abs_path: abs_a.clone(), + rel_path: "a.kt".into(), + scope: crate::lsp::backend::ReformatScope::File, + optimize_imports: false, + }; + let out = super::render_reformat(&mut be, root, &query); + assert!(out.contains("changed files: 2"), "got: {out}"); + assert!( + !out.contains("unchanged"), + "Jetbrains arm must not report unchanged; got: {out}" + ); + + // The real effect: BOTH changed paths were invalidated. The stub never rewrites + // the files, so their content hash is unchanged — the only way these reads can + // Miss is an actual eviction by the invalidate loop. + assert!( + !is_cached(&abs_a), + "render_reformat must invalidate a.kt (B2 regression: invalidate loop skipped)" + ); + assert!( + !is_cached(&abs_b), + "render_reformat must invalidate b.kt (B2 regression: invalidate loop skipped)" + ); +} diff --git a/rust/tests/ctx_read_lmd_md_raw.rs b/rust/tests/ctx_read_lmd_md_raw.rs new file mode 100644 index 0000000000..4ec73d4b2e --- /dev/null +++ b/rust/tests/ctx_read_lmd_md_raw.rs @@ -0,0 +1,52 @@ +//! `.lmd.md` reads are raw — like any other file. +//! +//! After the lmd reverse-cut, lean-ctx has no `.lmd.md`-specific code path: a +//! read returns the raw source verbatim (never an error, never a half-rendered +//! document). Rendering `.lmd.md` is owned entirely by the external lean-md +//! addon (`ctx_md_render` / CLI `lean-md render`) and is out of scope here. +//! This end-to-end check drives the freshly built `lean-ctx` binary and asserts +//! a `.lmd.md` read surfaces the raw marker. +use std::process::Command; + +/// The lean-ctx binary under test — the freshly built one, never the (possibly +/// stale) `lean-ctx` on PATH. +const LEAN_CTX_BIN: &str = env!("CARGO_BIN_EXE_lean-ctx"); + +#[test] +fn ctx_read_lmd_md_returns_raw_source() { + // No addon installed (default CI state) → a read of a `.lmd.md` must surface + // the raw source, never an error or a half-rendered document. We assert the + // marker survives the read. + // + // Hermetic isolation: a direct CLI `read` caches by design (read_cmd.rs), and + // the persistent stub index lives under `LEAN_CTX_DATA_DIR`. If the fixture + // path (or a prior run/retry on the same runner) already seeded that index, + // the read returns an `[unchanged …]` cache stub instead of the body and this + // assertion breaks — an artefact of test hygiene, not of `.lmd.md` handling. + // So we give the spawned binary a fresh, private data dir and force `--fresh`, + // making this an unconditional first read that never depends on nor pollutes + // the real store. + let fixture = tempfile::tempdir().expect("fixture dir"); + let data_dir = tempfile::tempdir().expect("isolated LEAN_CTX_DATA_DIR"); + let f = fixture.path().join("d.lmd.md"); + std::fs::write(&f, "@date\nRAW_DELEGATION_MARKER\n").unwrap(); + + let out = Command::new(LEAN_CTX_BIN) + .env("LEAN_CTX_DATA_DIR", data_dir.path()) + .args(["read", f.to_str().unwrap(), "--mode", "full", "--fresh"]) + .output() + .expect("lean-ctx read"); + let text = String::from_utf8_lossy(&out.stdout); + + assert!( + text.contains("RAW_DELEGATION_MARKER"), + "without an addon a .lmd.md read must return raw text (no error, no half-render): {text}" + ); + // The `@date` directive is what discriminates raw from rendered: any renderer + // consumes it and substitutes a date. Asserting on the plain marker alone would + // survive a re-introduced render pass, so this is the line that makes the gate bite. + assert!( + text.contains("@date"), + "a rendered .lmd.md would have consumed the @date directive; the read must be raw: {text}" + ); +} diff --git a/rust/tests/reverse_cut_gate.rs b/rust/tests/reverse_cut_gate.rs new file mode 100644 index 0000000000..2a40a6ede2 --- /dev/null +++ b/rust/tests/reverse_cut_gate.rs @@ -0,0 +1,169 @@ +//! Reverse-cut gate tests (#6) — verifies Cut-Invariante §4.1: +//! - no stray lmd engine symbols outside the allowed survivor paths +//! - no rushdown/evalexpr render-engine deps in Cargo.toml +//! - root lean-md/ seed dir was removed +//! - ctx_* outbound tool surface (P2-Task6 contract) survives the cut + +use std::path::PathBuf; +use std::process::Command; + +fn repo_root() -> PathBuf { + let rust_dir = PathBuf::from(env!("CARGO_MANIFEST_DIR")); + rust_dir.parent().map(PathBuf::from).unwrap_or(rust_dir) +} + +/// Test 1 — Cut-Invariante: no lmd engine code outside the allowed survivor paths. +/// +/// Allowed survivors (all are addon-name/slug references or fixtures, never the +/// removed in-tree render ENGINE): +/// - `rust/src/core/addons/registry.rs` — flagship/search tests use "lmd"/"lean-md" as addon name/slug +/// - `rust/src/core/addons/manifest.rs` — is_slug test uses "lmd" slug + manifest fixture uses "lean-md" +/// - `rust/src/core/addons/audit.rs` — fixture uses "lean-md" as addon name in TOML +/// - `rust/src/core/config/tests.rs` — lean_md_removal regression test +/// - `rust/src/core/addons/publish.rs` — addon-pack publish tests use "lean-md" as example addon name +/// - `rust/src/core/context_package/skills.rs` — skills-pack test uses "@das-tholo/lean-md-skills" example name +/// - `rust/src/core/context_package/verify.rs` — package-verify fixtures use "lean-md" as example addon name +/// - `rust/src/cli/addon_deps.rs` — self-dependency guard tests use "@dasTholo/lean-md" as scoped addon slug +/// - `rust/src/core/addons/pack_env.rs` — {pack_dir:} expander tests use "@dasTholo/lean-md-skills" example pack +/// +/// The reverse-cut removed the in-tree lmd ENGINE (was entirely in src/lmd/, now deleted). +/// Only addon-name/slug references remain, in exactly these nine survivor paths. +/// `ctx_read.rs` is deliberately NOT among them: after the reverse-cut it carries +/// no lmd knowledge at all, so the gate scans it like any other file. +/// Any NEW lmd code line in any other file will be caught by this gate. +/// +/// Doc/line comments (`//`, `///`) and block-comment continuation lines (`*`) are +/// tolerated everywhere — only code hits trigger failure. +/// `git grep` exit code 1 means no matches at all — that is the expected success case. +#[test] +fn no_lmd_symbols_outside_docs_and_hook() { + let root = repo_root(); + + let out = Command::new("git") + .current_dir(&root) + .args([ + "grep", + "-nIE", + "lmd|lean[-_]md|LeanMd|CtxMd", + "--", + "rust/src", + ":!rust/src/core/addons/registry.rs", + ":!rust/src/core/addons/manifest.rs", + ":!rust/src/core/addons/audit.rs", + ":!rust/src/core/config/tests.rs", + ":!rust/src/core/addons/publish.rs", + ":!rust/src/core/context_package/skills.rs", + ":!rust/src/core/context_package/verify.rs", + ":!rust/src/cli/addon_deps.rs", + ":!rust/src/core/addons/pack_env.rs", + ]) + .output() + .expect("git grep failed to run"); + + // exit code 1 = no matches (not an error); only inspect stdout + let stdout = String::from_utf8_lossy(&out.stdout); + + let code_hits: Vec<&str> = stdout + .lines() + .filter(|line| { + // format: path:linenum:BODY — skip comment-only lines (// prefix) + // and block-comment continuation lines (* prefix after trim) + let parts: Vec<&str> = line.splitn(3, ':').collect(); + if parts.len() < 3 { + return false; + } + let body = parts[2].trim_start(); + !body.is_empty() && !body.starts_with("//") && !body.starts_with('*') + }) + .collect(); + + assert!( + code_hits.is_empty(), + "stray lmd code symbols found outside allowed paths:\n{}", + code_hits.join("\n") + ); +} + +/// Test 2 — No rushdown/evalexpr render-engine dependencies in Cargo.toml. +#[test] +fn cargo_manifest_has_no_lmd_render_deps() { + let root = repo_root(); + let toml = + std::fs::read_to_string(root.join("rust/Cargo.toml")).expect("rust/Cargo.toml not found"); + + assert!( + !toml.contains("rushdown"), + "rushdown must not appear in rust/Cargo.toml after the reverse-cut" + ); + assert!( + !toml.contains("evalexpr"), + "evalexpr must not appear in rust/Cargo.toml after the reverse-cut" + ); +} + +/// Test 3 — The root lean-md/ seed directory was removed by the reverse-cut. +#[test] +fn root_lean_md_seed_dir_is_removed() { + let root = repo_root(); + assert!( + !root.join("lean-md").exists(), + "root lean-md/ seed dir must be removed after the reverse-cut" + ); +} + +/// Test 4 — The ctx_* outbound-tool surface (P2-Task6 contract) survives the cut. +/// +/// Only `ctx_md_render` and `ctx_md_check` were removed; the core ctx_* tools +/// that the lean-md addon would call back into must still be registered. +#[test] +fn lean_md_addon_outbound_tool_surface_survives() { + let reg = lean_ctx::server::registry::build_registry(); + + let required_tools = [ + "ctx_read", + "ctx_shell", + "ctx_edit", + "ctx_search", + "ctx_outline", + "ctx_impact", + "ctx_repomap", + "ctx_review", + "ctx_routes", + "ctx_smells", + "ctx_architecture", + "ctx_graph", + "ctx_callgraph", + "ctx_knowledge", + "ctx_handoff", + "ctx_agent", + "ctx_refactor", + "ctx_analyze", + "ctx_tree", + ]; + + for tool in required_tools { + assert!( + reg.get(tool).is_some(), + "outbound contract tool `{tool}` must stay registered after the reverse-cut" + ); + } +} + +/// Test 5 — The two lmd render-engine tools were removed by the reverse-cut. +/// +/// Complements Test 4: the gate asserts BOTH sides of the invariant: +/// - outbound ctx_* surface survives (Test 4) +/// - in-tree render tools are gone (this test) +#[test] +fn ctx_md_render_and_check_are_removed() { + let reg = lean_ctx::server::registry::build_registry(); + + assert!( + reg.get("ctx_md_render").is_none(), + "ctx_md_render must be cut by the reverse-cut" + ); + assert!( + reg.get("ctx_md_check").is_none(), + "ctx_md_check must be cut by the reverse-cut" + ); +}