diff --git a/CHANGELOG.md b/CHANGELOG.md index 830e20c..84d437b 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -46,6 +46,12 @@ here rather than left to be discovered. ### Changed +- **The chat image in the documentation is a real session.** It was reconstructed from string + literals in `ChatCommand.cs`, because the REPL cannot be piped to a file — a fact disclosed only + inside the generator script, where nobody reads it. It was captured by hand against a live + workspace on 2026-08-06, which also verified the REPL, a context-keeping follow-up and `/sql` + for the first time. `ask.svg`'s SQL box, misaligned since an earlier identifier substitution + dropped its padding, is fixed at the same time. - The README no longer states a test count. The number had drifted from 89 to 175 without anyone noticing, which is what prose claims do. - **Positioning corrected.** Databricks CLI v1.10.0 ships `databricks genie ask`, which holds a diff --git a/README.md b/README.md index f99542d..3d26050 100644 --- a/README.md +++ b/README.md @@ -205,7 +205,8 @@ promise; it is a record of evidence. |---|---| | Unit and contract tests | Run on Windows and Linux in CI, on every push and pull request | | Azure Databricks, live workspace | `agents list`, `ask`, `pack run` and every output format verified against a real Genie Agent on 2026-08-01 | -| `chat`, feedback, full-result download, visualizations | Contract tests only — **not** exercised live | +| `chat` | Verified live on 2026-08-06 — the REPL, a follow-up keeping its context, and `/sql`. Its other slash commands were not exercised | +| Feedback, full-result download, visualizations | Contract tests only — **not** exercised live | | Chunked results assembled beyond the first chunk | Mechanism verified live on 2026-08-05 — chunk reads on a Genie-executed statement are permitted, and the link, response shape and row arithmetic all behave as the client assumes. Genie emitting a multi-chunk result was **not** reproduced | | Unattended service-principal authentication | Request shape verified against the live token endpoint; the exchange with real service-principal credentials was **not** run | | AWS Databricks | Not tested | diff --git a/ROADMAP.md b/ROADMAP.md index 74b8267..5063d42 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -21,9 +21,10 @@ Status: in development. - [x] CLI output tests across terminal widths, unicode, nulls and escape-sequence injection - [ ] Live coverage for three paths that resist automation — each blocked for a different reason, recorded here rather than left as one vague gap: - - `chat` — the REPL refuses to start without an interactive terminal, by design, so it cannot be - driven from CI or an agent session. Its underlying follow-up call *is* covered live; the loop - around it is not. Needs a human at a terminal. + - ~~`chat`~~ — **done, 2026-08-06.** The REPL refuses to start without an interactive terminal, + so it needed a human at one. A live session covering a question, a follow-up that kept its + context, and `/sql` is recorded in [compatibility.md](docs/compatibility.md) and is now the + transcript behind the documentation's chat image. Its other slash commands remain uncovered. - Chunked results — assembled by following the link Databricks supplies ([ADR 0004](docs/decisions/0004-complete-a-chunked-result-by-following-the-link-databricks-supplies.md)). The mechanism was verified live on 2026-08-05, including the permission question that was the diff --git a/docs/assets/ask.svg b/docs/assets/ask.svg index 821f780..c9b2091 100644 --- a/docs/assets/ask.svg +++ b/docs/assets/ask.svg @@ -22,7 +22,7 @@ ╰────────────────────────┴─────────────────────────────────────────────────────╯ ╭─Generated SQL─────────────────────────────────────────────╮ │ SELECT `region`, SUM(`revenue_eur`) AS total_revenue_eur │ - │ FROM `sales`.`public`.`regional_revenue` │ + │ FROM `sales`.`public`.`regional_revenue` │ │ WHERE `region` IS NOT NULL AND `revenue_eur` IS NOT NULL │ │ GROUP BY `region` │ ╰───────────────────────────────────────────────────────────╯ diff --git a/docs/assets/chat.svg b/docs/assets/chat.svg index 241f7a4..4b7e8e7 100644 --- a/docs/assets/chat.svg +++ b/docs/assets/chat.svg @@ -1,32 +1,57 @@ - + lakespeak chat --agent sales - - + + - lakespeak chat --agent sales + lakespeak chat --agent sales LakeSpeak.NET — independent, not a Databricks product Agent: Sales Intelligence Type /help for commands, /exit to leave. - You: How did revenue change last quarter? - Revenue increased by 14.2% compared with the previous quarter. - ╭────────────────────────┬───────────────────────────╮ - │ region │ total_revenue_eur │ - ├────────────────────────┼───────────────────────────┤ - │ Germany │ 4500000.00 │ - │ France │ 3350000.50 │ - │ Spain │ 1780000.25 │ - ╰────────────────────────┴───────────────────────────╯ - You: and break that down by product - Electronics led in Germany; Home & Garden led in Spain. - You: /sql - ╭─Generated SQL──────────────────────────────────────╮ - │ SELECT region, product_line, SUM(revenue_eur) │ - │ FROM sales.public.regional_revenue │ - │ GROUP BY region, product_line │ - ╰────────────────────────────────────────────────────╯ - You: /export q3-by-product.csv - Wrote q3-by-product.csv (12 rows). It contains governed data; look after it. - You: /exit + You: What is the total revenue by region? + Genie is analyzing your question… + Waiting for a SQL warehouse… + Preparing answer… + The total revenue by region is as follows: **Germany** generated €4,500,000.00, **France** generated €3,350,000.50, and + **Spain** generated €1,780,000.25. Among these regions, Germany had the highest total revenue, while Spain had the + lowest. + ╭───────────────────────────────────┬──────────────────────────────────────────────────────────────────────────────────╮ + │ region │ total_revenue_eur │ + ├───────────────────────────────────┼──────────────────────────────────────────────────────────────────────────────────┤ + │ Germany │ 4500000.00 │ + │ France │ 3350000.50 │ + │ Spain │ 1780000.25 │ + ╰───────────────────────────────────┴──────────────────────────────────────────────────────────────────────────────────╯ + /sql /result /export + You: and break that down by quarter + Genie is analyzing your question… + Waiting for a SQL warehouse… + Genie is analyzing your question… + Preparing answer… + The total revenue by region is broken down by quarter as follows: + - **France**: €1,650,000.00 in 2025-Q4 and €1,700,000.50 in 2026-Q1 + - **Germany**: €2,100,000.00 in 2025-Q4 and €2,400,000.00 in 2026-Q1 + - **Spain**: €880,000.00 in 2025-Q4 and €900,000.25 in 2026-Q1 + Revenue increased from Q4 2025 to Q1 2026 in all regions, with Germany showing the largest growth. + ╭───────────────────────────┬───────────────────────────┬──────────────────────────────────────────────────────────────╮ + │ region │ quarter │ total_revenue_eur │ + ├───────────────────────────┼───────────────────────────┼──────────────────────────────────────────────────────────────┤ + │ France │ 2025-Q4 │ 1650000.00 │ + │ France │ 2026-Q1 │ 1700000.50 │ + │ Germany │ 2025-Q4 │ 2100000.00 │ + │ Germany │ 2026-Q1 │ 2400000.00 │ + │ Spain │ 2025-Q4 │ 880000.00 │ + │ Spain │ 2026-Q1 │ 900000.25 │ + ╰───────────────────────────┴───────────────────────────┴──────────────────────────────────────────────────────────────╯ + /sql /result /export + You: /sql + ╭─Generated SQL──────────────────────────────────────────────────────────────────────╮ + │ SELECT `region`, `quarter`, SUM(`revenue_eur`) AS total_revenue_eur │ + │ FROM `sales`.`public`.`regional_revenue` │ + │ WHERE `region` IS NOT NULL AND `quarter` IS NOT NULL AND `revenue_eur` IS NOT NULL │ + │ GROUP BY `region`, `quarter` │ + │ ORDER BY `region`, `quarter` │ + ╰────────────────────────────────────────────────────────────────────────────────────╯ + You: /exit diff --git a/docs/assets/generate.py b/docs/assets/generate.py index 65ad84e..fdb9cd5 100644 --- a/docs/assets/generate.py +++ b/docs/assets/generate.py @@ -3,10 +3,17 @@ python docs/assets/generate.py -The transcripts are real output captured from the CLI against a live Databricks workspace, with -that workspace's identifiers replaced by synthetic ones before they were stored. `chat.txt` is the -exception: the chat REPL refuses to start without an interactive terminal, so it cannot be piped -to a file. That one is reconstructed from the literal strings in `ChatCommand.cs`. +Every transcript here is real output captured from the CLI against a live Databricks workspace, +with that workspace's identifiers replaced by synthetic ones before it was stored. + +`chat.txt` used to be the exception, reconstructed from the literal strings in `ChatCommand.cs`, +because the REPL refuses to start without an interactive terminal and so cannot be piped to a +file. It was captured for real on 2026-08-06 by copying the terminal buffer of a live session -- +which is the only way to do it, and is the way to do it again if the output changes. + +When substituting identifiers, keep the box-drawing borders aligned: the synthetic table name is +shorter than the real one, and replacing it without restoring the displaced padding leaves a +visibly ragged box. That had already happened once, in `ask.txt`. SVG rather than screenshots, deliberately: the assets stay text, so they diff, scale, and need no binary blobs in the repository -- and when the CLI's output changes, regenerating produces a diff --git a/docs/assets/transcripts/ask.txt b/docs/assets/transcripts/ask.txt index 3b456ca..000d2b3 100644 --- a/docs/assets/transcripts/ask.txt +++ b/docs/assets/transcripts/ask.txt @@ -16,7 +16,7 @@ Germany generated the highest revenue among the listed regions. ╰────────────────────────┴─────────────────────────────────────────────────────╯ ╭─Generated SQL─────────────────────────────────────────────╮ │ SELECT `region`, SUM(`revenue_eur`) AS total_revenue_eur │ -│ FROM `sales`.`public`.`regional_revenue` │ +│ FROM `sales`.`public`.`regional_revenue` │ │ WHERE `region` IS NOT NULL AND `revenue_eur` IS NOT NULL │ │ GROUP BY `region` │ ╰───────────────────────────────────────────────────────────╯ diff --git a/docs/assets/transcripts/chat.txt b/docs/assets/transcripts/chat.txt index abc3daa..c33d6ca 100644 --- a/docs/assets/transcripts/chat.txt +++ b/docs/assets/transcripts/chat.txt @@ -2,31 +2,54 @@ LakeSpeak.NET — independent, not a Databricks product Agent: Sales Intelligence Type /help for commands, /exit to leave. -You: How did revenue change last quarter? - -Revenue increased by 14.2% compared with the previous quarter. - -╭────────────────────────┬───────────────────────────╮ -│ region │ total_revenue_eur │ -├────────────────────────┼───────────────────────────┤ -│ Germany │ 4500000.00 │ -│ France │ 3350000.50 │ -│ Spain │ 1780000.25 │ -╰────────────────────────┴───────────────────────────╯ - -You: and break that down by product - -Electronics led in Germany; Home & Garden led in Spain. +You: What is the total revenue by region? +Genie is analyzing your question… +Waiting for a SQL warehouse… +Preparing answer… + +The total revenue by region is as follows: **Germany** generated €4,500,000.00, **France** generated €3,350,000.50, and +**Spain** generated €1,780,000.25. Among these regions, Germany had the highest total revenue, while Spain had the +lowest. + +╭───────────────────────────────────┬──────────────────────────────────────────────────────────────────────────────────╮ +│ region │ total_revenue_eur │ +├───────────────────────────────────┼──────────────────────────────────────────────────────────────────────────────────┤ +│ Germany │ 4500000.00 │ +│ France │ 3350000.50 │ +│ Spain │ 1780000.25 │ +╰───────────────────────────────────┴──────────────────────────────────────────────────────────────────────────────────╯ +/sql /result /export + +You: and break that down by quarter +Genie is analyzing your question… +Waiting for a SQL warehouse… +Genie is analyzing your question… +Preparing answer… + +The total revenue by region is broken down by quarter as follows: +- **France**: €1,650,000.00 in 2025-Q4 and €1,700,000.50 in 2026-Q1 +- **Germany**: €2,100,000.00 in 2025-Q4 and €2,400,000.00 in 2026-Q1 +- **Spain**: €880,000.00 in 2025-Q4 and €900,000.25 in 2026-Q1 +Revenue increased from Q4 2025 to Q1 2026 in all regions, with Germany showing the largest growth. + +╭───────────────────────────┬───────────────────────────┬──────────────────────────────────────────────────────────────╮ +│ region │ quarter │ total_revenue_eur │ +├───────────────────────────┼───────────────────────────┼──────────────────────────────────────────────────────────────┤ +│ France │ 2025-Q4 │ 1650000.00 │ +│ France │ 2026-Q1 │ 1700000.50 │ +│ Germany │ 2025-Q4 │ 2100000.00 │ +│ Germany │ 2026-Q1 │ 2400000.00 │ +│ Spain │ 2025-Q4 │ 880000.00 │ +│ Spain │ 2026-Q1 │ 900000.25 │ +╰───────────────────────────┴───────────────────────────┴──────────────────────────────────────────────────────────────╯ +/sql /result /export You: /sql - -╭─Generated SQL──────────────────────────────────────╮ -│ SELECT region, product_line, SUM(revenue_eur) │ -│ FROM sales.public.regional_revenue │ -│ GROUP BY region, product_line │ -╰────────────────────────────────────────────────────╯ - -You: /export q3-by-product.csv -Wrote q3-by-product.csv (12 rows). It contains governed data; look after it. - +╭─Generated SQL──────────────────────────────────────────────────────────────────────╮ +│ SELECT `region`, `quarter`, SUM(`revenue_eur`) AS total_revenue_eur │ +│ FROM `sales`.`public`.`regional_revenue` │ +│ WHERE `region` IS NOT NULL AND `quarter` IS NOT NULL AND `revenue_eur` IS NOT NULL │ +│ GROUP BY `region`, `quarter` │ +│ ORDER BY `region`, `quarter` │ +╰────────────────────────────────────────────────────────────────────────────────────╯ You: /exit diff --git a/docs/compatibility.md b/docs/compatibility.md index d52ee62..efb3b94 100644 --- a/docs/compatibility.md +++ b/docs/compatibility.md @@ -58,9 +58,27 @@ The live suite in `tests/LakeSpeak.LiveIntegrationTests` reproduces all of this. `DATABRICKS_HOST`, `DATABRICKS_TOKEN` and `LAKESPEAK_LIVE_AGENT` set: `dotnet test -c Release --filter "Category=Live"`. -What this still does **not** exercise: `chat` (needs an interactive terminal), chunked result -assembly, the Genie full-result download endpoints, visualizations, and `QUERY_RESULT_EXPIRED` -recovery. Those remain covered by contract tests only. +What this still does **not** exercise: the Genie full-result download endpoints, visualizations, +and `QUERY_RESULT_EXPIRED` recovery. Those remain covered by contract tests only. + +## `chat`, verified live — 2026-08-06 + +The one path that resists automation entirely: the REPL refuses to start without an interactive +terminal, by design, so no CI job or agent session can drive it. It was run by hand instead. + +| Path | Result | +|---|---| +| REPL start against a live Agent | Banner, Agent name, prompt | +| A question | Answer text plus a rendered result table | +| **A follow-up** — "and break that down by quarter" | Kept its context and re-grouped the same figures by quarter. This is the behaviour the whole project is built around, and it had never been observed end to end before | +| `/sql` | Printed the generated SQL for the follow-up, not the first question | +| `/exit` | Clean exit | + +`docs/assets/transcripts/chat.txt` is that session, with the workspace's identifiers replaced by +the synthetic ones the other transcripts use. It was previously reconstructed from string +literals; it is now captured output like every other transcript here. + +Not exercised: `/new`, `/agents`, `/use`, `/result`, `/export`, `/thumbs-up`, `/thumbs-down`. ## Local verification — 2026-08-05 diff --git a/docs/getting-started.md b/docs/getting-started.md index ca20ffc..c0bf6ac 100644 --- a/docs/getting-started.md +++ b/docs/getting-started.md @@ -118,7 +118,7 @@ Follow-up questions keep their context, so "and break that down by product" work for the slash commands, `/sql` to see the statement behind the last answer, `/export report.csv` to save the result, `/exit` to leave. -a lakespeak chat session with follow-up questions, /sql and /export +a real lakespeak chat session: a question, a follow-up that keeps its context, and /sql showing the generated query `chat` needs a real terminal. For scripts, use `ask`.