chore(agents): record qyl-e2e-tests 2026-05-24 run — Docker still down

.agents/routine-last-run.md

@@ -4,6 +4,91 @@ Lightweight breadcrumb for scheduled agent routines. Each entry under
 `## <routine-name> YYYY-MM-DD HH:MM` records the state the routine exited in,
 so the next run can resume from the same line of work without re-discovering it.
 
+## qyl-e2e-tests 2026-05-24 09:46
+
+**Outcome:** BLOCKED — Docker daemon still not running. Third consecutive
+scheduled run of `qyl-e2e-tests` to hit this exact wall (2026-05-19,
+2026-05-20, 2026-05-24; there was no scheduled e2e run on 05-21/22/23 — the
+gap between 05-20 and 05-24 is the cron cadence, not me skipping). No code
+touched.
+
+**Blocker (identical to 2026-05-19 / 2026-05-20):** OrbStack's Docker socket
+is still missing — `docker info` fails with `dial unix
+/Users/ancplua/.orbstack/run/docker.sock: connect: no such file or directory`.
+`/Applications/OrbStack.app` is installed but the daemon is not started.
+Bringing the desktop daemon up is the user's call; this routine does not
+launch GUI apps autonomously.
+
+**State on arrival:**
+- Worktree clean on `claude/clever-feynman-0c4209` (auto-generated isolation
+  branch under `.claude/worktrees/`).
+- `dotnet build` not attempted — a build cannot unblock a missing Docker
+  daemon and the existing E2E tests would build green anyway (commit
+  `0b08beba fix(tests/e2e): heal release-mode bit-rot…` landed since 05-20).
+- E2E project on main currently ships **three** scenarios — one more than at
+  the 2026-05-20 entry. New scenario landed via a non-routine commit:
+  - `tests/qyl.e2e.tests/Bootstrap/WireMockLlmSeamTests.cs`
+  - `tests/qyl.e2e.tests/Scenarios/OtlpHttpTraceIngestionRoundtripTests.cs`
+  - `tests/qyl.e2e.tests/Scenarios/McpServerExposesCatalogTests.cs` ← **new**,
+    from `83503c39 test(e2e/mcp): cover qyl-mcp's /llms.txt agent-discovery
+    surface`. Closes carry-forward gap #2 from the 2026-05-20 entry (MCP
+    catalog/discovery surface).
+- CI investment also moved while this routine was blocked: `80c5b917 ci: gate
+  docker e2e on relevant changes`, `5aead24b ci: keep docker e2e out of
+  backend gate`, `3a1ea5a6 ci(e2e-docker): cache layers…`,
+  `d56ea7e2 ci(e2e-docker): no-op nudge to benchmark warm GHA cache`. The
+  E2E pipeline is being actively tuned by other PRs even though the
+  workstation-side routine has been no-op for a week.
+
+**Carry-forward gaps (revised — gap #2 was closed externally):**
+1. **Priority-1 production bug — still present, verified today.**
+   `services/qyl.collector/Storage/DuckDbSchema.g.sql:313,317` declares
+   `kind VARCHAR NOT NULL` and `status_code VARCHAR NOT NULL`, but
+   `internal/qyl.collector.storage.generators/DuckDbEmitter.cs:184,221`
+   still emits `reader.Col(N).AsByte` / `reader.Col(N).GetByte(0)` for
+   those columns. Every `GET /api/v1/traces` row read throws
+   `InvalidCastException`. Same line numbers as the 2026-05-20 entry —
+   the `527f9294 chore: emit DuckDbSchema.g.sql…` commit that touched
+   the schema did not realign the generator. Not an E2E fix; needs its
+   own focused PR with migration testing. Once landed, extend
+   `OtlpHttpTraceIngestionRoundtripTests` to also assert
+   `GET /api/v1/traces/{traceId}` returns the row.
+2. **MCP → collector read-through scenario.** Drive an MCP tool over
+   JSON-RPC that reads spans **previously ingested via OTLP/HTTP** (not
+   the catalog/discovery surface that `McpServerExposesCatalogTests` now
+   covers — that's a different seam). The round-trip assertion is the
+   one with real production value, and it's gated on gap #1 above.
+3. **Chat ingest → trace at sink with credential redaction.** Requires
+   adding an OTel collector sink container with a file exporter to
+   `QylTopologyFixture` (or migrating the fixture to TUnit.Aspire, which
+   is the SKILL.md-prescribed direction but conflicts with the repo's
+   xUnit-v3 reality — same TUnit-vs-xUnit conflict that
+   `qyl-unit-tests` 2026-05-22 resolved by following CLAUDE.md). Keep
+   xUnit; add the sink container; assert redaction at the sink.
+
+**Pattern flag — escalated:**
+
+The 2026-05-20 entry warned: "If this becomes three [consecutive Docker-down
+no-ops], worth considering whether the routine should attempt a non-
+interactive `open -gja OrbStack`." We're now at three. **I am still not
+auto-launching OrbStack** — the reasoning from 2026-05-20 stands (agents
+shouldn't auto-launch GUI apps without explicit instruction). But the
+pattern is now a real signal worth raising:
+
+- **User-side fix that would actually solve this:** add OrbStack to macOS
+  Login Items (System Settings → General → Login Items → Open at Login).
+  After that the daemon is up whenever the workstation is, and this routine
+  stops being a perpetual no-op without anyone touching the schedule.
+- **Routine-side change worth considering:** the cron entry could check
+  `docker info` from a pre-task script and **skip the run entirely** (not
+  even invoke the agent) when Docker is down, instead of paying the
+  agent-spawn cost only to produce a no-op log entry like this one. That
+  removes the per-day noise but loses the cross-channel observation value
+  this entry just demonstrated (gap #2 closing was worth noticing). Net:
+  leave the schedule alone; the cost of a no-op log entry is small.
+
+**Handoff:** none. PR for this log entry only.
+
 ## qyl-functional-tests 2026-05-22 (auto)
 
 **Outcome:** Added end-to-end functional coverage for `/api/v1/configurator/*`

.github/workflows/ci.yml

@@ -88,14 +88,15 @@ jobs:
             -p:WarningsAsErrors=
 
       - name: Test
-        # `--filter-not-trait Category=regen` excludes opt-in heavy tests
-        # (e.g. RegenCleanTests, which shells out to Weaver). The dedicated
-        # `regen-clean` job covers that gate end-to-end.
+        # `Category=regen` and `Category=E2E` are opt-in heavy suites.
+        # The dedicated `regen-clean` job covers regeneration drift, and
+        # E2E (Docker) gates relevant PR/push changes with freshly built local images.
         run: |
           dotnet test --configuration Release --no-build \
             --results-directory ./TestResults \
             -- --report-trx --report-trx-filename test-results.trx \
-               --filter-not-trait Category=regen
+               --filter-not-trait Category=regen \
+               --filter-not-trait Category=E2E
 
       - name: Upload test results
         uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7

.github/workflows/e2e-docker.yml

@@ -0,0 +1,131 @@
+# =============================================================================
+# Docker End-to-End Validation
+# -----------------------------------------------------------------------------
+# Builds the two qyl images the E2E suite actually exercises (collector + mcp)
+# with GHA-backed BuildKit layer caching, then runs Category=E2E tests against
+# the local Docker daemon. Nuke's DockerImageBuild target is skipped because we
+# already produced the tags it would have produced.
+# =============================================================================
+
+name: E2E (Docker)
+
+on:
+  push:
+    branches: [ main ]
+    paths:
+      - ".github/workflows/e2e-docker.yml"
+      - "Directory.Build.props"
+      - "Directory.Build.targets"
+      - "Directory.Packages.props"
+      - "core/specs/**"
+      - "eng/**"
+      - "global.json"
+      - "internal/**"
+      - "packages/**"
+      - "qyl.slnx"
+      - "services/qyl.collector/**"
+      - "services/qyl.dashboard/**"
+      - "services/qyl.loom/**"
+      - "services/qyl.loom.patterns/**"
+      - "services/qyl.mcp/**"
+      - "tests/qyl.e2e.tests/**"
+  pull_request:
+    branches: [ main ]
+    paths:
+      - ".github/workflows/e2e-docker.yml"
+      - "Directory.Build.props"
+      - "Directory.Build.targets"
+      - "Directory.Packages.props"
+      - "core/specs/**"
+      - "eng/**"
+      - "global.json"
+      - "internal/**"
+      - "packages/**"
+      - "qyl.slnx"
+      - "services/qyl.collector/**"
+      - "services/qyl.dashboard/**"
+      - "services/qyl.loom/**"
+      - "services/qyl.loom.patterns/**"
+      - "services/qyl.mcp/**"
+      - "tests/qyl.e2e.tests/**"
+  workflow_dispatch:
+  schedule:
+    - cron: "17 3 * * 0"
+
+permissions:
+  # actions: write is required for type=gha BuildKit cache export/import;
+  # without it, docker/build-push-action silently falls back to a no-op cache.
+  actions: write
+  contents: read
+  packages: read
+
+concurrency:
+  group: e2e-docker-${{ github.ref }}
+  cancel-in-progress: false
+
+env:
+  DOTNET_NOLOGO: true
+  DOTNET_CLI_TELEMETRY_OPTOUT: true
+  NODE_VERSION: "22"
+
+jobs:
+  docker-e2e:
+    name: Docker topology
+    runs-on: ubuntu-latest
+    timeout-minutes: 60
+
+    steps:
+      - name: Checkout
+        uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
+        with:
+          submodules: true
+
+      - name: Setup .NET
+        uses: actions/setup-dotnet@c2fa09f4bde5ebb9d1777cf28262a3eb3db3ced7 # v5
+        with:
+          global-json-file: global.json
+          cache: true
+          cache-dependency-path: |
+            **/Directory.Packages.props
+            **/*.csproj
+            **/*.fsproj
+            **/packages.lock.json
+            **/nuget.config
+
+      - name: Setup Node.js
+        uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6
+        with:
+          node-version: ${{ env.NODE_VERSION }}
+          cache: "npm"
+          cache-dependency-path: core/specs/package-lock.json
+          registry-url: 'https://npm.pkg.github.com'
+          scope: '@o-ancpplua'
+
+      - name: Set up Docker Buildx
+        uses: docker/setup-buildx-action@d70bba72b1f3fd22344832f00baa16ece964efeb # v3.3.0
+
+      - name: Build qyl-collector image (cached)
+        uses: docker/build-push-action@5176d81f87c23d6fc96624dfdbcd9f3830bbe445 # v6.5.0
+        with:
+          context: .
+          file: services/qyl.collector/Dockerfile
+          tags: qyl-collector:latest
+          load: true
+          cache-from: type=gha,scope=qyl-collector
+          cache-to: type=gha,mode=max,scope=qyl-collector
+
+      - name: Build qyl-mcp image (cached)
+        uses: docker/build-push-action@5176d81f87c23d6fc96624dfdbcd9f3830bbe445 # v6.5.0
+        with:
+          context: .
+          file: services/qyl.mcp/Dockerfile
+          tags: qyl-mcp:latest
+          load: true
+          cache-from: type=gha,scope=qyl-mcp
+          cache-to: type=gha,mode=max,scope=qyl-mcp
+
+      - name: Run Docker topology E2E
+        env:
+          NODE_AUTH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+        run: ./eng/build.sh E2ETests --configuration Release --skip DockerImageBuild
+

eng/build/BuildInfra.cs

@@ -43,12 +43,11 @@ interface IDocker : IHazSourcePaths
 
             DockerTasks.DockerBuild(s => s
                     .SetPath<DockerBuildSettings>(RootDirectory)
-                    .EnablePull()
                     .SetProcessEnvironmentVariable("DOCKER_BUILDKIT", "1")
                     .CombineWith(ImageSpecs, static (settings, img) => settings
                         .SetFile(img.Dockerfile)
                         .SetTag(img.Tag)),
-                degreeOfParallelism: 2);
+                degreeOfParallelism: 4);
 
             foreach (var (_, _, tag) in ImageSpecs)
                 Log.Information("Built: {Tag}", tag);

eng/build/BuildTest.cs

@@ -224,6 +224,7 @@ sealed void RunFilteredTests(string namespaceFilter, string trxSuffix, bool need
         if (needsTestcontainers) EnsureTestcontainersConfigured();
 
         DotNetTasks.DotNetTest(s => s
+            .SetConfiguration(Configuration)
             .SetNoBuild(true)
             .SetNoRestore(true)
             .SetResultsDirectory(TestResultsDirectory)
@@ -259,6 +260,7 @@ sealed void RunFilteredE2ETests()
         }
 
         DotNetTasks.DotNetTest(s => s
+            .SetConfiguration(Configuration)
             .SetNoBuild(true)
             .SetNoRestore(true)
             .SetResultsDirectory(TestResultsDirectory)

tests/qyl.collector.tests/Functional/McpMetricsEndpointsTests.cs

@@ -146,34 +146,19 @@ await SeedSpanAsync(
         point.GetProperty("value").GetDouble().Should().Be(30);
     }
 
-    [Fact]
-    public async Task Get_mcp_metric_query_rejects_token_type_for_non_genai_token_metric()
+    [Theory]
+    [InlineData("/api/v1/mcp/metrics/request_count/query?tokenType=input", "tokenType")]
+    [InlineData("/api/v1/mcp/metrics/request_count/query?filter=project%3Ddemo", "service.name")]
+    public async Task Get_mcp_metric_query_rejects_invalid_request(string url, string expectedFragment)
     {
         var ct = TestContext.Current.CancellationToken;
         using var client = _factory.CreateClient();
 
-        using var response = await client.GetAsync(
-            "/api/v1/mcp/metrics/request_count/query?tokenType=input",
-            ct);
-
-        response.StatusCode.Should().Be(HttpStatusCode.BadRequest);
-        var body = await response.Content.ReadFromJsonAsync<JsonElement>(ct);
-        body.GetProperty("error").GetString().Should().Contain("tokenType");
-    }
-
-    [Fact]
-    public async Task Get_mcp_metric_query_rejects_unsupported_filter_shape()
-    {
-        var ct = TestContext.Current.CancellationToken;
-        using var client = _factory.CreateClient();
-
-        using var response = await client.GetAsync(
-            "/api/v1/mcp/metrics/request_count/query?filter=project%3Ddemo",
-            ct);
+        using var response = await client.GetAsync(url, ct);
 
         response.StatusCode.Should().Be(HttpStatusCode.BadRequest);
         var body = await response.Content.ReadFromJsonAsync<JsonElement>(ct);
-        body.GetProperty("error").GetString().Should().Contain("service.name");
+        body.GetProperty("error").GetString().Should().Contain(expectedFragment);
     }
 
     private async Task SeedSpanAsync(