diff --git a/.agents/routine-last-run.md b/.agents/routine-last-run.md index 8d740933a..70f76799a 100644 --- a/.agents/routine-last-run.md +++ b/.agents/routine-last-run.md @@ -4,6 +4,91 @@ Lightweight breadcrumb for scheduled agent routines. Each entry under `## YYYY-MM-DD HH:MM` records the state the routine exited in, so the next run can resume from the same line of work without re-discovering it. +## qyl-e2e-tests 2026-05-24 09:46 + +**Outcome:** BLOCKED — Docker daemon still not running. Third consecutive +scheduled run of `qyl-e2e-tests` to hit this exact wall (2026-05-19, +2026-05-20, 2026-05-24; there was no scheduled e2e run on 05-21/22/23 — the +gap between 05-20 and 05-24 is the cron cadence, not me skipping). No code +touched. + +**Blocker (identical to 2026-05-19 / 2026-05-20):** OrbStack's Docker socket +is still missing — `docker info` fails with `dial unix +/Users/ancplua/.orbstack/run/docker.sock: connect: no such file or directory`. +`/Applications/OrbStack.app` is installed but the daemon is not started. +Bringing the desktop daemon up is the user's call; this routine does not +launch GUI apps autonomously. + +**State on arrival:** +- Worktree clean on `claude/clever-feynman-0c4209` (auto-generated isolation + branch under `.claude/worktrees/`). +- `dotnet build` not attempted — a build cannot unblock a missing Docker + daemon and the existing E2E tests would build green anyway (commit + `0b08beba fix(tests/e2e): heal release-mode bit-rot…` landed since 05-20). +- E2E project on main currently ships **three** scenarios — one more than at + the 2026-05-20 entry. New scenario landed via a non-routine commit: + - `tests/qyl.e2e.tests/Bootstrap/WireMockLlmSeamTests.cs` + - `tests/qyl.e2e.tests/Scenarios/OtlpHttpTraceIngestionRoundtripTests.cs` + - `tests/qyl.e2e.tests/Scenarios/McpServerExposesCatalogTests.cs` ← **new**, + from `83503c39 test(e2e/mcp): cover qyl-mcp's /llms.txt agent-discovery + surface`. Closes carry-forward gap #2 from the 2026-05-20 entry (MCP + catalog/discovery surface). +- CI investment also moved while this routine was blocked: `80c5b917 ci: gate + docker e2e on relevant changes`, `5aead24b ci: keep docker e2e out of + backend gate`, `3a1ea5a6 ci(e2e-docker): cache layers…`, + `d56ea7e2 ci(e2e-docker): no-op nudge to benchmark warm GHA cache`. The + E2E pipeline is being actively tuned by other PRs even though the + workstation-side routine has been no-op for a week. + +**Carry-forward gaps (revised — gap #2 was closed externally):** +1. **Priority-1 production bug — still present, verified today.** + `services/qyl.collector/Storage/DuckDbSchema.g.sql:313,317` declares + `kind VARCHAR NOT NULL` and `status_code VARCHAR NOT NULL`, but + `internal/qyl.collector.storage.generators/DuckDbEmitter.cs:184,221` + still emits `reader.Col(N).AsByte` / `reader.Col(N).GetByte(0)` for + those columns. Every `GET /api/v1/traces` row read throws + `InvalidCastException`. Same line numbers as the 2026-05-20 entry — + the `527f9294 chore: emit DuckDbSchema.g.sql…` commit that touched + the schema did not realign the generator. Not an E2E fix; needs its + own focused PR with migration testing. Once landed, extend + `OtlpHttpTraceIngestionRoundtripTests` to also assert + `GET /api/v1/traces/{traceId}` returns the row. +2. **MCP → collector read-through scenario.** Drive an MCP tool over + JSON-RPC that reads spans **previously ingested via OTLP/HTTP** (not + the catalog/discovery surface that `McpServerExposesCatalogTests` now + covers — that's a different seam). The round-trip assertion is the + one with real production value, and it's gated on gap #1 above. +3. **Chat ingest → trace at sink with credential redaction.** Requires + adding an OTel collector sink container with a file exporter to + `QylTopologyFixture` (or migrating the fixture to TUnit.Aspire, which + is the SKILL.md-prescribed direction but conflicts with the repo's + xUnit-v3 reality — same TUnit-vs-xUnit conflict that + `qyl-unit-tests` 2026-05-22 resolved by following CLAUDE.md). Keep + xUnit; add the sink container; assert redaction at the sink. + +**Pattern flag — escalated:** + +The 2026-05-20 entry warned: "If this becomes three [consecutive Docker-down +no-ops], worth considering whether the routine should attempt a non- +interactive `open -gja OrbStack`." We're now at three. **I am still not +auto-launching OrbStack** — the reasoning from 2026-05-20 stands (agents +shouldn't auto-launch GUI apps without explicit instruction). But the +pattern is now a real signal worth raising: + +- **User-side fix that would actually solve this:** add OrbStack to macOS + Login Items (System Settings → General → Login Items → Open at Login). + After that the daemon is up whenever the workstation is, and this routine + stops being a perpetual no-op without anyone touching the schedule. +- **Routine-side change worth considering:** the cron entry could check + `docker info` from a pre-task script and **skip the run entirely** (not + even invoke the agent) when Docker is down, instead of paying the + agent-spawn cost only to produce a no-op log entry like this one. That + removes the per-day noise but loses the cross-channel observation value + this entry just demonstrated (gap #2 closing was worth noticing). Net: + leave the schedule alone; the cost of a no-op log entry is small. + +**Handoff:** none. PR for this log entry only. + ## qyl-functional-tests 2026-05-22 (auto) **Outcome:** Added end-to-end functional coverage for `/api/v1/configurator/*` diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 1c9f14cba..c60613269 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -88,14 +88,15 @@ jobs: -p:WarningsAsErrors= - name: Test - # `--filter-not-trait Category=regen` excludes opt-in heavy tests - # (e.g. RegenCleanTests, which shells out to Weaver). The dedicated - # `regen-clean` job covers that gate end-to-end. + # `Category=regen` and `Category=E2E` are opt-in heavy suites. + # The dedicated `regen-clean` job covers regeneration drift, and + # E2E (Docker) gates relevant PR/push changes with freshly built local images. run: | dotnet test --configuration Release --no-build \ --results-directory ./TestResults \ -- --report-trx --report-trx-filename test-results.trx \ - --filter-not-trait Category=regen + --filter-not-trait Category=regen \ + --filter-not-trait Category=E2E - name: Upload test results uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7 diff --git a/.github/workflows/e2e-docker.yml b/.github/workflows/e2e-docker.yml new file mode 100644 index 000000000..d8256ce3d --- /dev/null +++ b/.github/workflows/e2e-docker.yml @@ -0,0 +1,131 @@ +# ============================================================================= +# Docker End-to-End Validation +# ----------------------------------------------------------------------------- +# Builds the two qyl images the E2E suite actually exercises (collector + mcp) +# with GHA-backed BuildKit layer caching, then runs Category=E2E tests against +# the local Docker daemon. Nuke's DockerImageBuild target is skipped because we +# already produced the tags it would have produced. +# ============================================================================= + +name: E2E (Docker) + +on: + push: + branches: [ main ] + paths: + - ".github/workflows/e2e-docker.yml" + - "Directory.Build.props" + - "Directory.Build.targets" + - "Directory.Packages.props" + - "core/specs/**" + - "eng/**" + - "global.json" + - "internal/**" + - "packages/**" + - "qyl.slnx" + - "services/qyl.collector/**" + - "services/qyl.dashboard/**" + - "services/qyl.loom/**" + - "services/qyl.loom.patterns/**" + - "services/qyl.mcp/**" + - "tests/qyl.e2e.tests/**" + pull_request: + branches: [ main ] + paths: + - ".github/workflows/e2e-docker.yml" + - "Directory.Build.props" + - "Directory.Build.targets" + - "Directory.Packages.props" + - "core/specs/**" + - "eng/**" + - "global.json" + - "internal/**" + - "packages/**" + - "qyl.slnx" + - "services/qyl.collector/**" + - "services/qyl.dashboard/**" + - "services/qyl.loom/**" + - "services/qyl.loom.patterns/**" + - "services/qyl.mcp/**" + - "tests/qyl.e2e.tests/**" + workflow_dispatch: + schedule: + - cron: "17 3 * * 0" + +permissions: + # actions: write is required for type=gha BuildKit cache export/import; + # without it, docker/build-push-action silently falls back to a no-op cache. + actions: write + contents: read + packages: read + +concurrency: + group: e2e-docker-${{ github.ref }} + cancel-in-progress: false + +env: + DOTNET_NOLOGO: true + DOTNET_CLI_TELEMETRY_OPTOUT: true + NODE_VERSION: "22" + +jobs: + docker-e2e: + name: Docker topology + runs-on: ubuntu-latest + timeout-minutes: 60 + + steps: + - name: Checkout + uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6 + with: + submodules: true + + - name: Setup .NET + uses: actions/setup-dotnet@c2fa09f4bde5ebb9d1777cf28262a3eb3db3ced7 # v5 + with: + global-json-file: global.json + cache: true + cache-dependency-path: | + **/Directory.Packages.props + **/*.csproj + **/*.fsproj + **/packages.lock.json + **/nuget.config + + - name: Setup Node.js + uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6 + with: + node-version: ${{ env.NODE_VERSION }} + cache: "npm" + cache-dependency-path: core/specs/package-lock.json + registry-url: 'https://npm.pkg.github.com' + scope: '@o-ancpplua' + + - name: Set up Docker Buildx + uses: docker/setup-buildx-action@d70bba72b1f3fd22344832f00baa16ece964efeb # v3.3.0 + + - name: Build qyl-collector image (cached) + uses: docker/build-push-action@5176d81f87c23d6fc96624dfdbcd9f3830bbe445 # v6.5.0 + with: + context: . + file: services/qyl.collector/Dockerfile + tags: qyl-collector:latest + load: true + cache-from: type=gha,scope=qyl-collector + cache-to: type=gha,mode=max,scope=qyl-collector + + - name: Build qyl-mcp image (cached) + uses: docker/build-push-action@5176d81f87c23d6fc96624dfdbcd9f3830bbe445 # v6.5.0 + with: + context: . + file: services/qyl.mcp/Dockerfile + tags: qyl-mcp:latest + load: true + cache-from: type=gha,scope=qyl-mcp + cache-to: type=gha,mode=max,scope=qyl-mcp + + - name: Run Docker topology E2E + env: + NODE_AUTH_TOKEN: ${{ secrets.GITHUB_TOKEN }} + run: ./eng/build.sh E2ETests --configuration Release --skip DockerImageBuild + diff --git a/eng/build/BuildInfra.cs b/eng/build/BuildInfra.cs index 8759d77d6..20e89e256 100644 --- a/eng/build/BuildInfra.cs +++ b/eng/build/BuildInfra.cs @@ -43,12 +43,11 @@ interface IDocker : IHazSourcePaths DockerTasks.DockerBuild(s => s .SetPath(RootDirectory) - .EnablePull() .SetProcessEnvironmentVariable("DOCKER_BUILDKIT", "1") .CombineWith(ImageSpecs, static (settings, img) => settings .SetFile(img.Dockerfile) .SetTag(img.Tag)), - degreeOfParallelism: 2); + degreeOfParallelism: 4); foreach (var (_, _, tag) in ImageSpecs) Log.Information("Built: {Tag}", tag); diff --git a/eng/build/BuildTest.cs b/eng/build/BuildTest.cs index d7fb22988..5b103c5fb 100644 --- a/eng/build/BuildTest.cs +++ b/eng/build/BuildTest.cs @@ -224,6 +224,7 @@ sealed void RunFilteredTests(string namespaceFilter, string trxSuffix, bool need if (needsTestcontainers) EnsureTestcontainersConfigured(); DotNetTasks.DotNetTest(s => s + .SetConfiguration(Configuration) .SetNoBuild(true) .SetNoRestore(true) .SetResultsDirectory(TestResultsDirectory) @@ -259,6 +260,7 @@ sealed void RunFilteredE2ETests() } DotNetTasks.DotNetTest(s => s + .SetConfiguration(Configuration) .SetNoBuild(true) .SetNoRestore(true) .SetResultsDirectory(TestResultsDirectory) diff --git a/tests/qyl.collector.tests/Functional/McpMetricsEndpointsTests.cs b/tests/qyl.collector.tests/Functional/McpMetricsEndpointsTests.cs index 83f1f828d..d6e6760ab 100644 --- a/tests/qyl.collector.tests/Functional/McpMetricsEndpointsTests.cs +++ b/tests/qyl.collector.tests/Functional/McpMetricsEndpointsTests.cs @@ -146,34 +146,19 @@ await SeedSpanAsync( point.GetProperty("value").GetDouble().Should().Be(30); } - [Fact] - public async Task Get_mcp_metric_query_rejects_token_type_for_non_genai_token_metric() + [Theory] + [InlineData("/api/v1/mcp/metrics/request_count/query?tokenType=input", "tokenType")] + [InlineData("/api/v1/mcp/metrics/request_count/query?filter=project%3Ddemo", "service.name")] + public async Task Get_mcp_metric_query_rejects_invalid_request(string url, string expectedFragment) { var ct = TestContext.Current.CancellationToken; using var client = _factory.CreateClient(); - using var response = await client.GetAsync( - "/api/v1/mcp/metrics/request_count/query?tokenType=input", - ct); - - response.StatusCode.Should().Be(HttpStatusCode.BadRequest); - var body = await response.Content.ReadFromJsonAsync(ct); - body.GetProperty("error").GetString().Should().Contain("tokenType"); - } - - [Fact] - public async Task Get_mcp_metric_query_rejects_unsupported_filter_shape() - { - var ct = TestContext.Current.CancellationToken; - using var client = _factory.CreateClient(); - - using var response = await client.GetAsync( - "/api/v1/mcp/metrics/request_count/query?filter=project%3Ddemo", - ct); + using var response = await client.GetAsync(url, ct); response.StatusCode.Should().Be(HttpStatusCode.BadRequest); var body = await response.Content.ReadFromJsonAsync(ct); - body.GetProperty("error").GetString().Should().Contain("service.name"); + body.GetProperty("error").GetString().Should().Contain(expectedFragment); } private async Task SeedSpanAsync( diff --git a/tests/qyl.collector.tests/Functional/MetricsEndpointsTests.cs b/tests/qyl.collector.tests/Functional/MetricsEndpointsTests.cs index 11b34007d..c51602071 100644 --- a/tests/qyl.collector.tests/Functional/MetricsEndpointsTests.cs +++ b/tests/qyl.collector.tests/Functional/MetricsEndpointsTests.cs @@ -637,201 +637,74 @@ await SeedSpanAsync( .GetProperty("value").GetDouble().Should().Be(35); } - [Fact] - public async Task Post_metrics_query_rejects_missing_explicit_window() - { - var ct = TestContext.Current.CancellationToken; - using var client = _factory.CreateClient(); - - using var response = await client.PostAsync( - "/api/v1/metrics/query", - JsonContent(""" - { - "metric_name": "request_count" - } - """), - ct); - - response.StatusCode.Should().Be(HttpStatusCode.BadRequest); - var body = await response.Content.ReadFromJsonAsync(ct); - body.GetProperty("error").GetString().Should().Contain("start_time"); - } - - [Fact] - public async Task Post_metrics_query_rejects_duplicate_service_filter_aliases() - { - var ct = TestContext.Current.CancellationToken; - using var client = _factory.CreateClient(); - - using var response = await client.PostAsync( - "/api/v1/metrics/query", - JsonContent(""" - { - "metric_name": "request_count", - "filters": { - "service.name": "orders-api", - "service": "checkout-api" - }, - "start_time": "2026-05-23T09:59:00Z", - "end_time": "2026-05-23T11:00:00Z" - } - """), - ct); - - response.StatusCode.Should().Be(HttpStatusCode.BadRequest); - var body = await response.Content.ReadFromJsonAsync(ct); - body.GetProperty("error").GetString().Should().Contain("service.name"); - body.GetProperty("error").GetString().Should().Contain("more than once"); - } - - [Fact] - public async Task Post_metrics_query_rejects_token_type_filter_for_non_token_metric() + public static TheoryData PostQueryRejectionCases() => new() { - var ct = TestContext.Current.CancellationToken; - using var client = _factory.CreateClient(); - - using var response = await client.PostAsync( - "/api/v1/metrics/query", - JsonContent(""" - { - "metric_name": "request_count", - "filters": { - "gen_ai.token.type": "input" - }, - "start_time": "2026-05-23T09:59:00Z", - "end_time": "2026-05-23T11:00:00Z" - } - """), - ct); - - response.StatusCode.Should().Be(HttpStatusCode.BadRequest); - var body = await response.Content.ReadFromJsonAsync(ct); - body.GetProperty("error").GetString().Should().Contain("gen_ai.token.type"); - body.GetProperty("error").GetString().Should().Contain("gen_ai.client.token.usage"); - } - - [Fact] - public async Task Post_metrics_query_rejects_unknown_token_type_filter_value() - { - var ct = TestContext.Current.CancellationToken; - using var client = _factory.CreateClient(); - - using var response = await client.PostAsync( - "/api/v1/metrics/query", - JsonContent(""" - { - "metric_name": "gen_ai.client.token.usage", - "filters": { - "gen_ai.token.type": "total" - }, - "start_time": "2026-05-23T09:59:00Z", - "end_time": "2026-05-23T11:00:00Z" - } - """), - ct); - - response.StatusCode.Should().Be(HttpStatusCode.BadRequest); - var body = await response.Content.ReadFromJsonAsync(ct); - body.GetProperty("error").GetString().Should().Contain("input"); - body.GetProperty("error").GetString().Should().Contain("output"); - } - - [Fact] - public async Task Post_metrics_query_rejects_unsupported_grouping() - { - var ct = TestContext.Current.CancellationToken; - using var client = _factory.CreateClient(); - - using var response = await client.PostAsync( - "/api/v1/metrics/query", - JsonContent(""" - { - "metric_name": "request_count", - "start_time": "2026-05-23T09:59:00Z", - "end_time": "2026-05-23T11:00:00Z", - "group_by": [ "host.name" ] - } - """), - ct); - - response.StatusCode.Should().Be(HttpStatusCode.BadRequest); - var body = await response.Content.ReadFromJsonAsync(ct); - body.GetProperty("error").GetString().Should().Contain("service.name"); - } - - [Fact] - public async Task Post_metrics_query_rejects_empty_grouping_label() - { - var ct = TestContext.Current.CancellationToken; - using var client = _factory.CreateClient(); - - using var response = await client.PostAsync( - "/api/v1/metrics/query", - JsonContent(""" - { - "metric_name": "request_count", - "start_time": "2026-05-23T09:59:00Z", - "end_time": "2026-05-23T11:00:00Z", - "group_by": [ null ] - } - """), - ct); - - response.StatusCode.Should().Be(HttpStatusCode.BadRequest); - var body = await response.Content.ReadFromJsonAsync(ct); - body.GetProperty("error").GetString().Should().Contain("non-empty"); - } - - [Theory] - [InlineData(0)] - [InlineData(1001)] - public async Task Post_metrics_query_rejects_series_limit_outside_contract_bounds(int seriesLimit) - { - var ct = TestContext.Current.CancellationToken; - using var client = _factory.CreateClient(); - - using var response = await client.PostAsync( - "/api/v1/metrics/query", - JsonContent($$""" - { - "metric_name": "request_count", - "start_time": "2026-05-23T09:59:00Z", - "end_time": "2026-05-23T11:00:00Z", - "series_limit": {{seriesLimit}} - } - """), - ct); - - response.StatusCode.Should().Be(HttpStatusCode.BadRequest); - var body = await response.Content.ReadFromJsonAsync(ct); - body.GetProperty("error").GetString().Should().Contain("series_limit"); - body.GetProperty("error").GetString().Should().Contain("1000"); - } + // missing window + { + """{ "metric_name": "request_count" }""", + ["start_time"] + }, + // duplicate service alias + { + """{ "metric_name": "request_count", "filters": { "service.name": "orders-api", "service": "checkout-api" }, "start_time": "2026-05-23T09:59:00Z", "end_time": "2026-05-23T11:00:00Z" }""", + ["service.name", "more than once"] + }, + // token_type filter on non-token metric + { + """{ "metric_name": "request_count", "filters": { "gen_ai.token.type": "input" }, "start_time": "2026-05-23T09:59:00Z", "end_time": "2026-05-23T11:00:00Z" }""", + ["gen_ai.token.type", "gen_ai.client.token.usage"] + }, + // unknown token_type value + { + """{ "metric_name": "gen_ai.client.token.usage", "filters": { "gen_ai.token.type": "total" }, "start_time": "2026-05-23T09:59:00Z", "end_time": "2026-05-23T11:00:00Z" }""", + ["input", "output"] + }, + // unsupported grouping + { + """{ "metric_name": "request_count", "start_time": "2026-05-23T09:59:00Z", "end_time": "2026-05-23T11:00:00Z", "group_by": [ "host.name" ] }""", + ["service.name"] + }, + // empty grouping label + { + """{ "metric_name": "request_count", "start_time": "2026-05-23T09:59:00Z", "end_time": "2026-05-23T11:00:00Z", "group_by": [ null ] }""", + ["non-empty"] + }, + // series_limit below contract bound + { + """{ "metric_name": "request_count", "start_time": "2026-05-23T09:59:00Z", "end_time": "2026-05-23T11:00:00Z", "series_limit": 0 }""", + ["series_limit", "1000"] + }, + // series_limit above contract bound + { + """{ "metric_name": "request_count", "start_time": "2026-05-23T09:59:00Z", "end_time": "2026-05-23T11:00:00Z", "series_limit": 1001 }""", + ["series_limit", "1000"] + }, + // point_limit below contract bound + { + """{ "metric_name": "request_count", "start_time": "2026-05-23T09:59:00Z", "end_time": "2026-05-23T11:00:00Z", "point_limit": 0 }""", + ["point_limit", "100000"] + }, + // point_limit above contract bound + { + """{ "metric_name": "request_count", "start_time": "2026-05-23T09:59:00Z", "end_time": "2026-05-23T11:00:00Z", "point_limit": 100001 }""", + ["point_limit", "100000"] + }, + }; [Theory] - [InlineData(0)] - [InlineData(100001)] - public async Task Post_metrics_query_rejects_point_limit_outside_contract_bounds(int pointLimit) + [MemberData(nameof(PostQueryRejectionCases))] + public async Task Post_metrics_query_rejects_invalid_request(string body, string[] expectedFragments) { var ct = TestContext.Current.CancellationToken; using var client = _factory.CreateClient(); - using var response = await client.PostAsync( - "/api/v1/metrics/query", - JsonContent($$""" - { - "metric_name": "request_count", - "start_time": "2026-05-23T09:59:00Z", - "end_time": "2026-05-23T11:00:00Z", - "point_limit": {{pointLimit}} - } - """), - ct); + using var response = await client.PostAsync("/api/v1/metrics/query", JsonContent(body), ct); response.StatusCode.Should().Be(HttpStatusCode.BadRequest); - var body = await response.Content.ReadFromJsonAsync(ct); - body.GetProperty("error").GetString().Should().Contain("point_limit"); - body.GetProperty("error").GetString().Should().Contain("100000"); + var error = (await response.Content.ReadFromJsonAsync(ct)) + .GetProperty("error").GetString(); + foreach (var fragment in expectedFragments) + error.Should().Contain(fragment); } [Fact] diff --git a/tests/qyl.collector.tests/Functional/ObserveSubscriptionEndpointsTests.cs b/tests/qyl.collector.tests/Functional/ObserveSubscriptionEndpointsTests.cs index 801bc9cda..83a8b6e8f 100644 --- a/tests/qyl.collector.tests/Functional/ObserveSubscriptionEndpointsTests.cs +++ b/tests/qyl.collector.tests/Functional/ObserveSubscriptionEndpointsTests.cs @@ -84,55 +84,20 @@ public async Task Get_catalog_lists_genai_token_metric_with_ucum_unit() metric.GetProperty("unit").GetString() == "{token}"); } - [Fact] - public async Task Post_subscription_with_missing_filter_returns_400() + [Theory] + [InlineData("", "http://localhost:4318/v1/traces", "filter")] + [InlineData("qyl.collector", "", "endpoint")] + [InlineData("qyl.collector", "not-a-real-uri", "absolute")] + public async Task Post_subscription_with_invalid_payload_returns_400(string filter, string endpoint, string expectedFragment) { var ct = TestContext.Current.CancellationToken; using var client = _factory.CreateClient(); - using var response = await client.PostAsJsonAsync(SubscriptionsPath, new - { - filter = "", - endpoint = "http://localhost:4318/v1/traces" - }, ct); + using var response = await client.PostAsJsonAsync(SubscriptionsPath, new { filter, endpoint }, ct); response.StatusCode.Should().Be(HttpStatusCode.BadRequest); var body = await response.Content.ReadFromJsonAsync(ct); - body.GetProperty("error").GetString().Should().Contain("filter"); - } - - [Fact] - public async Task Post_subscription_with_missing_endpoint_returns_400() - { - var ct = TestContext.Current.CancellationToken; - using var client = _factory.CreateClient(); - - using var response = await client.PostAsJsonAsync(SubscriptionsPath, new - { - filter = "qyl.collector", - endpoint = MissingEndpoint() - }, ct); - - response.StatusCode.Should().Be(HttpStatusCode.BadRequest); - var body = await response.Content.ReadFromJsonAsync(ct); - body.GetProperty("error").GetString().Should().Contain("endpoint"); - } - - [Fact] - public async Task Post_subscription_with_non_absolute_endpoint_returns_400() - { - var ct = TestContext.Current.CancellationToken; - using var client = _factory.CreateClient(); - - using var response = await client.PostAsJsonAsync(SubscriptionsPath, new - { - filter = "qyl.collector", - endpoint = NonAbsoluteEndpoint() - }, ct); - - response.StatusCode.Should().Be(HttpStatusCode.BadRequest); - var body = await response.Content.ReadFromJsonAsync(ct); - body.GetProperty("error").GetString().Should().Contain("absolute"); + body.GetProperty("error").GetString().Should().Contain(expectedFragment); } [Fact] @@ -245,7 +210,4 @@ public sealed class CollectorFactory() : CollectorFunctionalFactory("observe") { } - private static string MissingEndpoint() => string.Empty; - - private static string NonAbsoluteEndpoint() => string.Join('-', "not", "a", "real", "uri"); } diff --git a/tests/qyl.collector.tests/Functional/SchemaPromotionEndpointsTests.cs b/tests/qyl.collector.tests/Functional/SchemaPromotionEndpointsTests.cs index 9786c406a..0b95be265 100644 --- a/tests/qyl.collector.tests/Functional/SchemaPromotionEndpointsTests.cs +++ b/tests/qyl.collector.tests/Functional/SchemaPromotionEndpointsTests.cs @@ -15,42 +15,25 @@ public sealed class SchemaPromotionEndpointsTests public SchemaPromotionEndpointsTests(CollectorFactory factory) => _factory = factory; - [Fact] - public async Task Post_promotion_with_missing_target_table_returns_400() + [Theory] + [InlineData("add_column", "", "TargetTable")] + [InlineData("", "qyl_test_table", "ChangeType")] + public async Task Post_promotion_with_invalid_payload_returns_400(string changeType, string targetTable, string expectedFragment) { var ct = TestContext.Current.CancellationToken; using var client = _factory.CreateClient(); using var response = await client.PostAsJsonAsync(PromotionsPath, new { - changeType = "add_column", - targetTable = "", - targetColumn = "extra", - columnType = "VARCHAR" - }, ct); - - response.StatusCode.Should().Be(HttpStatusCode.BadRequest); - var body = await response.Content.ReadFromJsonAsync(ct); - body.GetProperty("error").GetString().Should().Contain("TargetTable"); - } - - [Fact] - public async Task Post_promotion_with_missing_change_type_returns_400() - { - var ct = TestContext.Current.CancellationToken; - using var client = _factory.CreateClient(); - - using var response = await client.PostAsJsonAsync(PromotionsPath, new - { - changeType = "", - targetTable = "qyl_test_table", + changeType, + targetTable, targetColumn = "extra", columnType = "VARCHAR" }, ct); response.StatusCode.Should().Be(HttpStatusCode.BadRequest); var body = await response.Content.ReadFromJsonAsync(ct); - body.GetProperty("error").GetString().Should().Contain("ChangeType"); + body.GetProperty("error").GetString().Should().Contain(expectedFragment); } [Fact] diff --git a/tests/qyl.collector.tests/Ingestion/OtlpConstantsTests.cs b/tests/qyl.collector.tests/Ingestion/OtlpConstantsTests.cs index 7f59d41f1..633283470 100644 --- a/tests/qyl.collector.tests/Ingestion/OtlpConstantsTests.cs +++ b/tests/qyl.collector.tests/Ingestion/OtlpConstantsTests.cs @@ -6,33 +6,20 @@ namespace Qyl.Collector.Tests.Ingestion; public sealed class OtlpConstantsTests { [Theory] - [InlineData("/v1/traces")] - [InlineData("/v1/logs")] - [InlineData("/v1/profiles")] - public void IsOtlpPath_ReturnsTrue_ForMappedOtlpEndpoints(string path) - { - OtlpConstants.IsOtlpPath(path).Should().BeTrue(); - } + [InlineData("/v1/traces", true)] + [InlineData("/v1/logs", true)] + [InlineData("/v1/profiles", true)] + [InlineData("/v1/metrics", false)] + [InlineData("/healthz", false)] + [InlineData("", false)] + public void IsOtlpPath_RecognisesMappedOtlpEndpoints(string path, bool expected) => + OtlpConstants.IsOtlpPath(path).Should().Be(expected); - [Fact] - public void IsOtlpPath_ReturnsFalse_ForUnmappedMetricsEndpoint() - { - OtlpConstants.IsOtlpPath("/v1/metrics").Should().BeFalse(); - } - - [Fact] - public void TokenAuthDefaults_DoNotBypassUnmappedMetricsEndpoint() - { - var options = new TokenAuthOptions(); - - options.ExcludedPaths.Should().NotContain("/v1/metrics"); - } - - [Fact] - public void TokenAuthDefaults_BypassMappedProfilesEndpoint() - { - var options = new TokenAuthOptions(); - - options.ExcludedPaths.Should().Contain("/v1/profiles"); - } + [Theory] + [InlineData("/v1/traces", true)] + [InlineData("/v1/logs", true)] + [InlineData("/v1/profiles", true)] + [InlineData("/v1/metrics", false)] + public void TokenAuthDefaults_BypassMatchesMappedOtlpPaths(string path, bool isBypassed) => + new TokenAuthOptions().ExcludedPaths.Contains(path).Should().Be(isBypassed); } diff --git a/tests/qyl.collector.tests/Instrumentation/ChatClientToolInstrumentationTests.cs b/tests/qyl.collector.tests/Instrumentation/ChatClientToolInstrumentationTests.cs deleted file mode 100644 index e8f7150cd..000000000 --- a/tests/qyl.collector.tests/Instrumentation/ChatClientToolInstrumentationTests.cs +++ /dev/null @@ -1,19 +0,0 @@ -using ANcpLua.Agents.Instrumentation; -using ANcpLua.Agents.Testing.ChatClients; -using Microsoft.Extensions.AI; -using Qyl.Instrumentation.Instrumentation.GenAi; - -namespace Qyl.Collector.Tests.Instrumentation; - -public sealed class ChatClientToolInstrumentationTests -{ - [Fact] - public void WithQylTelemetry_wraps_plain_client_with_tool_decorator() - { - var inner = new FakeChatClient { Metadata = new ChatClientMetadata("test-provider", null, "test-model") }; - - var result = inner.WithQylTelemetry(); - - result.Should().NotBeSameAs(inner); - } -} diff --git a/tests/qyl.collector.tests/Instrumentation/GenAiInstrumentationTests.cs b/tests/qyl.collector.tests/Instrumentation/GenAiInstrumentationTests.cs deleted file mode 100644 index 4e9276614..000000000 --- a/tests/qyl.collector.tests/Instrumentation/GenAiInstrumentationTests.cs +++ /dev/null @@ -1,41 +0,0 @@ -using ANcpLua.Agents.Instrumentation; -using ANcpLua.Agents.Testing.ChatClients; -using Microsoft.Extensions.AI; -using Qyl.Instrumentation.Instrumentation.GenAi; - -namespace Qyl.Collector.Tests.Instrumentation; - -public sealed class GenAiInstrumentationTests -{ - [Fact] - public void WithQylTelemetry_wraps_OpenTelemetryChatClient_in_ToolInstrumenting() - { - var inner = new FakeChatClient { Metadata = new ChatClientMetadata("test-provider", null, "test-model") }; - var otel = new OpenTelemetryChatClient(inner, sourceName: "test"); - - var result = otel.WithQylTelemetry(); - - result.Should().BeOfType(); - } - - [Fact] - public void WithQylTelemetry_does_not_double_wrap_ToolDecoratingChatClient() - { - var inner = new FakeChatClient { Metadata = new ChatClientMetadata("test-provider", null, "test-model") }; - var toolClient = new ToolDecoratingChatClient(inner, GenAiInstrumentation.WrapTool); - - var result = toolClient.WithQylTelemetry(); - - result.Should().BeSameAs(toolClient); - } - - [Fact] - public void WithQylTelemetry_wraps_plain_client_with_full_pipeline() - { - var inner = new FakeChatClient { Metadata = new ChatClientMetadata("test-provider", null, "test-model") }; - - var result = inner.WithQylTelemetry(); - - result.Should().NotBeSameAs(inner); - } -} diff --git a/tests/qyl.collector.tests/Instrumentation/WithQylTelemetryEmissionTests.cs b/tests/qyl.collector.tests/Instrumentation/WithQylTelemetryEmissionTests.cs index 0bcb22597..0771c93be 100644 --- a/tests/qyl.collector.tests/Instrumentation/WithQylTelemetryEmissionTests.cs +++ b/tests/qyl.collector.tests/Instrumentation/WithQylTelemetryEmissionTests.cs @@ -1,4 +1,3 @@ - using ANcpLua.Agents.Testing.ChatClients; using ANcpLua.Agents.Testing.Diagnostics; using ANcpLua.Roslyn.Utilities; @@ -10,56 +9,55 @@ namespace Qyl.Collector.Tests.Instrumentation; public sealed class WithQylTelemetryEmissionTests { [Fact] - public async Task WithQylTelemetry_emits_qyl_genai_activity_on_GetResponseAsync() + public async Task WithQylTelemetry_EmitsActivityOn_qyl_genai_Source() { using var collector = new ActivityCollector("qyl.genai"); + using var inner = new FakeChatClient { Metadata = new ChatClientMetadata("openai", null, "gpt-4o-mini") }; + inner.WithResponse("Hello from fake."); + using var client = inner.WithQylTelemetry("qyl.genai"); - var inner = new FakeChatClient - { - Metadata = new ChatClientMetadata( - "openai", - null, - "gpt-4o-mini") - } - .WithResponse("Hello from fake."); - - var instrumented = inner.WithQylTelemetry("qyl.genai"); - - var response = await instrumented.GetResponseAsync( + await client.GetResponseAsync( [new ChatMessage(ChatRole.User, "Hi")], new ChatOptions { ModelId = "gpt-4o-mini" }, - CancellationToken.None); + TestContext.Current.CancellationToken); - response.Text.Should().Contain("Hello from fake."); + collector.Activities.Should().NotBeEmpty(); + } - collector.Activities.Should().NotBeEmpty( - "WithQylTelemetry must emit at least one Activity on 'qyl.genai' per invocation"); + [Theory] + [InlineData("gen_ai.operation.name")] + [InlineData("gen_ai.request.model")] + [InlineData("gen_ai.provider.name")] + public async Task WithQylTelemetry_EmittedActivity_Carries(string expectedTagKey) + { + using var collector = new ActivityCollector("qyl.genai"); + using var inner = new FakeChatClient { Metadata = new ChatClientMetadata("openai", null, "gpt-4o-mini") }; + inner.WithResponse("Hello from fake."); + using var client = inner.WithQylTelemetry("qyl.genai"); - var chatActivity = collector.Activities - .First(static a => a.OperationName.ContainsIgnoreCase("chat") - || a.Tags.Any(static t => t.Key == "gen_ai.operation.name")); + await client.GetResponseAsync( + [new ChatMessage(ChatRole.User, "Hi")], + new ChatOptions { ModelId = "gpt-4o-mini" }, + TestContext.Current.CancellationToken); - chatActivity.AssertHasTag("gen_ai.operation.name"); - chatActivity.AssertHasTag("gen_ai.request.model"); + var chat = collector.Activities.First(static a => + a.OperationName.ContainsIgnoreCase("chat") + || a.Tags.Any(static t => t.Key == "gen_ai.operation.name")); - chatActivity.Tags.Should().Contain( - static t => t.Key == "gen_ai.provider.name", - "GenAI spans must identify the provider via the 1.40 attribute"); + chat.Tags.Should().Contain(t => t.Key == expectedTagKey); } [Fact] - public async Task WithQylTelemetry_records_call_through_inner_client() + public async Task WithQylTelemetry_DelegatesGetResponse_ToInnerClient() { - var inner = new FakeChatClient { Metadata = new ChatClientMetadata("openai", null, "gpt-4o-mini") } - .WithResponse("ok"); - - var instrumented = inner.WithQylTelemetry("qyl.genai"); + using var inner = new FakeChatClient { Metadata = new ChatClientMetadata("openai", null, "gpt-4o-mini") }; + inner.WithResponse("ok"); + using var client = inner.WithQylTelemetry("qyl.genai"); - await instrumented.GetResponseAsync( + await client.GetResponseAsync( [new ChatMessage(ChatRole.User, "ping")], - cancellationToken: CancellationToken.None); + cancellationToken: TestContext.Current.CancellationToken); inner.CallCount.Should().Be(1); - inner.LastOptions.Should().BeNull(); } } diff --git a/tests/qyl.collector.tests/Instrumentation/WithQylTelemetryWrapTests.cs b/tests/qyl.collector.tests/Instrumentation/WithQylTelemetryWrapTests.cs new file mode 100644 index 000000000..04787c45c --- /dev/null +++ b/tests/qyl.collector.tests/Instrumentation/WithQylTelemetryWrapTests.cs @@ -0,0 +1,53 @@ +using ANcpLua.Agents.Instrumentation; +using ANcpLua.Agents.Testing.ChatClients; +using Microsoft.Extensions.AI; +using Qyl.Instrumentation.Instrumentation.GenAi; + +namespace Qyl.Collector.Tests.Instrumentation; + +public sealed class WithQylTelemetryWrapTests +{ + private static FakeChatClient NewFake() => + new() { Metadata = new ChatClientMetadata("openai", null, "gpt-4o-mini") }; + + [Fact] + public void WithQylTelemetry_WrapsPlainClient() + { + using var inner = NewFake(); + using var client = inner.WithQylTelemetry(); + + client.Should().NotBeSameAs(inner); + } + + [Fact] + public void WithQylTelemetry_WrapsExistingOpenTelemetryClient_InToolDecorator() + { + using var inner = NewFake(); + using var otel = new OpenTelemetryChatClient(inner, sourceName: "test"); + using var client = otel.WithQylTelemetry(); + + client.Should().BeOfType(); + } + + [Fact] + public void WithQylTelemetry_ReturnsSameInstance_WhenAlreadyToolDecorated() + { + using var inner = NewFake(); + using var decorated = new ToolDecoratingChatClient(inner, GenAiInstrumentation.WrapTool); + + decorated.WithQylTelemetry().Should().BeSameAs(decorated); + } + + [Theory] + [InlineData(true)] + [InlineData(false)] + public void WithQylTelemetry_FlipsSensitiveDataFlag_OnExistingOpenTelemetryClient(bool enable) + { + using var inner = NewFake(); + using var otel = new OpenTelemetryChatClient(inner, sourceName: "test") { EnableSensitiveData = !enable }; + + using var client = otel.WithQylTelemetry(enableSensitiveData: enable); + + otel.EnableSensitiveData.Should().Be(enable); + } +} diff --git a/tests/qyl.collector.tests/Telemetry/GenAiMetricsTests.cs b/tests/qyl.collector.tests/Telemetry/GenAiMetricsTests.cs index 686008161..b5fde8b03 100644 --- a/tests/qyl.collector.tests/Telemetry/GenAiMetricsTests.cs +++ b/tests/qyl.collector.tests/Telemetry/GenAiMetricsTests.cs @@ -36,7 +36,10 @@ public async Task ExecuteAsync_Records_Token_And_Duration_Metrics_Without_Activi AssertCommonGenAiTags(outputTokens); var duration = measurements.Should().ContainSingle(static measurement => - measurement.Name == "gen_ai.client.operation.duration").Subject; + measurement.Name == "gen_ai.client.operation.duration" && + measurement.HasTag(GenAiAttributes.OperationName, GenAiAttributes.OperationNameValues.Chat) && + measurement.HasTag(GenAiAttributes.ProviderName, "openai") && + measurement.HasTag(GenAiAttributes.RequestModel, "gpt-5.5")).Subject; duration.Unit.Should().Be("s"); duration.Description.Should().Be("Operation duration"); duration.Value.Should().BeGreaterThanOrEqualTo(0d); diff --git a/tests/qyl.instrumentation.generators.tests/MeterEmitterTests.cs b/tests/qyl.instrumentation.generators.tests/MeterEmitterTests.cs index 99dc7ac54..74fb3849b 100644 --- a/tests/qyl.instrumentation.generators.tests/MeterEmitterTests.cs +++ b/tests/qyl.instrumentation.generators.tests/MeterEmitterTests.cs @@ -1,7 +1,5 @@ using ANcpLua.Roslyn.Utilities.Testing.GeneratorHelpers; -using AwesomeAssertions; -using Qyl.Instrumentation.Generators; -using Xunit; +using Microsoft.CodeAnalysis; namespace Qyl.Instrumentation.Generators.Tests; @@ -10,42 +8,7 @@ public sealed class MeterEmitterTests [Fact] public void Multi_Tag_Measurements_Use_TagList_Instead_Of_Array() { - const string source = """ - using System; - - namespace Qyl.Instrumentation - { - public static class QylServiceDefaults; - } - - namespace Qyl.Instrumentation.Instrumentation - { - [AttributeUsage(AttributeTargets.Class)] - public sealed class MeterAttribute(string name) : Attribute - { - public string Name { get; } = name; - public string? Version { get; set; } - } - - [AttributeUsage(AttributeTargets.Method)] - public sealed class HistogramAttribute(string name) : Attribute - { - public string Name { get; } = name; - public string? Unit { get; set; } - public string? Description { get; set; } - } - - [AttributeUsage(AttributeTargets.Parameter)] - public sealed class TagAttribute(string name) : Attribute - { - public string Name { get; } = name; - } - } - - namespace MyApp - { - using Qyl.Instrumentation.Instrumentation; - + var generated = RunAndGetMeter(MeterTestSources.InMyAppNamespace(""" [Meter("myapp.metrics")] public static partial class MyAppMetrics { @@ -55,18 +18,8 @@ public static partial void RecordRequest( [Tag("route")] string route, [Tag("status_code")] int statusCode); } - } - """; - - var result = GeneratorTestHelper.RunGenerator(source); - - var generatedTree = result.RunResult.GeneratedTrees - .Single(static t => t.FilePath.EndsWith("MeterImplementations.g.cs", StringComparison.Ordinal)); - var generated = generatedTree.ToString(); + """)); - generatedTree.GetDiagnostics(TestContext.Current.CancellationToken) - .Where(static diagnostic => diagnostic.Severity == Microsoft.CodeAnalysis.DiagnosticSeverity.Error) - .Should().BeEmpty(); generated.Should() .Contain("var tags = new global::System.Diagnostics.TagList { { \"route\", route }, { \"status_code\", statusCode } };") .And.Contain("_myappRequestDuration.Record(value, in tags);") @@ -76,42 +29,7 @@ public static partial void RecordRequest( [Fact] public void Gauge_Measurements_Record_Through_Standard_Gauge() { - const string source = """ - using System; - - namespace Qyl.Instrumentation - { - public static class QylServiceDefaults; - } - - namespace Qyl.Instrumentation.Instrumentation - { - [AttributeUsage(AttributeTargets.Class)] - public sealed class MeterAttribute(string name) : Attribute - { - public string Name { get; } = name; - public string? Version { get; set; } - } - - [AttributeUsage(AttributeTargets.Method)] - public sealed class GaugeAttribute(string name) : Attribute - { - public string Name { get; } = name; - public string? Unit { get; set; } - public string? Description { get; set; } - } - - [AttributeUsage(AttributeTargets.Parameter)] - public sealed class TagAttribute(string name) : Attribute - { - public string Name { get; } = name; - } - } - - namespace MyApp - { - using Qyl.Instrumentation.Instrumentation; - + var generated = RunAndGetMeter(MeterTestSources.InMyAppNamespace(""" [Meter("myapp.metrics")] public static partial class MyAppMetrics { @@ -121,18 +39,8 @@ public static partial void RecordQueueDepth( [Tag("queue")] string queue, [Tag("priority")] string priority); } - } - """; + """)); - var result = GeneratorTestHelper.RunGenerator(source); - - var generatedTree = result.RunResult.GeneratedTrees - .Single(static t => t.FilePath.EndsWith("MeterImplementations.g.cs", StringComparison.Ordinal)); - var generated = generatedTree.ToString(); - - generatedTree.GetDiagnostics(TestContext.Current.CancellationToken) - .Where(static diagnostic => diagnostic.Severity == Microsoft.CodeAnalysis.DiagnosticSeverity.Error) - .Should().BeEmpty(); generated.Should() .Contain("private static readonly global::System.Diagnostics.Metrics.Gauge _myappQueueDepth =") .And.Contain("_meter.CreateGauge(\"myapp.queue.depth\", \"{item}\", \"Queued items.\");") @@ -145,60 +53,7 @@ public static partial void RecordQueueDepth( [Fact] public void Standard_Instruments_Without_Value_Parameters_Are_Not_Emitted_Except_Parameterless_Counters() { - const string source = """ - using System; - - namespace Qyl.Instrumentation - { - public static class QylServiceDefaults; - } - - namespace Qyl.Instrumentation.Instrumentation - { - [AttributeUsage(AttributeTargets.Class)] - public sealed class MeterAttribute(string name) : Attribute - { - public string Name { get; } = name; - public string? Version { get; set; } - } - - [AttributeUsage(AttributeTargets.Method)] - public sealed class CounterAttribute(string name) : Attribute - { - public string Name { get; } = name; - public string? Unit { get; set; } - public string? Description { get; set; } - } - - [AttributeUsage(AttributeTargets.Method)] - public sealed class HistogramAttribute(string name) : Attribute - { - public string Name { get; } = name; - public string? Unit { get; set; } - public string? Description { get; set; } - } - - [AttributeUsage(AttributeTargets.Method)] - public sealed class GaugeAttribute(string name) : Attribute - { - public string Name { get; } = name; - public string? Unit { get; set; } - public string? Description { get; set; } - } - - [AttributeUsage(AttributeTargets.Method)] - public sealed class UpDownCounterAttribute(string name) : Attribute - { - public string Name { get; } = name; - public string? Unit { get; set; } - public string? Description { get; set; } - } - } - - namespace MyApp - { - using Qyl.Instrumentation.Instrumentation; - + var generated = RunAndGetMeter(MeterTestSources.InMyAppNamespace(""" [Meter("myapp.metrics")] public static partial class MyAppMetrics { @@ -214,14 +69,7 @@ public static partial class MyAppMetrics [UpDownCounter("myapp.inflight")] private static partial void AddInflight(); } - } - """; - - var result = GeneratorTestHelper.RunGenerator(source); - - var generatedTree = result.RunResult.GeneratedTrees - .Single(static t => t.FilePath.EndsWith("MeterImplementations.g.cs", StringComparison.Ordinal)); - var generated = generatedTree.ToString(); + """)); generated.Should() .Contain("_meter.CreateCounter(\"myapp.events\")") @@ -236,52 +84,7 @@ public static partial class MyAppMetrics [Fact] public void Instruments_With_Unsupported_Value_Types_Are_Not_Emitted() { - const string source = """ - using System; - - namespace Qyl.Instrumentation - { - public static class QylServiceDefaults; - } - - namespace Qyl.Instrumentation.Instrumentation - { - [AttributeUsage(AttributeTargets.Class)] - public sealed class MeterAttribute(string name) : Attribute - { - public string Name { get; } = name; - public string? Version { get; set; } - } - - [AttributeUsage(AttributeTargets.Method)] - public sealed class CounterAttribute(string name) : Attribute - { - public string Name { get; } = name; - public string? Unit { get; set; } - public string? Description { get; set; } - } - - [AttributeUsage(AttributeTargets.Method)] - public sealed class HistogramAttribute(string name) : Attribute - { - public string Name { get; } = name; - public string? Unit { get; set; } - public string? Description { get; set; } - } - - [AttributeUsage(AttributeTargets.Method)] - public sealed class ObservableGaugeAttribute(string name) : Attribute - { - public string Name { get; } = name; - public string? Unit { get; set; } - public string? Description { get; set; } - } - } - - namespace MyApp - { - using Qyl.Instrumentation.Instrumentation; - + var generated = RunAndGetMeter(MeterTestSources.InMyAppNamespace(""" [Meter("myapp.metrics")] public static partial class MyAppMetrics { @@ -297,14 +100,7 @@ public static partial class MyAppMetrics [ObservableGauge("myapp.state")] private static string ObserveState() => "ready"; } - } - """; - - var result = GeneratorTestHelper.RunGenerator(source); - - var generatedTree = result.RunResult.GeneratedTrees - .Single(static t => t.FilePath.EndsWith("MeterImplementations.g.cs", StringComparison.Ordinal)); - var generated = generatedTree.ToString(); + """)); generated.Should() .Contain("_meter.CreateCounter(\"myapp.events\")") @@ -319,36 +115,7 @@ public static partial class MyAppMetrics [Fact] public void Colliding_Metric_Names_Get_Unique_Field_Names() { - const string source = """ - using System; - - namespace Qyl.Instrumentation - { - public static class QylServiceDefaults; - } - - namespace Qyl.Instrumentation.Instrumentation - { - [AttributeUsage(AttributeTargets.Class)] - public sealed class MeterAttribute(string name) : Attribute - { - public string Name { get; } = name; - public string? Version { get; set; } - } - - [AttributeUsage(AttributeTargets.Method)] - public sealed class CounterAttribute(string name) : Attribute - { - public string Name { get; } = name; - public string? Unit { get; set; } - public string? Description { get; set; } - } - } - - namespace MyApp - { - using Qyl.Instrumentation.Instrumentation; - + var generated = RunAndGetMeter(MeterTestSources.InMyAppNamespace(""" [Meter("myapp.metrics")] public static partial class MyAppMetrics { @@ -358,18 +125,8 @@ public static partial class MyAppMetrics [Counter("myapp.cache.hit")] public static partial void RecordCacheDotHit(); } - } - """; - - var result = GeneratorTestHelper.RunGenerator(source); + """)); - var generatedTree = result.RunResult.GeneratedTrees - .Single(static t => t.FilePath.EndsWith("MeterImplementations.g.cs", StringComparison.Ordinal)); - var generated = generatedTree.ToString(); - - generatedTree.GetDiagnostics(TestContext.Current.CancellationToken) - .Where(static diagnostic => diagnostic.Severity == Microsoft.CodeAnalysis.DiagnosticSeverity.Error) - .Should().BeEmpty(); generated.Should() .Contain("private static readonly global::System.Diagnostics.Metrics.Counter _myappCacheHit =") .And.Contain("private static readonly global::System.Diagnostics.Metrics.Counter _myappCacheHit2 =") @@ -380,60 +137,15 @@ public static partial class MyAppMetrics [Fact] public void Valued_Counter_Uses_Method_Value_Type() { - const string source = """ - using System; - - namespace Qyl.Instrumentation - { - public static class QylServiceDefaults; - } - - namespace Qyl.Instrumentation.Instrumentation - { - [AttributeUsage(AttributeTargets.Class)] - public sealed class MeterAttribute(string name) : Attribute - { - public string Name { get; } = name; - public string? Version { get; set; } - } - - [AttributeUsage(AttributeTargets.Method)] - public sealed class CounterAttribute(string name) : Attribute - { - public string Name { get; } = name; - public string? Unit { get; set; } - public string? Description { get; set; } - } - - [AttributeUsage(AttributeTargets.Parameter)] - public sealed class TagAttribute(string name) : Attribute - { - public string Name { get; } = name; - } - } - - namespace MyApp - { - using Qyl.Instrumentation.Instrumentation; - + var generated = RunAndGetMeter(MeterTestSources.InMyAppNamespace(""" [Meter("myapp.metrics")] public static partial class MyAppMetrics { [Counter("myapp.cost", Unit = "USD", Description = "Cost total.")] public static partial void AddCost(double value, [Tag("provider")] string provider); } - } - """; - - var result = GeneratorTestHelper.RunGenerator(source); - - var generatedTree = result.RunResult.GeneratedTrees - .Single(static t => t.FilePath.EndsWith("MeterImplementations.g.cs", StringComparison.Ordinal)); - var generated = generatedTree.ToString(); + """)); - generatedTree.GetDiagnostics(TestContext.Current.CancellationToken) - .Where(static diagnostic => diagnostic.Severity == Microsoft.CodeAnalysis.DiagnosticSeverity.Error) - .Should().BeEmpty(); generated.Should() .Contain("private static readonly global::System.Diagnostics.Metrics.Counter _myappCost =") .And.Contain("_meter.CreateCounter(\"myapp.cost\", \"USD\", \"Cost total.\");") @@ -443,44 +155,7 @@ public static partial class MyAppMetrics [Fact] public void Standard_Metric_Partial_Implementations_Preserve_Method_Accessibility() { - const string source = """ - using System; - - namespace Qyl.Instrumentation - { - public static class QylServiceDefaults; - } - - namespace Qyl.Instrumentation.Instrumentation - { - [AttributeUsage(AttributeTargets.Class)] - public sealed class MeterAttribute(string name) : Attribute - { - public string Name { get; } = name; - public string? Version { get; set; } - } - - [AttributeUsage(AttributeTargets.Method)] - public sealed class CounterAttribute(string name) : Attribute - { - public string Name { get; } = name; - public string? Unit { get; set; } - public string? Description { get; set; } - } - - [AttributeUsage(AttributeTargets.Method)] - public sealed class HistogramAttribute(string name) : Attribute - { - public string Name { get; } = name; - public string? Unit { get; set; } - public string? Description { get; set; } - } - } - - namespace MyApp - { - using Qyl.Instrumentation.Instrumentation; - + var generated = RunAndGetMeter(MeterTestSources.InMyAppNamespace(""" [Meter("myapp.metrics")] public static partial class MyAppMetrics { @@ -493,70 +168,27 @@ public static partial class MyAppMetrics [Histogram("myapp.private")] private static partial void RecordPrivate(double value); } - } - """; - - var result = GeneratorTestHelper.RunGenerator(source); - - var generatedTree = result.RunResult.GeneratedTrees - .Single(static t => t.FilePath.EndsWith("MeterImplementations.g.cs", StringComparison.Ordinal)); - var generated = generatedTree.ToString(); - - generated.Should() - .Contain("public static partial void AddPublic()") - .And.Contain("internal static partial void AddInternal(long value)") - .And.Contain("private static partial void RecordPrivate(double value)") - .And.NotContain("public static partial void AddInternal") - .And.NotContain("public static partial void RecordPrivate"); - } - - [Fact] - public void Meter_Partial_Class_Preserves_Source_Type_Modifiers() - { - const string source = """ - using System; - - namespace Qyl.Instrumentation - { - public static class QylServiceDefaults; - } - - namespace Qyl.Instrumentation.Instrumentation - { - [AttributeUsage(AttributeTargets.Class)] - public sealed class MeterAttribute(string name) : Attribute - { - public string Name { get; } = name; - public string? Version { get; set; } - } - - [AttributeUsage(AttributeTargets.Method)] - public sealed class CounterAttribute(string name) : Attribute - { - public string Name { get; } = name; - public string? Unit { get; set; } - public string? Description { get; set; } - } - } + """)); - namespace MyApp - { - using Qyl.Instrumentation.Instrumentation; + generated.Should() + .Contain("public static partial void AddPublic()") + .And.Contain("internal static partial void AddInternal(long value)") + .And.Contain("private static partial void RecordPrivate(double value)") + .And.NotContain("public static partial void AddInternal") + .And.NotContain("public static partial void RecordPrivate"); + } + [Fact] + public void Meter_Partial_Class_Preserves_Source_Type_Modifiers() + { + var generated = RunAndGetMeter(MeterTestSources.InMyAppNamespace(""" [Meter("myapp.metrics")] public static partial class MyAppMetrics { [Counter("myapp.requests")] public static partial void AddRequest(); } - } - """; - - var result = GeneratorTestHelper.RunGenerator(source); - - var generatedTree = result.RunResult.GeneratedTrees - .Single(static t => t.FilePath.EndsWith("MeterImplementations.g.cs", StringComparison.Ordinal)); - var generated = generatedTree.ToString(); + """)); generated.Should() .Contain("public static partial class MyAppMetrics") @@ -566,36 +198,7 @@ public static partial class MyAppMetrics [Fact] public void Nested_Meter_Class_Generates_Inside_Containing_Partial_Type() { - const string source = """ - using System; - - namespace Qyl.Instrumentation - { - public static class QylServiceDefaults; - } - - namespace Qyl.Instrumentation.Instrumentation - { - [AttributeUsage(AttributeTargets.Class)] - public sealed class MeterAttribute(string name) : Attribute - { - public string Name { get; } = name; - public string? Version { get; set; } - } - - [AttributeUsage(AttributeTargets.Method)] - public sealed class ObservableGaugeAttribute(string name) : Attribute - { - public string Name { get; } = name; - public string? Unit { get; set; } - public string? Description { get; set; } - } - } - - namespace MyApp - { - using Qyl.Instrumentation.Instrumentation; - + var generated = RunAndGetMeter(MeterTestSources.InMyAppNamespace(""" public static partial class Diagnostics { [Meter("myapp.metrics")] @@ -605,14 +208,7 @@ public static partial class MyAppMetrics private static long ObserveQueueDepth() => 42; } } - } - """; - - var result = GeneratorTestHelper.RunGenerator(source); - - var generatedTree = result.RunResult.GeneratedTrees - .Single(static t => t.FilePath.EndsWith("MeterImplementations.g.cs", StringComparison.Ordinal)); - var generated = generatedTree.ToString(); + """)); generated.Should() .Contain(" public static partial class Diagnostics\n {\n public static partial class MyAppMetrics") @@ -627,6 +223,8 @@ public static partial class MyAppMetrics [Fact] public void Private_Nested_Observable_Meter_Generates_Accessible_Module_Initializer() { + // Custom source: this test stubs System.Diagnostics.Metrics.Meter and ObservableGauge + // so we can't reuse the shared preamble (which imports the real namespace). const string source = """ using System; @@ -693,59 +291,17 @@ private static partial class MyAppMetrics } """; - var result = GeneratorTestHelper.RunGenerator(source); - GeneratorTestHelper.AssertCompilationSucceeds(source); - - var generatedTree = result.RunResult.GeneratedTrees - .Single(static t => t.FilePath.EndsWith("MeterImplementations.g.cs", StringComparison.Ordinal)); - var generated = generatedTree.ToString(); + var generated = RunAndGetMeter(source); - generatedTree.GetDiagnostics(TestContext.Current.CancellationToken) - .Where(static diagnostic => diagnostic.Severity == Microsoft.CodeAnalysis.DiagnosticSeverity.Error) - .Should().BeEmpty(); generated.Should() - .Contain(" public static partial class Diagnostics\n {\n private static partial class MyAppMetrics") - .And.Contain("[global::System.Runtime.CompilerServices.ModuleInitializer]") - .And.Contain("internal static void __QylInitializeObservableInstruments()") - .And.Contain("_ = _myappQueueDepth;") - .And.Contain("MyAppMetrics.__QylInitializeObservableInstruments();") - .And.NotContain("global::MyApp.Diagnostics.MyAppMetrics.__QylInitializeObservableInstruments();") - .And.NotContain("QylObservableMeterInitializer"); + .Contain("internal static void __QylInitializeObservableInstruments()") + .And.Contain("MyAppMetrics.__QylInitializeObservableInstruments();"); } [Fact] public void Nested_Meter_Class_Under_Non_Partial_Containing_Type_Is_Not_Emitted() { - const string source = """ - using System; - - namespace Qyl.Instrumentation - { - public static class QylServiceDefaults; - } - - namespace Qyl.Instrumentation.Instrumentation - { - [AttributeUsage(AttributeTargets.Class)] - public sealed class MeterAttribute(string name) : Attribute - { - public string Name { get; } = name; - public string? Version { get; set; } - } - - [AttributeUsage(AttributeTargets.Method)] - public sealed class CounterAttribute(string name) : Attribute - { - public string Name { get; } = name; - public string? Unit { get; set; } - public string? Description { get; set; } - } - } - - namespace MyApp - { - using Qyl.Instrumentation.Instrumentation; - + var generated = RunAndGetMeter(MeterTestSources.InMyAppNamespace(""" [Meter("myapp.valid")] public static partial class ValidMetrics { @@ -762,14 +318,7 @@ public static partial class InvalidMetrics public static partial void AddInvalidRequest(); } } - } - """; - - var result = GeneratorTestHelper.RunGenerator(source); - - var generatedTree = result.RunResult.GeneratedTrees - .Single(static t => t.FilePath.EndsWith("MeterImplementations.g.cs", StringComparison.Ordinal)); - var generated = generatedTree.ToString(); + """)); generated.Should() .Contain("myapp.valid.requests") @@ -781,36 +330,7 @@ public static partial class InvalidMetrics [Fact] public void Generic_Meter_Type_Shapes_Are_Not_Emitted() { - const string source = """ - using System; - - namespace Qyl.Instrumentation - { - public static class QylServiceDefaults; - } - - namespace Qyl.Instrumentation.Instrumentation - { - [AttributeUsage(AttributeTargets.Class)] - public sealed class MeterAttribute(string name) : Attribute - { - public string Name { get; } = name; - public string? Version { get; set; } - } - - [AttributeUsage(AttributeTargets.Method)] - public sealed class CounterAttribute(string name) : Attribute - { - public string Name { get; } = name; - public string? Unit { get; set; } - public string? Description { get; set; } - } - } - - namespace MyApp - { - using Qyl.Instrumentation.Instrumentation; - + var generated = RunAndGetMeter(MeterTestSources.InMyAppNamespace(""" [Meter("myapp.valid")] public static partial class ValidMetrics { @@ -834,14 +354,7 @@ public static partial class NestedMetrics public static partial void AddNestedRequest(); } } - } - """; - - var result = GeneratorTestHelper.RunGenerator(source); - - var generatedTree = result.RunResult.GeneratedTrees - .Single(static t => t.FilePath.EndsWith("MeterImplementations.g.cs", StringComparison.Ordinal)); - var generated = generatedTree.ToString(); + """)); generated.Should() .Contain("myapp.valid.requests") @@ -856,37 +369,7 @@ public static partial class NestedMetrics [Fact] public void Escaped_CSharp_Identifiers_Are_Preserved_In_Generated_Meter_Code() { - const string source = """ - using System; - - namespace Qyl.Instrumentation - { - public static class QylServiceDefaults; - } - - namespace Qyl.Instrumentation.Instrumentation - { - [AttributeUsage(AttributeTargets.Class)] - public sealed class MeterAttribute(string name) : Attribute - { - public string Name { get; } = name; - public string? Version { get; set; } - } - - [AttributeUsage(AttributeTargets.Method)] - public sealed class CounterAttribute(string name) : Attribute - { - public string Name { get; } = name; - public string? Unit { get; set; } - public string? Description { get; set; } - } - - [AttributeUsage(AttributeTargets.Parameter)] - public sealed class TagAttribute(string name) : Attribute - { - public string Name { get; } = name; - } - } + var generated = RunAndGetMeter(MeterTestSources.Preamble + """ namespace @event { @@ -899,13 +382,7 @@ public static partial class @class public static partial void @default(long @long, [Tag("route")] string @string); } } - """; - - var result = GeneratorTestHelper.RunGenerator(source); - - var generatedTree = result.RunResult.GeneratedTrees - .Single(static t => t.FilePath.EndsWith("MeterImplementations.g.cs", StringComparison.Ordinal)); - var generated = generatedTree.ToString(); + """); generated.Should() .Contain("namespace @event") @@ -915,114 +392,35 @@ public static partial class @class .And.NotContain("namespace event") .And.NotContain("partial class class") .And.NotContain("partial void default") - .And.NotContain("long long") - .And.NotContain("string string"); - } - - [Fact] - public void Standard_Metric_Partial_Implementations_Preserve_Value_Parameter_Name() - { - const string source = """ - using System; - - namespace Qyl.Instrumentation - { - public static class QylServiceDefaults; - } - - namespace Qyl.Instrumentation.Instrumentation - { - [AttributeUsage(AttributeTargets.Class)] - public sealed class MeterAttribute(string name) : Attribute - { - public string Name { get; } = name; - public string? Version { get; set; } - } - - [AttributeUsage(AttributeTargets.Method)] - public sealed class HistogramAttribute(string name) : Attribute - { - public string Name { get; } = name; - public string? Unit { get; set; } - public string? Description { get; set; } - } - - [AttributeUsage(AttributeTargets.Parameter)] - public sealed class TagAttribute(string name) : Attribute - { - public string Name { get; } = name; - } - } - - namespace MyApp - { - using Qyl.Instrumentation.Instrumentation; - - [Meter("myapp.metrics")] - public static partial class MyAppMetrics - { - [Histogram("myapp.request.duration")] - public static partial void RecordDuration( - double durationMs, - [Tag("route")] string route); - } - } - """; - - var result = GeneratorTestHelper.RunGenerator(source); - - var generatedTree = result.RunResult.GeneratedTrees - .Single(static t => t.FilePath.EndsWith("MeterImplementations.g.cs", StringComparison.Ordinal)); - var generated = generatedTree.ToString(); - - generated.Should() - .Contain("public static partial void RecordDuration(double durationMs, string route)") - .And.Contain("_myappRequestDuration.Record(durationMs, new global::System.Collections.Generic.KeyValuePair(\"route\", route));") - .And.NotContain("RecordDuration(double value, string route)") - .And.NotContain("_myappRequestDuration.Record(value"); - } - - [Fact] - public void Standard_Instruments_With_Unsupported_Partial_Method_Shapes_Are_Not_Emitted() - { - const string source = """ - using System; - - namespace Qyl.Instrumentation - { - public static class QylServiceDefaults; - } - - namespace Qyl.Instrumentation.Instrumentation - { - [AttributeUsage(AttributeTargets.Class)] - public sealed class MeterAttribute(string name) : Attribute - { - public string Name { get; } = name; - public string? Version { get; set; } - } - - [AttributeUsage(AttributeTargets.Method)] - public sealed class CounterAttribute(string name) : Attribute - { - public string Name { get; } = name; - public string? Unit { get; set; } - public string? Description { get; set; } - } + .And.NotContain("long long") + .And.NotContain("string string"); + } - [AttributeUsage(AttributeTargets.Method)] - public sealed class HistogramAttribute(string name) : Attribute + [Fact] + public void Standard_Metric_Partial_Implementations_Preserve_Value_Parameter_Name() + { + var generated = RunAndGetMeter(MeterTestSources.InMyAppNamespace(""" + [Meter("myapp.metrics")] + public static partial class MyAppMetrics { - public string Name { get; } = name; - public string? Unit { get; set; } - public string? Description { get; set; } + [Histogram("myapp.request.duration")] + public static partial void RecordDuration( + double durationMs, + [Tag("route")] string route); } - } + """)); - namespace MyApp - { - using Qyl.Instrumentation.Instrumentation; + generated.Should() + .Contain("public static partial void RecordDuration(double durationMs, string route)") + .And.Contain("_myappRequestDuration.Record(durationMs, new global::System.Collections.Generic.KeyValuePair(\"route\", route));") + .And.NotContain("RecordDuration(double value, string route)") + .And.NotContain("_myappRequestDuration.Record(value"); + } + [Fact] + public void Standard_Instruments_With_Unsupported_Partial_Method_Shapes_Are_Not_Emitted() + { + var generated = RunAndGetMeter(MeterTestSources.InMyAppNamespace(""" [Meter("myapp.metrics")] public static partial class MyAppMetrics { @@ -1038,14 +436,7 @@ public static partial class MyAppMetrics [Counter("myapp.byref")] public static partial void AddByRef(ref long value); } - } - """; - - var result = GeneratorTestHelper.RunGenerator(source); - - var generatedTree = result.RunResult.GeneratedTrees - .Single(static t => t.FilePath.EndsWith("MeterImplementations.g.cs", StringComparison.Ordinal)); - var generated = generatedTree.ToString(); + """)); generated.Should() .Contain("_meter.CreateCounter(\"myapp.valid\")") @@ -1060,54 +451,15 @@ public static partial class MyAppMetrics [Fact] public void Observable_Gauge_Callback_Generates_Observable_Instrument_And_Module_Initializer() { - const string source = """ - using System; - - namespace Qyl.Instrumentation - { - public static class QylServiceDefaults; - } - - namespace Qyl.Instrumentation.Instrumentation - { - [AttributeUsage(AttributeTargets.Class)] - public sealed class MeterAttribute(string name) : Attribute - { - public string Name { get; } = name; - public string? Version { get; set; } - } - - [AttributeUsage(AttributeTargets.Method)] - public sealed class ObservableGaugeAttribute(string name) : Attribute - { - public string Name { get; } = name; - public string? Unit { get; set; } - public string? Description { get; set; } - } - } - - namespace MyApp - { - using Qyl.Instrumentation.Instrumentation; - + var generated = RunAndGetMeter(MeterTestSources.InMyAppNamespace(""" [Meter("myapp.metrics")] public static partial class MyAppMetrics { [ObservableGauge("myapp.queue.depth", Unit = "{item}", Description = "Queued items.")] private static long ObserveQueueDepth() => 42; } - } - """; - - var result = GeneratorTestHelper.RunGenerator(source); - - var generatedTree = result.RunResult.GeneratedTrees - .Single(static t => t.FilePath.EndsWith("MeterImplementations.g.cs", StringComparison.Ordinal)); - var generated = generatedTree.ToString(); + """)); - generatedTree.GetDiagnostics(TestContext.Current.CancellationToken) - .Where(static diagnostic => diagnostic.Severity == Microsoft.CodeAnalysis.DiagnosticSeverity.Error) - .Should().BeEmpty(); generated.Should() .Contain("private static readonly global::System.Diagnostics.Metrics.ObservableGauge _myappQueueDepth =") .And.Contain("_meter.CreateObservableGauge(\"myapp.queue.depth\", new global::System.Func(ObserveQueueDepth), \"{item}\", \"Queued items.\");") @@ -1122,38 +474,7 @@ public static partial class MyAppMetrics [Fact] public void Observable_Counter_Callback_Can_Return_Tagged_Measurements() { - const string source = """ - using System; - using System.Collections.Generic; - using System.Diagnostics.Metrics; - - namespace Qyl.Instrumentation - { - public static class QylServiceDefaults; - } - - namespace Qyl.Instrumentation.Instrumentation - { - [AttributeUsage(AttributeTargets.Class)] - public sealed class MeterAttribute(string name) : Attribute - { - public string Name { get; } = name; - public string? Version { get; set; } - } - - [AttributeUsage(AttributeTargets.Method)] - public sealed class ObservableCounterAttribute(string name) : Attribute - { - public string Name { get; } = name; - public string? Unit { get; set; } - public string? Description { get; set; } - } - } - - namespace MyApp - { - using Qyl.Instrumentation.Instrumentation; - + var generated = RunAndGetMeter(MeterTestSources.InMyAppNamespace(""" [Meter("myapp.metrics")] public static partial class MyAppMetrics { @@ -1164,18 +485,8 @@ private static IEnumerable> ObserveRequests() => new(7, new KeyValuePair("route", "/checkout")) ]; } - } - """; - - var result = GeneratorTestHelper.RunGenerator(source); - - var generatedTree = result.RunResult.GeneratedTrees - .Single(static t => t.FilePath.EndsWith("MeterImplementations.g.cs", StringComparison.Ordinal)); - var generated = generatedTree.ToString(); + """)); - generatedTree.GetDiagnostics(TestContext.Current.CancellationToken) - .Where(static diagnostic => diagnostic.Severity == Microsoft.CodeAnalysis.DiagnosticSeverity.Error) - .Should().BeEmpty(); generated.Should() .Contain("private static readonly global::System.Diagnostics.Metrics.ObservableCounter _myappRequests =") .And.Contain("_meter.CreateObservableCounter(\"myapp.requests\", new global::System.Func>>(ObserveRequests), \"{request}\", \"Observed requests.\");") @@ -1186,38 +497,7 @@ private static IEnumerable> ObserveRequests() => [Fact] public void Observable_UpDownCounter_Callback_Can_Return_Tagged_Measurement() { - const string source = """ - using System; - using System.Collections.Generic; - using System.Diagnostics.Metrics; - - namespace Qyl.Instrumentation - { - public static class QylServiceDefaults; - } - - namespace Qyl.Instrumentation.Instrumentation - { - [AttributeUsage(AttributeTargets.Class)] - public sealed class MeterAttribute(string name) : Attribute - { - public string Name { get; } = name; - public string? Version { get; set; } - } - - [AttributeUsage(AttributeTargets.Method)] - public sealed class ObservableUpDownCounterAttribute(string name) : Attribute - { - public string Name { get; } = name; - public string? Unit { get; set; } - public string? Description { get; set; } - } - } - - namespace MyApp - { - using Qyl.Instrumentation.Instrumentation; - + var generated = RunAndGetMeter(MeterTestSources.InMyAppNamespace(""" [Meter("myapp.metrics")] public static partial class MyAppMetrics { @@ -1225,18 +505,8 @@ public static partial class MyAppMetrics private static Measurement ObserveWorkItems() => new(1.5, new KeyValuePair("queue", "main")); } - } - """; - - var result = GeneratorTestHelper.RunGenerator(source); - - var generatedTree = result.RunResult.GeneratedTrees - .Single(static t => t.FilePath.EndsWith("MeterImplementations.g.cs", StringComparison.Ordinal)); - var generated = generatedTree.ToString(); + """)); - generatedTree.GetDiagnostics(TestContext.Current.CancellationToken) - .Where(static diagnostic => diagnostic.Severity == Microsoft.CodeAnalysis.DiagnosticSeverity.Error) - .Should().BeEmpty(); generated.Should() .Contain("private static readonly global::System.Diagnostics.Metrics.ObservableUpDownCounter _myappWorkItems =") .And.Contain("_meter.CreateObservableUpDownCounter(\"myapp.work.items\", new global::System.Func>(ObserveWorkItems), \"{item}\", \"Observed work.\");") @@ -1247,31 +517,8 @@ private static Measurement ObserveWorkItems() => [Fact] public void Global_Namespace_Meter_Class_Generates_Valid_Partial_Class() { - const string source = """ - using System; - - namespace Qyl.Instrumentation - { - public static class QylServiceDefaults; - } - - namespace Qyl.Instrumentation.Instrumentation - { - [AttributeUsage(AttributeTargets.Class)] - public sealed class MeterAttribute(string name) : Attribute - { - public string Name { get; } = name; - public string? Version { get; set; } - } - - [AttributeUsage(AttributeTargets.Method)] - public sealed class ObservableGaugeAttribute(string name) : Attribute - { - public string Name { get; } = name; - public string? Unit { get; set; } - public string? Description { get; set; } - } - } + // Custom source: the meter class is at the global namespace, not inside MyApp. + var generated = RunAndGetMeter(MeterTestSources.Preamble + """ [Qyl.Instrumentation.Instrumentation.Meter("global.metrics")] public static partial class GlobalMetrics @@ -1279,17 +526,8 @@ public static partial class GlobalMetrics [Qyl.Instrumentation.Instrumentation.ObservableGauge("global.queue.depth")] private static long ObserveQueueDepth() => 1; } - """; - - var result = GeneratorTestHelper.RunGenerator(source); + """); - var generatedTree = result.RunResult.GeneratedTrees - .Single(static t => t.FilePath.EndsWith("MeterImplementations.g.cs", StringComparison.Ordinal)); - var generated = generatedTree.ToString(); - - generatedTree.GetDiagnostics(TestContext.Current.CancellationToken) - .Where(static diagnostic => diagnostic.Severity == Microsoft.CodeAnalysis.DiagnosticSeverity.Error) - .Should().BeEmpty(); generated.Should() .Contain("partial class GlobalMetrics") .And.Contain("[global::System.Runtime.CompilerServices.ModuleInitializer]") @@ -1300,42 +538,7 @@ public static partial class GlobalMetrics [Fact] public void String_Metadata_Is_Emitted_As_CSharp_Literals() { - const string source = """ - using System; - - namespace Qyl.Instrumentation - { - public static class QylServiceDefaults; - } - - namespace Qyl.Instrumentation.Instrumentation - { - [AttributeUsage(AttributeTargets.Class)] - public sealed class MeterAttribute(string name) : Attribute - { - public string Name { get; } = name; - public string? Version { get; set; } - } - - [AttributeUsage(AttributeTargets.Method)] - public sealed class HistogramAttribute(string name) : Attribute - { - public string Name { get; } = name; - public string? Unit { get; set; } - public string? Description { get; set; } - } - - [AttributeUsage(AttributeTargets.Parameter)] - public sealed class TagAttribute(string name) : Attribute - { - public string Name { get; } = name; - } - } - - namespace MyApp - { - using Qyl.Instrumentation.Instrumentation; - + var generated = RunAndGetMeter(MeterTestSources.InMyAppNamespace(""" [Meter("myapp.\"metrics", Version = "2026\n05")] public static partial class MyAppMetrics { @@ -1344,18 +547,8 @@ public static partial void RecordRequest( double value, [Tag("http.route\"quoted")] string route); } - } - """; - - var result = GeneratorTestHelper.RunGenerator(source); + """)); - var generatedTree = result.RunResult.GeneratedTrees - .Single(static t => t.FilePath.EndsWith("MeterImplementations.g.cs", StringComparison.Ordinal)); - var generated = generatedTree.ToString(); - - generatedTree.GetDiagnostics(TestContext.Current.CancellationToken) - .Where(static diagnostic => diagnostic.Severity == Microsoft.CodeAnalysis.DiagnosticSeverity.Error) - .Should().BeEmpty(); generated.Should() .Contain("new global::System.Diagnostics.Metrics.Meter(\"myapp.\\\"metrics\", \"2026\\n05\")") .And.Contain("_meter.CreateHistogram(\"myapp.request.\\\"duration\", \"ms\\n\", \"Request \\\"duration\\\".\\nLine two.\");") @@ -1365,56 +558,31 @@ public static partial void RecordRequest( [Fact] public void Non_Ascii_Metadata_Is_Emitted_As_Escaped_CSharp_Literals() { - const string source = """ - using System; - - namespace Qyl.Instrumentation - { - public static class QylServiceDefaults; - } - - namespace Qyl.Instrumentation.Instrumentation - { - [AttributeUsage(AttributeTargets.Class)] - public sealed class MeterAttribute(string name) : Attribute - { - public string Name { get; } = name; - public string? Version { get; set; } - } - - [AttributeUsage(AttributeTargets.Method)] - public sealed class CounterAttribute(string name) : Attribute - { - public string Name { get; } = name; - public string? Unit { get; set; } - public string? Description { get; set; } - } - } - - namespace MyApp - { - using Qyl.Instrumentation.Instrumentation; - - [Meter("myapp.m\u00e9trics")] + var generated = RunAndGetMeter(MeterTestSources.InMyAppNamespace(""" + [Meter("myapp.métrics")] public static partial class MyAppMetrics { - [Counter("myapp.r\u00e9quests", Description = "D\u00e9j\u00e0 vu.")] + [Counter("myapp.réquests", Description = "Déjà vu.")] public static partial void AddRequest(); } - } - """; + """)); + + generated.Should() + .Contain("[assembly: global::Qyl.Instrumentation.GeneratedMeterAttribute(\"myapp.m\\u00e9trics\")]") + .And.Contain("_meter.CreateCounter(\"myapp.r\\u00e9quests\", null, \"D\\u00e9j\\u00e0 vu.\");"); + } + private static string RunAndGetMeter(string source) + { var result = GeneratorTestHelper.RunGenerator(source); - var generatedTree = result.RunResult.GeneratedTrees + var tree = result.RunResult.GeneratedTrees .Single(static t => t.FilePath.EndsWith("MeterImplementations.g.cs", StringComparison.Ordinal)); - var generated = generatedTree.ToString(); - generatedTree.GetDiagnostics(TestContext.Current.CancellationToken) - .Where(static diagnostic => diagnostic.Severity == Microsoft.CodeAnalysis.DiagnosticSeverity.Error) + tree.GetDiagnostics(TestContext.Current.CancellationToken) + .Where(static d => d.Severity == DiagnosticSeverity.Error) .Should().BeEmpty(); - generated.Should() - .Contain("[assembly: global::Qyl.Instrumentation.GeneratedMeterAttribute(\"myapp.m\\u00e9trics\")]") - .And.Contain("_meter.CreateCounter(\"myapp.r\\u00e9quests\", null, \"D\\u00e9j\\u00e0 vu.\");"); + + return tree.ToString(); } } diff --git a/tests/qyl.instrumentation.generators.tests/MeterTestSources.cs b/tests/qyl.instrumentation.generators.tests/MeterTestSources.cs new file mode 100644 index 000000000..54551b708 --- /dev/null +++ b/tests/qyl.instrumentation.generators.tests/MeterTestSources.cs @@ -0,0 +1,100 @@ +namespace Qyl.Instrumentation.Generators.Tests; + +/// +/// Shared boilerplate for MeterEmitterTests: every test fixture needs the same +/// Qyl.Instrumentation marker class plus the full set of attribute declarations. +/// Extracting them here lets each test focus on the meter-and-instrument code that +/// is unique to that scenario. +/// +internal static class MeterTestSources +{ + public const string Preamble = """ + using System; + using System.Collections.Generic; + using System.Diagnostics.Metrics; + + namespace Qyl.Instrumentation + { + public static class QylServiceDefaults; + } + + namespace Qyl.Instrumentation.Instrumentation + { + [AttributeUsage(AttributeTargets.Class)] + public sealed class MeterAttribute(string name) : Attribute + { + public string Name { get; } = name; + public string? Version { get; set; } + } + + [AttributeUsage(AttributeTargets.Method)] + public sealed class CounterAttribute(string name) : Attribute + { + public string Name { get; } = name; + public string? Unit { get; set; } + public string? Description { get; set; } + } + + [AttributeUsage(AttributeTargets.Method)] + public sealed class HistogramAttribute(string name) : Attribute + { + public string Name { get; } = name; + public string? Unit { get; set; } + public string? Description { get; set; } + } + + [AttributeUsage(AttributeTargets.Method)] + public sealed class GaugeAttribute(string name) : Attribute + { + public string Name { get; } = name; + public string? Unit { get; set; } + public string? Description { get; set; } + } + + [AttributeUsage(AttributeTargets.Method)] + public sealed class UpDownCounterAttribute(string name) : Attribute + { + public string Name { get; } = name; + public string? Unit { get; set; } + public string? Description { get; set; } + } + + [AttributeUsage(AttributeTargets.Method)] + public sealed class ObservableGaugeAttribute(string name) : Attribute + { + public string Name { get; } = name; + public string? Unit { get; set; } + public string? Description { get; set; } + } + + [AttributeUsage(AttributeTargets.Method)] + public sealed class ObservableCounterAttribute(string name) : Attribute + { + public string Name { get; } = name; + public string? Unit { get; set; } + public string? Description { get; set; } + } + + [AttributeUsage(AttributeTargets.Method)] + public sealed class ObservableUpDownCounterAttribute(string name) : Attribute + { + public string Name { get; } = name; + public string? Unit { get; set; } + public string? Description { get; set; } + } + + [AttributeUsage(AttributeTargets.Parameter)] + public sealed class TagAttribute(string name) : Attribute + { + public string Name { get; } = name; + } + } + """; + + /// + /// Wraps the supplied meter code in namespace MyApp { using Qyl.Instrumentation.Instrumentation; … } + /// — the most common shape used by these tests. + /// + public static string InMyAppNamespace(string meterCode) => + Preamble + "\n\nnamespace MyApp\n{\n using Qyl.Instrumentation.Instrumentation;\n\n" + meterCode + "\n}\n"; +} diff --git a/tests/qyl.mcp.tests/Formatting/ErrorFormatterTests.cs b/tests/qyl.mcp.tests/Formatting/ErrorFormatterTests.cs index 2bffb9516..32262258a 100644 --- a/tests/qyl.mcp.tests/Formatting/ErrorFormatterTests.cs +++ b/tests/qyl.mcp.tests/Formatting/ErrorFormatterTests.cs @@ -1,3 +1,4 @@ +using System.Net; using qyl.mcp; using qyl.mcp.Formatting; @@ -5,19 +6,66 @@ namespace Qyl.Mcp.Tests.Formatting; public sealed class ErrorFormatterTests { + [Theory] + [InlineData(HttpStatusCode.NotFound, "**Not Found**")] + [InlineData(HttpStatusCode.BadRequest, "**Invalid Request**")] + [InlineData(HttpStatusCode.Unauthorized, "**Authentication Required**")] + [InlineData(HttpStatusCode.Forbidden, "**Access Denied**")] + [InlineData(HttpStatusCode.InternalServerError, "**Collector Error**")] + [InlineData(HttpStatusCode.BadGateway, "**Collector Error**")] + [InlineData(HttpStatusCode.RequestTimeout, "**Connection Error**")] + public void FormatForLlm_CategorisesHttpErrorsByStatus(HttpStatusCode status, string category) => + ErrorFormatter.FormatForLlm(new HttpRequestException("boom", inner: null, status), McpTransportMode.Http) + .Should().StartWith(category); + [Fact] - public async Task FormatForLlm_TreatsCancelledTaskAsCancellationNotCollectorTimeout() + public void FormatForLlm_TreatsCancelledTaskAsCancellationNotTimeout() { using var cts = new CancellationTokenSource(); - await cts.CancelAsync(); + cts.Cancel(); - var error = new TaskCanceledException( - "The operation was canceled.", - innerException: null, - token: cts.Token); + ErrorFormatter.FormatForLlm(new TaskCanceledException("op cancelled", innerException: null, cts.Token), McpTransportMode.Stdio) + .Should().Be("**Cancelled:** The operation was cancelled."); + } - var output = ErrorFormatter.FormatForLlm(error, McpTransportMode.Stdio); + [Fact] + public void FormatForLlm_TreatsTaskCanceledWithoutTokenAsTimeout() => + ErrorFormatter.FormatForLlm(new TaskCanceledException("timeout"), McpTransportMode.Stdio) + .Should().StartWith("**Timeout:**"); - output.Should().Be("**Cancelled:** The operation was cancelled."); - } + [Fact] + public void FormatForLlm_FormatsOperationCancelledWithBudgetHint_WhenMessageMentionsToolCallLimit() => + ErrorFormatter.FormatForLlm(new OperationCanceledException("tool call limit exceeded"), McpTransportMode.Stdio) + .Should().StartWith("**Investigation Budget Reached**"); + + [Theory] + [InlineData(true, "MCP server process is running")] + [InlineData(false, "endpoint URL and network reachability")] + public void FormatForLlm_AdaptsTransportHintForIoException(bool stdio, string hintFragment) => + ErrorFormatter.FormatForLlm(new IOException("pipe closed"), Transport(stdio)) + .Should().Contain(hintFragment); + + [Theory] + [InlineData(true, "Check your environment variables")] + [InlineData(false, "Contact the administrator")] + public void FormatForLlm_AdaptsTransportHintForConfigError(bool stdio, string hintFragment) => + ErrorFormatter.FormatForLlm(new InvalidOperationException("misconfigured"), Transport(stdio)) + .Should().Contain(hintFragment); + + [Theory] + [InlineData(true, "kaboom")] + [InlineData(false, "An unexpected error occurred")] + public void FormatForLlm_LeaksMessageOnlyOverStdio_ForUnknownExceptions(bool stdio, string fragment) => + ErrorFormatter.FormatForLlm(new ArgumentException("kaboom"), Transport(stdio)) + .Should().Contain(fragment); + + [Theory] + [InlineData(HttpStatusCode.Unauthorized, true, "QYL_MCP_TOKEN")] + [InlineData(HttpStatusCode.Unauthorized, false, "Re-authenticate")] + public void FormatForLlm_AdaptsAuthHintByTransport(HttpStatusCode status, bool stdio, string hintFragment) => + ErrorFormatter.FormatForLlm(new HttpRequestException("no auth", inner: null, status), Transport(stdio)) + .Should().Contain(hintFragment); + + private static McpTransportMode Transport(bool stdio) => + stdio ? McpTransportMode.Stdio : McpTransportMode.Http; } diff --git a/tests/qyl.mcp.tests/Scoping/QylScopeInjectorTests.cs b/tests/qyl.mcp.tests/Scoping/QylScopeInjectorTests.cs index 4e6710998..cc55c12ae 100644 --- a/tests/qyl.mcp.tests/Scoping/QylScopeInjectorTests.cs +++ b/tests/qyl.mcp.tests/Scoping/QylScopeInjectorTests.cs @@ -8,97 +8,52 @@ public sealed class QylScopeInjectorTests private static readonly QylScopeInjector Injector = new(); [Fact] - public void Inject_PreservesArguments_WhenScopeIsEmpty() - { - var args = new Dictionary(StringComparer.OrdinalIgnoreCase) - { - ["someExisting"] = Str("existing-value") - }; - - var result = Injector.Inject(args, QylScope.ForTest()); - - result.Should().BeSameAs(args); - result["someExisting"].GetString().Should().Be("existing-value"); - result.Should().NotContainKey("serviceName"); - result.Should().NotContainKey("sessionId"); - } - - [Fact] - public void Inject_ReturnsNull_WhenScopeIsEmptyAndArgsAreNull() - { - var result = Injector.Inject(arguments: null, QylScope.ForTest()); - - result.Should().BeNull(); - } + public void Inject_ReturnsNull_WhenScopeIsEmptyAndArgsAreNull() => + Injector.Inject(arguments: null, QylScope.ForTest()).Should().BeNull(); [Fact] - public void Inject_CreatesNewDict_WhenArgsAreNullAndScopeIsPresent() + public void Inject_ReturnsArgsUnchanged_WhenScopeIsEmpty() { - var scope = QylScope.ForTest(serviceName: "svc", sessionId: "sess"); + var args = Args(("someExisting", Str("existing-value"))); - var result = Injector.Inject(arguments: null, scope); - - var injected = RequireInjected(result); - injected["serviceName"].GetString().Should().Be("svc"); - injected["sessionId"].GetString().Should().Be("sess"); - } - - [Fact] - public void Inject_AddsServiceNameOnly_WhenScopeHasOnlyServiceName() - { - var args = new Dictionary(StringComparer.OrdinalIgnoreCase); - var scope = QylScope.ForTest(serviceName: "only-service"); - - var result = Injector.Inject(args, scope); - - result.Should().BeSameAs(args); - result["serviceName"].GetString().Should().Be("only-service"); - result.Should().NotContainKey("sessionId"); + Injector.Inject(args, QylScope.ForTest()).Should().BeSameAs(args); + args.Should().NotContainKey("serviceName").And.NotContainKey("sessionId"); } [Fact] - public void Inject_AddsSessionIdOnly_WhenScopeHasOnlySessionId() + public void Inject_MutatesArgsInPlace_AndReturnsSameReference() { - var args = new Dictionary(StringComparer.OrdinalIgnoreCase); - var scope = QylScope.ForTest(sessionId: "only-session"); + var args = Args(); - var result = Injector.Inject(args, scope); + var result = Injector.Inject(args, QylScope.ForTest(serviceName: "svc")); result.Should().BeSameAs(args); - result["sessionId"].GetString().Should().Be("only-session"); - result.Should().NotContainKey("serviceName"); + args.Should().ContainKey("serviceName"); } - [Fact] - public void Inject_PreservesCallerServiceName_WhenCallerSetsNonEmptyString() + [Theory] + [InlineData("svc", null, "svc", null)] + [InlineData(null, "sess", null, "sess")] + [InlineData("svc", "sess", "svc", "sess")] + public void Inject_PopulatesMissingKeys_FromScope(string? scopeService, string? scopeSession, string? expectedService, string? expectedSession) { - var args = new Dictionary(StringComparer.OrdinalIgnoreCase) - { - ["serviceName"] = Str("caller-service") - }; - var scope = QylScope.ForTest(serviceName: "scope-service", sessionId: "scope-session"); - - var result = Injector.Inject(args, scope); + var result = Injector.Inject(Args(), QylScope.ForTest(scopeService, scopeSession)); - result.Should().NotBeNull(); - result["serviceName"].GetString().Should().Be("caller-service"); - result["sessionId"].GetString().Should().Be("scope-session"); + Read(result, "serviceName").Should().Be(expectedService); + Read(result, "sessionId").Should().Be(expectedSession); } - [Fact] - public void Inject_PreservesCallerSessionId_WhenCallerSetsNonEmptyString() + [Theory] + [InlineData("serviceName", "caller-service", "caller-service", "scope-session")] + [InlineData("sessionId", "caller-session", "scope-service", "caller-session")] + public void Inject_PreservesCallerValue_WhenExistingIsNonEmptyString(string callerKey, string callerValue, string expectedService, string expectedSession) { - var args = new Dictionary(StringComparer.OrdinalIgnoreCase) - { - ["sessionId"] = Str("caller-session") - }; - var scope = QylScope.ForTest(serviceName: "scope-service", sessionId: "scope-session"); + var args = Args((callerKey, Str(callerValue))); - var result = Injector.Inject(args, scope); + var result = Injector.Inject(args, QylScope.ForTest("scope-service", "scope-session")); - result.Should().NotBeNull(); - result["sessionId"].GetString().Should().Be("caller-session"); - result["serviceName"].GetString().Should().Be("scope-service"); + Read(result, "serviceName").Should().Be(expectedService); + Read(result, "sessionId").Should().Be(expectedSession); } [Theory] @@ -108,79 +63,47 @@ public void Inject_PreservesCallerSessionId_WhenCallerSetsNonEmptyString() [InlineData("null")] [InlineData("[1,2]")] [InlineData("{\"a\":1}")] - public void Inject_OverwritesCallerServiceName_WhenExistingValueIsNotNonEmptyString(string existingJson) + public void Inject_OverwritesCallerServiceName_WhenExistingIsNotNonEmptyString(string existingJson) { - var args = new Dictionary(StringComparer.OrdinalIgnoreCase) - { - ["serviceName"] = Json(existingJson) - }; - var scope = QylScope.ForTest(serviceName: "scope-service"); + var args = Args(("serviceName", Json(existingJson))); - var result = Injector.Inject(args, scope); + var result = Injector.Inject(args, QylScope.ForTest(serviceName: "scope-service")); - result.Should().NotBeNull(); - result["serviceName"].ValueKind.Should().Be(JsonValueKind.String); - result["serviceName"].GetString().Should().Be("scope-service"); + Read(result, "serviceName").Should().Be("scope-service"); } [Fact] - public void Inject_PreservesMixedCaseCallerKey_WhenDictIsCaseInsensitive() + public void Inject_PreservesMixedCaseCallerKey_OnCaseInsensitiveDict() { - var args = new Dictionary(StringComparer.OrdinalIgnoreCase) - { - ["ServiceName"] = Str("caller-service") - }; - var scope = QylScope.ForTest(serviceName: "scope-service"); + var args = Args(("ServiceName", Str("caller-service"))); - var result = Injector.Inject(args, scope); + var result = Injector.Inject(args, QylScope.ForTest(serviceName: "scope-service")); result.Should().HaveCount(1); - result["ServiceName"].GetString().Should().Be("caller-service"); - } - - [Fact] - public void Inject_MapsServiceNameAndSessionId_ToTheirOwnKeys() - { - var args = new Dictionary(StringComparer.OrdinalIgnoreCase); - var scope = QylScope.ForTest(serviceName: "svc-A", sessionId: "sess-B"); - - var result = Injector.Inject(args, scope); - - result.Should().NotBeNull(); - result["serviceName"].GetString().Should().Be("svc-A"); - result["sessionId"].GetString().Should().Be("sess-B"); + Read(result, "ServiceName").Should().Be("caller-service"); } [Fact] - public void Inject_MutatesArgsInPlace_AndReturnsSameReference() + public void Inject_CreatesCaseInsensitiveDict_WhenArgsAreNull() { - var args = new Dictionary(StringComparer.OrdinalIgnoreCase); - var scope = QylScope.ForTest(serviceName: "svc"); - - var result = Injector.Inject(args, scope); + var result = Injector.Inject(arguments: null, QylScope.ForTest(serviceName: "svc")); - result.Should().BeSameAs(args); - args.Should().ContainKey("serviceName"); + result.Should().ContainKey("ServiceName").And.ContainKey("serviceName"); } - [Fact] - public void Inject_NewlyCreatedDict_IsCaseInsensitive() + private static Dictionary Args(params (string key, JsonElement value)[] entries) { - var scope = QylScope.ForTest(serviceName: "svc"); - - var result = Injector.Inject(arguments: null, scope); - - result.Should().ContainKey("ServiceName"); - result.Should().ContainKey("serviceName"); + var dict = new Dictionary(StringComparer.OrdinalIgnoreCase); + foreach (var (key, value) in entries) dict[key] = value; + return dict; } private static JsonElement Str(string value) => JsonSerializer.SerializeToElement(value); private static JsonElement Json(string json) => JsonSerializer.Deserialize(json); - private static IDictionary RequireInjected(IDictionary? result) - { - result.Should().NotBeNull(); - return result ?? throw new InvalidOperationException("Expected qyl scope injection to return arguments."); - } + private static string? Read(IDictionary? args, string key) => + args is not null && args.TryGetValue(key, out var value) && value.ValueKind is JsonValueKind.String + ? value.GetString() + : null; } diff --git a/tests/qyl.mcp.tests/Tools/AnomalyToolsTests.cs b/tests/qyl.mcp.tests/Tools/AnomalyToolsTests.cs index 3ab5037be..d9ad3cf04 100644 --- a/tests/qyl.mcp.tests/Tools/AnomalyToolsTests.cs +++ b/tests/qyl.mcp.tests/Tools/AnomalyToolsTests.cs @@ -1,160 +1,88 @@ using System.Net; -using System.Text; +using ANcpLua.Agents.Testing.Http; using qyl.mcp.Tools; namespace Qyl.Mcp.Tests.Tools; public sealed class AnomalyToolsTests { + private const string BaselineOk = """ + { "metric": "gen_ai.client.token.usage", "hours": 24, "mean": 1, "std_dev": 0, + "p50": 1, "p95": 1, "p99": 1, "sample_count": 1 } + """; + [Fact] public async Task GetMetricBaselineAsync_UsesCollectorServiceNameQueryParameter() { - using var client = CreateClient(static request => - { - request.RequestUri?.PathAndQuery.Should().Contain("metric=gen_ai.client.token.usage"); - request.RequestUri?.PathAndQuery.Should().Contain("serviceName=orders-api"); - request.RequestUri?.PathAndQuery.Should().NotContain("service=orders-api"); - - return JsonResponse(HttpStatusCode.OK, """ - { - "metric": "gen_ai.client.token.usage", - "hours": 24, - "mean": 50, - "std_dev": 0, - "p50": 50, - "p95": 50, - "p99": 50, - "sample_count": 1 - } - """); - }); - var tool = new AnomalyTools(client); + using var handler = new FakeHttpMessageHandler(); + handler.WithResponse("/api/v1/analytics/anomaly/baseline", HttpStatusCode.OK, BaselineOk); + using var client = handler.BuildHttpClient("https://collector.test"); - var output = await tool.GetMetricBaselineAsync( + var output = await new AnomalyTools(client).GetMetricBaselineAsync( "gen_ai.client.token.usage", service: "orders-api", ct: TestContext.Current.CancellationToken); + var url = handler.Requests.Single().Url.PathAndQuery; + url.Should().Contain("metric=gen_ai.client.token.usage"); + url.Should().Contain("serviceName=orders-api"); + url.Should().NotContain("service=orders-api"); output.Should().Contain("# Metric Baseline - gen_ai.client.token.usage"); - output.Should().Contain("Samples: 1"); + output.Should().Contain("Window: 24h, Samples: 1"); } [Fact] - public async Task GetMetricBaselineAsync_ForwardsCancellationToken() + public async Task GetMetricBaselineAsync_ReturnsCancelledMessage_WhenCancelledBeforeRequest() { using var cts = new CancellationTokenSource(); await cts.CancelAsync(); - var sawCancelledRequestToken = false; - using var client = CreateClient((_, cancellationToken) => - { - cancellationToken.IsCancellationRequested.Should().BeTrue(); - sawCancelledRequestToken = true; - - return JsonResponse(HttpStatusCode.OK, """ - { - "metric": "request_count", - "hours": 24, - "mean": 1, - "std_dev": 0, - "p50": 1, - "p95": 1, - "p99": 1, - "sample_count": 1 - } - """); - }); - var tool = new AnomalyTools(client); + using var handler = new FakeHttpMessageHandler(); + handler.WithResponse("/api/v1/analytics/anomaly/baseline", HttpStatusCode.OK, BaselineOk); + using var client = handler.BuildHttpClient("https://collector.test"); - var output = await tool.GetMetricBaselineAsync( - "request_count", - ct: cts.Token); + var output = await new AnomalyTools(client).GetMetricBaselineAsync("request_count", ct: cts.Token); - sawCancelledRequestToken.Should().BeTrue(); output.Should().Be("**Cancelled:** The operation was cancelled."); } - [Fact] - public async Task GetMetricBaselineAsync_ReturnsCollectorValidationMessage() - { - using var client = CreateClient(static _ => JsonResponse( - HttpStatusCode.BadRequest, - """{ "error": "Unknown metric 'missing_metric'. Valid metrics: request_count" }""")); - var tool = new AnomalyTools(client); - - var output = await tool.GetMetricBaselineAsync( - "missing_metric", - ct: TestContext.Current.CancellationToken); - - output.Should().Be( - "Metric baseline query rejected: Unknown metric 'missing_metric'. Valid metrics: request_count"); - } - - [Fact] - public async Task DetectAnomaliesAsync_ReturnsCollectorValidationMessage() - { - using var client = CreateClient(static _ => JsonResponse( - HttpStatusCode.BadRequest, - """{ "error": "Query parameter 'sensitivity' must be greater than zero." }""")); - var tool = new AnomalyTools(client); - - var output = await tool.DetectAnomaliesAsync( - "request_count", - sensitivity: 0, - ct: TestContext.Current.CancellationToken); - - output.Should().Be( - "Anomaly detection query rejected: Query parameter 'sensitivity' must be greater than zero."); - } - - [Fact] - public async Task ComparePeriodsAsync_ReturnsCollectorValidationMessage() - { - using var client = CreateClient(static _ => JsonResponse( - HttpStatusCode.BadRequest, - """{ "error": "period1Start must be earlier than period1End." }""")); - var tool = new AnomalyTools(client); - - var output = await tool.ComparePeriodsAsync( - "request_count", - "2026-05-23T10:00:00Z", - "2026-05-23T09:00:00Z", - "2026-05-22T10:00:00Z", - "2026-05-22T11:00:00Z", - ct: TestContext.Current.CancellationToken); - - output.Should().Be( - "Period comparison query rejected: period1Start must be earlier than period1End."); - } - - private static HttpClient CreateClient(Func send) - { - return CreateClient((request, _) => send(request)); - } - - private static HttpClient CreateClient(Func send) - { - return new HttpClient(new StubHttpMessageHandler(send)) + public static TheoryData>, string, string, string> RejectionCases() => + new() { - BaseAddress = new Uri("https://collector.test") + { + static tools => tools.GetMetricBaselineAsync("missing_metric", ct: TestContext.Current.CancellationToken), + "/api/v1/analytics/anomaly/baseline", + """{ "error": "Unknown metric 'missing_metric'. Valid metrics: request_count" }""", + "Metric baseline query rejected: Unknown metric 'missing_metric'. Valid metrics: request_count" + }, + { + static tools => tools.DetectAnomaliesAsync("request_count", sensitivity: 0, ct: TestContext.Current.CancellationToken), + "/api/v1/analytics/anomaly/anomalies", + """{ "error": "Query parameter 'sensitivity' must be greater than zero." }""", + "Anomaly detection query rejected: Query parameter 'sensitivity' must be greater than zero." + }, + { + static tools => tools.ComparePeriodsAsync( + "request_count", + "2026-05-23T10:00:00Z", "2026-05-23T09:00:00Z", + "2026-05-22T10:00:00Z", "2026-05-22T11:00:00Z", + ct: TestContext.Current.CancellationToken), + "/api/v1/analytics/anomaly/compare", + """{ "error": "period1Start must be earlier than period1End." }""", + "Period comparison query rejected: period1Start must be earlier than period1End." + }, }; - } - private static HttpResponseMessage JsonResponse(HttpStatusCode statusCode, string json) + [Theory] + [MemberData(nameof(RejectionCases))] + public async Task AnomalyTools_FormatsCollectorValidationMessage( + Func> call, string endpoint, string collectorBody, string expected) { - return new HttpResponseMessage(statusCode) - { - Content = new StringContent(json, Encoding.UTF8, "application/json") - }; - } + using var handler = new FakeHttpMessageHandler(); + handler.WithResponse(endpoint, HttpStatusCode.BadRequest, collectorBody); + using var client = handler.BuildHttpClient("https://collector.test"); - private sealed class StubHttpMessageHandler(Func send) - : HttpMessageHandler - { - protected override Task SendAsync( - HttpRequestMessage request, - CancellationToken cancellationToken) - { - return Task.FromResult(send(request, cancellationToken)); - } + var output = await call(new AnomalyTools(client)); + + output.Should().Be(expected); } } diff --git a/tests/qyl.mcp.tests/Tools/CollectorHelperTests.cs b/tests/qyl.mcp.tests/Tools/CollectorHelperTests.cs index 33235a7ac..7fcd8b116 100644 --- a/tests/qyl.mcp.tests/Tools/CollectorHelperTests.cs +++ b/tests/qyl.mcp.tests/Tools/CollectorHelperTests.cs @@ -1,3 +1,4 @@ +using qyl.mcp; using qyl.mcp.Tools; namespace Qyl.Mcp.Tests.Tools; @@ -5,11 +6,27 @@ namespace Qyl.Mcp.Tests.Tools; public sealed class CollectorHelperTests { [Fact] - public async Task ExecuteAsync_FormatsDirectOperationCancellation() - { - var output = await CollectorHelper.ExecuteAsync( - static () => throw new OperationCanceledException()); + public async Task ExecuteAsync_ReturnsOperationResult_WhenNoExceptionThrown() => + (await CollectorHelper.ExecuteAsync(static () => Task.FromResult("ok"))) + .Should().Be("ok"); - output.Should().Be("**Cancelled:** The operation was cancelled."); - } + [Fact] + public async Task ExecuteAsync_FormatsDirectOperationCancellation() => + (await CollectorHelper.ExecuteAsync(static () => throw new OperationCanceledException())) + .Should().Be("**Cancelled:** The operation was cancelled."); + + [Fact] + public async Task ExecuteAsync_FormatsTaskCanceledFromTimeout() => + (await CollectorHelper.ExecuteAsync(static () => throw new TaskCanceledException("hit timeout"))) + .Should().StartWith("**Timeout:**"); + + [Theory] + [InlineData(null, "**Cancelled:**")] + [InlineData("InvestigationBudget", "InvestigationBudget: **Cancelled:**")] + public async Task ExecuteAsync_PrefixesFormattedError_WhenPrefixSupplied(string? prefix, string expectedStart) => + (await CollectorHelper.ExecuteAsync( + static () => throw new OperationCanceledException(), + McpTransportMode.Stdio, + prefix)) + .Should().StartWith(expectedStart); } diff --git a/tests/qyl.mcp.tests/Tools/MetricsToolsTests.cs b/tests/qyl.mcp.tests/Tools/MetricsToolsTests.cs index 8fb38e6e1..1a0d21d01 100644 --- a/tests/qyl.mcp.tests/Tools/MetricsToolsTests.cs +++ b/tests/qyl.mcp.tests/Tools/MetricsToolsTests.cs @@ -1,522 +1,263 @@ using System.Net; -using System.Text; using System.Text.Json; +using ANcpLua.Agents.Testing.Http; using qyl.mcp.Tools.Metrics; namespace Qyl.Mcp.Tests.Tools; public sealed class MetricsToolsTests { - [Fact] - public async Task ListMetrics_FormatsSuccessfulCatalog() - { - using var client = CreateClient(static request => + private const string SuccessMetadataJson = """ { - request.RequestUri?.PathAndQuery.Should().Be("/api/v1/metrics"); - - return JsonResponse(HttpStatusCode.OK, """ - { - "items": [ - { - "name": "request_count", - "type": "sum", - "unit": "{span}", - "label_keys": [ "service.name" ], - "services": [ "orders-api" ], - "services_truncated": false, - "service_limit": 100, - "description": "Count of stored spans per time bucket." - } - ], - "next_cursor": null, - "prev_cursor": null, - "has_more": false - } - """); - }); - var tool = new ListMetricsTool(client); - - var output = await tool.ListMetrics(ct: TestContext.Current.CancellationToken); + "items": [ + { + "name": "gen_ai.client.token.usage", + "type": "histogram", + "description": "Token usage", + "unit": "{token}", + "label_keys": [ "gen_ai.token.type", "service.name" ], + "services": [ "orders-api" ], + "services_truncated": true, + "service_limit": 1 + } + ], + "next_cursor": "cursor-2", + "has_more": true + } + """; - output.Should().Contain("# Available Metrics (1)"); - output.Should().Contain("**Has more:** no"); - output.Should().Contain("| `request_count` | sum | {span} | `service.name` | `orders-api` | Count of stored spans per time bucket. |"); - } + private const string SuccessSeriesJson = """ + { + "metric_name": "gen_ai.client.token.usage", + "series": [ + { + "labels": { "service.name": "orders-api" }, + "points": [ { "timestamp": "2026-05-23T10:00:00.0000000Z", "value": 30 } ] + } + ] + } + """; [Fact] - public async Task ListMetrics_WithFilters_UsesPublicMetricPageContract() + public async Task ListMetrics_GET_v1_metrics() { - using var client = CreateClient(static request => - { - request.RequestUri?.PathAndQuery.Should().Be( - "/api/v1/metrics?serviceName=orders-api&namePattern=token&limit=5&serviceLimit=1&cursor=10"); - - return JsonResponse(HttpStatusCode.OK, """ - { - "items": [ - { - "name": "gen_ai.client.token.usage", - "type": "histogram", - "unit": "{token}", - "label_keys": [ "service.name", "gen_ai.token.type" ], - "services": [ "orders-api" ], - "services_truncated": true, - "service_limit": 1, - "description": "Number of input and output tokens used." - } - ], - "next_cursor": "15", - "prev_cursor": "5", - "has_more": true - } - """); - }); - var tool = new ListMetricsTool(client); - - var output = await tool.ListMetrics( - serviceName: "orders-api", - namePattern: "token", - limit: 5, - serviceLimit: 1, - cursor: "10", - ct: TestContext.Current.CancellationToken); + using var handler = new FakeHttpMessageHandler(); + handler.WithResponse("/api/v1/metrics", HttpStatusCode.OK, SuccessMetadataJson); + using var client = handler.BuildHttpClient("https://collector.test"); + var output = await new ListMetricsTool(client).ListMetrics(ct: TestContext.Current.CancellationToken); + + handler.Requests.Should().ContainSingle().Which.Url.PathAndQuery.Should().Be("/api/v1/metrics"); output.Should().Contain("# Available Metrics (1)"); output.Should().Contain("**Has more:** yes"); - output.Should().Contain("**Next cursor:** `15`"); - output.Should().Contain("**Previous cursor:** `5`"); - output.Should().Contain("`orders-api` ... truncated at 1"); + output.Should().Contain("**Next cursor:** `cursor-2`"); + output.Should().Contain("| `gen_ai.client.token.usage` | histogram | {token} | `gen_ai.token.type`, `service.name` | `orders-api` ... truncated at 1 | Token usage |"); } - [Fact] - public async Task ListMetrics_ReturnsCollectorValidationMessage() + [Theory] + [InlineData("orders-api", null, null, null, null, "/api/v1/metrics?serviceName=orders-api")] + [InlineData(null, "token", 5, 1, "10", "/api/v1/metrics?namePattern=token&limit=5&serviceLimit=1&cursor=10")] + [InlineData("orders-api", "token", 5, 1, "10", "/api/v1/metrics?serviceName=orders-api&namePattern=token&limit=5&serviceLimit=1&cursor=10")] + public async Task ListMetrics_ForwardsFiltersToQueryString( + string? serviceName, string? namePattern, int? limit, int? serviceLimit, string? cursor, string expectedPathAndQuery) { - using var client = CreateClient(static _ => JsonResponse( - HttpStatusCode.BadRequest, - """{ "error": "Project-scoped metrics are not available yet." }""")); - var tool = new ListMetricsTool(client); - - var output = await tool.ListMetrics(ct: TestContext.Current.CancellationToken); - - output.Should().Be("List metrics rejected: Project-scoped metrics are not available yet."); - } + using var handler = new FakeHttpMessageHandler(); + handler.WithResponse("/api/v1/metrics", HttpStatusCode.OK, """{ "items": [], "has_more": false }"""); + using var client = handler.BuildHttpClient("https://collector.test"); - [Fact] - public async Task QueryMetrics_FormatsSuccessfulSeries() - { - using var client = CreateClient(static async (request, ct) => - { - request.Method.Should().Be(HttpMethod.Post); - request.RequestUri?.PathAndQuery.Should().Be("/api/v1/metrics/query"); - - if (request.Content is null) - return JsonResponse(HttpStatusCode.BadRequest, """{ "error": "missing body" }"""); - - var json = await request.Content.ReadAsStringAsync(ct).ConfigureAwait(false); - using var document = JsonDocument.Parse(json); - var root = document.RootElement; - root.GetProperty("metric_name").GetString().Should().Be("gen_ai.client.token.usage"); - root.GetProperty("filters").GetProperty("service.name").GetString().Should().Be("orders-api"); - root.GetProperty("filters").GetProperty("gen_ai.token.type").GetString().Should().Be("input"); - root.GetProperty("start_time").GetString().Should().Be("2026-05-23T10:00:00Z"); - root.GetProperty("end_time").GetString().Should().Be("2026-05-23T11:00:00Z"); - root.GetProperty("step").GetString().Should().Be("1h"); - - return JsonResponse(HttpStatusCode.OK, """ - { - "metric_name": "gen_ai.client.token.usage", - "series": [ - { - "labels": { - "service.name": "orders-api", - "gen_ai.token.type": "input" - }, - "points": [ - { "timestamp": "2026-05-23T10:00:00.0000000Z", "value": 30 } - ] - } - ] - } - """); - }); - var tool = new QueryMetricsTool(client); - - var output = await tool.QueryMetrics( - "gen_ai.client.token.usage", - filter: "service.name=orders-api", - from: "2026-05-23T10:00:00Z", - to: "2026-05-23T11:00:00Z", - interval: "1h", - tokenType: "input", + await new ListMetricsTool(client).ListMetrics( + serviceName: serviceName, namePattern: namePattern, limit: limit, serviceLimit: serviceLimit, cursor: cursor, ct: TestContext.Current.CancellationToken); - output.Should().Contain("# Metric: `gen_ai.client.token.usage`"); - output.Should().Contain("**Series:** 1"); - output.Should().Contain("## Series: `service.name=orders-api`, `gen_ai.token.type=input`"); - output.Should().Contain("| 2026-05-23T10:00:00.0000000Z | 30 |"); + handler.Requests.Single().Url.PathAndQuery.Should().Be(expectedPathAndQuery); } [Fact] - public async Task QueryMetrics_WithGroupBy_UsesPublicMetricQueryContract() + public async Task QueryMetrics_POST_v1_metrics_query() { - using var client = CreateClient(static async (request, ct) => - { - request.Method.Should().Be(HttpMethod.Post); - request.RequestUri?.PathAndQuery.Should().Be("/api/v1/metrics/query"); - - if (request.Content is null) - return JsonResponse(HttpStatusCode.BadRequest, """{ "error": "missing body" }"""); - - var json = await request.Content.ReadAsStringAsync(ct).ConfigureAwait(false); - using var document = JsonDocument.Parse(json); - var root = document.RootElement; - root.GetProperty("metric_name").GetString().Should().Be("gen_ai.client.token.usage"); - root.GetProperty("filters").GetProperty("service.name").GetString().Should().Be("orders-api"); - root.GetProperty("filters").GetProperty("gen_ai.token.type").GetString().Should().Be("input"); - root.GetProperty("start_time").GetString().Should().Be("2026-05-23T10:00:00Z"); - root.GetProperty("end_time").GetString().Should().Be("2026-05-23T11:00:00Z"); - root.GetProperty("step").GetString().Should().Be("1h"); - - var groupBy = root.GetProperty("group_by").EnumerateArray(); - groupBy.MoveNext().Should().BeTrue(); - groupBy.Current.GetString().Should().Be("service.name"); - groupBy.MoveNext().Should().BeTrue(); - groupBy.Current.GetString().Should().Be("gen_ai.token.type"); - groupBy.MoveNext().Should().BeFalse(); - - return JsonResponse(HttpStatusCode.OK, """ - { - "metric_name": "gen_ai.client.token.usage", - "series": [ - { - "labels": { - "service.name": "orders-api", - "gen_ai.token.type": "input" - }, - "points": [ - { "timestamp": "2026-05-23T10:00:00.0000000Z", "value": 30 } - ] - } - ] - } - """); - }); - var tool = new QueryMetricsTool(client); - - var output = await tool.QueryMetrics( + using var handler = new FakeHttpMessageHandler(); + handler.WithResponse("/api/v1/metrics/query", HttpStatusCode.OK, SuccessSeriesJson); + using var client = handler.BuildHttpClient("https://collector.test"); + + var output = await new QueryMetricsTool(client).QueryMetrics( "gen_ai.client.token.usage", - filter: "service.name=orders-api", - from: "2026-05-23T10:00:00Z", - to: "2026-05-23T11:00:00Z", - interval: "1h", - tokenType: "input", - groupBy: "service.name, gen_ai.token.type", + from: "2026-05-23T10:00:00Z", to: "2026-05-23T11:00:00Z", ct: TestContext.Current.CancellationToken); + var request = handler.Requests.Should().ContainSingle().Subject; + request.Method.Should().Be(HttpMethod.Post); + request.Url.PathAndQuery.Should().Be("/api/v1/metrics/query"); output.Should().Contain("# Metric: `gen_ai.client.token.usage`"); output.Should().Contain("**Series:** 1"); - output.Should().Contain("## Series: `service.name=orders-api`, `gen_ai.token.type=input`"); + output.Should().Contain("## Series: `service.name=orders-api`"); output.Should().Contain("| 2026-05-23T10:00:00.0000000Z | 30 |"); } [Fact] - public async Task QueryMetrics_WithProviderAndRequestModel_UsesPublicMetricQueryContract() + public async Task QueryMetrics_SendsCanonicalPayloadShape() { - using var client = CreateClient(static async (request, ct) => + using var handler = new FakeHttpMessageHandler(); + handler.WithRequestValidator(static req => AssertJsonBody(req, static body => { - request.Method.Should().Be(HttpMethod.Post); - request.RequestUri?.PathAndQuery.Should().Be("/api/v1/metrics/query"); - - if (request.Content is null) - return JsonResponse(HttpStatusCode.BadRequest, """{ "error": "missing body" }"""); - - var json = await request.Content.ReadAsStringAsync(ct).ConfigureAwait(false); - using var document = JsonDocument.Parse(json); - var root = document.RootElement; - root.GetProperty("metric_name").GetString().Should().Be("gen_ai.client.cost"); - root.GetProperty("filters").GetProperty("service.name").GetString().Should().Be("orders-api"); - root.GetProperty("filters").GetProperty("gen_ai.provider.name").GetString().Should().Be("openai"); - root.GetProperty("filters").GetProperty("gen_ai.request.model").GetString().Should().Be("gpt-5.5"); - root.GetProperty("start_time").GetString().Should().Be("2026-05-23T10:00:00Z"); - root.GetProperty("end_time").GetString().Should().Be("2026-05-23T11:00:00Z"); - root.GetProperty("step").GetString().Should().Be("1h"); - - return JsonResponse(HttpStatusCode.OK, """ - { - "metric_name": "gen_ai.client.cost", - "series": [ - { - "labels": { - "service.name": "orders-api", - "gen_ai.provider.name": "openai", - "gen_ai.request.model": "gpt-5.5" - }, - "points": [ - { "timestamp": "2026-05-23T10:00:00.0000000Z", "value": 0.0025 } - ] - } - ] - } - """); - }); - var tool = new QueryMetricsTool(client); - - var output = await tool.QueryMetrics( - "gen_ai.client.cost", + body.GetProperty("metric_name").GetString().Should().Be("gen_ai.client.token.usage"); + body.GetProperty("filters").GetProperty("service.name").GetString().Should().Be("orders-api"); + body.GetProperty("filters").GetProperty("gen_ai.token.type").GetString().Should().Be("input"); + body.GetProperty("start_time").GetString().Should().Be("2026-05-23T10:00:00Z"); + body.GetProperty("end_time").GetString().Should().Be("2026-05-23T11:00:00Z"); + body.GetProperty("step").GetString().Should().Be("1h"); + })); + handler.WithResponse("/api/v1/metrics/query", HttpStatusCode.OK, SuccessSeriesJson); + using var client = handler.BuildHttpClient("https://collector.test"); + + await new QueryMetricsTool(client).QueryMetrics( + "gen_ai.client.token.usage", filter: "service.name=orders-api", - from: "2026-05-23T10:00:00Z", - to: "2026-05-23T11:00:00Z", - interval: "1h", - providerName: "openai", - requestModel: "gpt-5.5", + from: "2026-05-23T10:00:00Z", to: "2026-05-23T11:00:00Z", + interval: "1h", tokenType: "input", ct: TestContext.Current.CancellationToken); - - output.Should().Contain("# Metric: `gen_ai.client.cost`"); - output.Should().Contain("**Series:** 1"); - output.Should().Contain("## Series: `service.name=orders-api`, `gen_ai.provider.name=openai`, `gen_ai.request.model=gpt-5.5`"); - output.Should().Contain("| 2026-05-23T10:00:00.0000000Z | 0.0025 |"); } [Fact] - public async Task QueryMetrics_WithSeriesLimit_UsesPublicMetricQueryContractAndReportsTruncation() + public async Task QueryMetrics_ForwardsGroupByLabels() { - using var client = CreateClient(static async (request, ct) => + using var handler = new FakeHttpMessageHandler(); + handler.WithRequestValidator(static req => AssertJsonBody(req, static body => { - request.Method.Should().Be(HttpMethod.Post); - request.RequestUri?.PathAndQuery.Should().Be("/api/v1/metrics/query"); - - if (request.Content is null) - return JsonResponse(HttpStatusCode.BadRequest, """{ "error": "missing body" }"""); - - var json = await request.Content.ReadAsStringAsync(ct).ConfigureAwait(false); - using var document = JsonDocument.Parse(json); - var root = document.RootElement; - root.GetProperty("metric_name").GetString().Should().Be("request_count"); - root.GetProperty("series_limit").GetInt32().Should().Be(1); - root.GetProperty("start_time").GetString().Should().Be("2026-05-23T10:00:00Z"); - root.GetProperty("end_time").GetString().Should().Be("2026-05-23T11:00:00Z"); - - return JsonResponse(HttpStatusCode.OK, """ - { - "metric_name": "request_count", - "series_truncated": true, - "series_limit": 1, - "series": [ - { - "labels": { "service.name": "orders-api" }, - "points": [ - { "timestamp": "2026-05-23T10:00:00.0000000Z", "value": 7 } - ] - } - ] - } - """); - }); - var tool = new QueryMetricsTool(client); - - var output = await tool.QueryMetrics( - "request_count", - from: "2026-05-23T10:00:00Z", - to: "2026-05-23T11:00:00Z", - seriesLimit: 1, - ct: TestContext.Current.CancellationToken); + var groupBy = body.GetProperty("group_by").EnumerateArray() + .Select(static item => item.GetString()); - output.Should().Contain("# Metric: `request_count`"); - output.Should().Contain("**Series:** 1"); - output.Should().Contain("**Series limit:** 1 (truncated)"); - output.Should().Contain("## Series: `service.name=orders-api`"); - } + groupBy.Should().Equal("service.name", "gen_ai.token.type"); + })); + handler.WithResponse("/api/v1/metrics/query", HttpStatusCode.OK, SuccessSeriesJson); + using var client = handler.BuildHttpClient("https://collector.test"); - [Fact] - public async Task QueryMetrics_WithPointLimit_UsesPublicMetricQueryContractAndReportsTruncation() - { - using var client = CreateClient(static async (request, ct) => - { - request.Method.Should().Be(HttpMethod.Post); - request.RequestUri?.PathAndQuery.Should().Be("/api/v1/metrics/query"); - - if (request.Content is null) - return JsonResponse(HttpStatusCode.BadRequest, """{ "error": "missing body" }"""); - - var json = await request.Content.ReadAsStringAsync(ct).ConfigureAwait(false); - using var document = JsonDocument.Parse(json); - var root = document.RootElement; - root.GetProperty("metric_name").GetString().Should().Be("request_count"); - root.GetProperty("point_limit").GetInt32().Should().Be(2); - root.GetProperty("start_time").GetString().Should().Be("2026-05-23T10:00:00Z"); - root.GetProperty("end_time").GetString().Should().Be("2026-05-23T11:00:00Z"); - - return JsonResponse(HttpStatusCode.OK, """ - { - "metric_name": "request_count", - "points_truncated": true, - "point_limit": 2, - "series": [ - { - "labels": { "service.name": "orders-api" }, - "points": [ - { "timestamp": "2026-05-23T10:00:00.0000000Z", "value": 7 }, - { "timestamp": "2026-05-23T10:01:00.0000000Z", "value": 3 } - ] - } - ] - } - """); - }); - var tool = new QueryMetricsTool(client); - - var output = await tool.QueryMetrics( - "request_count", - from: "2026-05-23T10:00:00Z", - to: "2026-05-23T11:00:00Z", - pointLimit: 2, + await new QueryMetricsTool(client).QueryMetrics( + "gen_ai.client.token.usage", + filter: "service.name=orders-api", + from: "2026-05-23T10:00:00Z", to: "2026-05-23T11:00:00Z", + tokenType: "input", groupBy: "service.name, gen_ai.token.type", ct: TestContext.Current.CancellationToken); - - output.Should().Contain("# Metric: `request_count`"); - output.Should().Contain("**Point limit:** 2 (truncated)"); - output.Should().Contain("| 2026-05-23T10:01:00.0000000Z | 3 |"); } [Fact] - public async Task QueryMetrics_WithGenAiLabelFilter_UsesPublicMetricQueryContract() + public async Task QueryMetrics_ForwardsProviderModelAndLimits_AndReportsTruncation() { - var now = new DateTimeOffset(2026, 5, 23, 12, 0, 0, TimeSpan.Zero); - using var client = CreateClient(static async (request, ct) => + using var handler = new FakeHttpMessageHandler(); + handler.WithRequestValidator(static req => AssertJsonBody(req, static body => { - request.Method.Should().Be(HttpMethod.Post); - request.RequestUri?.PathAndQuery.Should().Be("/api/v1/metrics/query"); - - if (request.Content is null) - return JsonResponse(HttpStatusCode.BadRequest, """{ "error": "missing body" }"""); - - var json = await request.Content.ReadAsStringAsync(ct).ConfigureAwait(false); - using var document = JsonDocument.Parse(json); - var root = document.RootElement; - root.GetProperty("metric_name").GetString().Should().Be("gen_ai.client.cost"); - root.GetProperty("filters").GetProperty("gen_ai.provider.name").GetString().Should().Be("openai"); - root.GetProperty("start_time").GetString().Should().Be("2026-05-22T12:00:00.0000000+00:00"); - root.GetProperty("end_time").GetString().Should().Be("2026-05-23T12:00:00.0000000+00:00"); - - return JsonResponse(HttpStatusCode.OK, """ - { - "metric_name": "gen_ai.client.cost", - "series": [ - { - "labels": { "gen_ai.provider.name": "openai" }, - "points": [ - { "timestamp": "2026-05-23T10:00:00.0000000Z", "value": 0.0025 } - ] - } - ] - } - """); - }); - var tool = new QueryMetricsTool(client, new FixedTimeProvider(now)); - - var output = await tool.QueryMetrics( + var filters = body.GetProperty("filters"); + filters.GetProperty("service.name").GetString().Should().Be("orders-api"); + filters.GetProperty("gen_ai.provider.name").GetString().Should().Be("openai"); + filters.GetProperty("gen_ai.request.model").GetString().Should().Be("gpt-4o-mini"); + body.GetProperty("series_limit").GetInt32().Should().Be(1); + body.GetProperty("point_limit").GetInt32().Should().Be(2); + })); + handler.WithResponse("/api/v1/metrics/query", HttpStatusCode.OK, """ + { + "metric_name": "gen_ai.client.cost", + "series_truncated": true, + "series_limit": 1, + "points_truncated": true, + "point_limit": 2, + "series": [ + { + "labels": { + "service.name": "orders-api", + "gen_ai.provider.name": "openai", + "gen_ai.request.model": "gpt-4o-mini" + }, + "points": [ + { "timestamp": "2026-05-23T10:00:00.0000000Z", "value": 0.0025 }, + { "timestamp": "2026-05-23T10:01:00.0000000Z", "value": 0.0030 } + ] + } + ] + } + """); + using var client = handler.BuildHttpClient("https://collector.test"); + + var output = await new QueryMetricsTool(client).QueryMetrics( "gen_ai.client.cost", - filter: "gen_ai.provider.name=openai", + filter: "service.name=orders-api", + from: "2026-05-23T10:00:00Z", to: "2026-05-23T11:00:00Z", + providerName: "openai", requestModel: "gpt-4o-mini", + seriesLimit: 1, pointLimit: 2, ct: TestContext.Current.CancellationToken); - output.Should().Contain("# Metric: `gen_ai.client.cost`"); - output.Should().Contain("## Series: `gen_ai.provider.name=openai`"); + output.Should().Contain("**Series limit:** 1 (truncated)"); + output.Should().Contain("**Point limit:** 2 (truncated)"); + output.Should().Contain("`gen_ai.provider.name=openai`"); + output.Should().Contain("`gen_ai.request.model=gpt-4o-mini`"); } [Fact] - public async Task QueryMetrics_RejectsProviderParameterThatDuplicatesFilterLabel() + public async Task QueryMetrics_RejectsProviderDuplicatingFilterLabel_WithoutCallingCollector() { - using var client = CreateClient(static _ => throw new InvalidOperationException("collector should not be called")); - var tool = new QueryMetricsTool(client); + using var handler = new FakeHttpMessageHandler(); + using var client = handler.BuildHttpClient("https://collector.test"); - var output = await tool.QueryMetrics( + var output = await new QueryMetricsTool(client).QueryMetrics( "gen_ai.client.cost", filter: "gen_ai.provider.name=anthropic", providerName: "openai", ct: TestContext.Current.CancellationToken); - output.Should().Be( - "Metric query rejected: Query parameter 'providerName' duplicates filter label gen_ai.provider.name."); + output.Should().Contain("duplicates filter label gen_ai.provider.name"); + handler.Requests.Should().BeEmpty(); } [Fact] - public async Task QueryMetrics_RejectsEmptyGroupByBeforeCallingCollector() + public async Task QueryMetrics_RejectsEmptyGroupBy_WithoutCallingCollector() { - using var client = CreateClient(static _ => throw new InvalidOperationException("collector should not be called")); - var tool = new QueryMetricsTool(client); + using var handler = new FakeHttpMessageHandler(); + using var client = handler.BuildHttpClient("https://collector.test"); - var output = await tool.QueryMetrics( + var output = await new QueryMetricsTool(client).QueryMetrics( "request_count", groupBy: ",", ct: TestContext.Current.CancellationToken); - output.Should().Be("Metric query rejected: Query parameter 'groupBy' must include at least one label."); + output.Should().Contain("must include at least one label"); + handler.Requests.Should().BeEmpty(); } [Fact] - public async Task QueryMetrics_ReturnsCollectorValidationMessage() + public async Task ListMetrics_FormatsCollector400_AsRejection() { - using var client = CreateClient(static _ => JsonResponse( - HttpStatusCode.BadRequest, - """{ "error": "Query parameter 'filter' supports service.name= only." }""")); - var tool = new QueryMetricsTool(client); + using var handler = new FakeHttpMessageHandler(); + handler.WithResponse( + "/api/v1/metrics", HttpStatusCode.BadRequest, + """{ "error": "Project-scoped metrics are not available yet." }"""); + using var client = handler.BuildHttpClient("https://collector.test"); - var output = await tool.QueryMetrics( - "request_count", - filter: "project=demo", - ct: TestContext.Current.CancellationToken); + var output = await new ListMetricsTool(client).ListMetrics(ct: TestContext.Current.CancellationToken); - output.Should().Be("Metric query rejected: Query parameter 'filter' supports service.name= only."); + output.Should().Be("List metrics rejected: Project-scoped metrics are not available yet."); } - [Fact] - public async Task QueryMetrics_ReturnsUnknownMetricMessage() + [Theory] + [InlineData(HttpStatusCode.BadRequest, """{ "error": "Query parameter 'filter' supports service.name= only." }""", "Metric query rejected: Query parameter 'filter' supports service.name= only.")] + [InlineData(HttpStatusCode.NotFound, """{ "error": "Unknown metric 'request_count'." }""", "Metric `request_count` was not found. Unknown metric 'request_count'.")] + public async Task QueryMetrics_FormatsCollectorError(HttpStatusCode status, string body, string expected) { - using var client = CreateClient(static _ => JsonResponse( - HttpStatusCode.NotFound, - """{ "error": "Unknown metric 'missing_metric'." }""")); - var tool = new QueryMetricsTool(client); + using var handler = new FakeHttpMessageHandler(); + handler.WithResponse("/api/v1/metrics/query", status, body); + using var client = handler.BuildHttpClient("https://collector.test"); - var output = await tool.QueryMetrics( - "missing_metric", + var output = await new QueryMetricsTool(client).QueryMetrics( + "request_count", from: "2026-05-23T10:00:00Z", to: "2026-05-23T11:00:00Z", ct: TestContext.Current.CancellationToken); - output.Should().Be("Metric `missing_metric` was not found. Unknown metric 'missing_metric'."); + output.Should().Be(expected); } - private static HttpClient CreateClient(Func send) + private static void AssertJsonBody(HttpRequestMessage request, Action assert) { - return CreateClient((request, _) => Task.FromResult(send(request))); - } + if (request.Content is null) + throw new InvalidOperationException("Expected request body."); - private static HttpClient CreateClient(Func> send) - { - return new HttpClient(new StubHttpMessageHandler(send)) - { - BaseAddress = new Uri("https://collector.test") - }; - } - - private static HttpResponseMessage JsonResponse(HttpStatusCode statusCode, string json) - { - return new HttpResponseMessage(statusCode) - { - Content = new StringContent(json, Encoding.UTF8, "application/json") - }; - } - - private sealed class FixedTimeProvider(DateTimeOffset now) : TimeProvider - { - public override DateTimeOffset GetUtcNow() => now; - } - - private sealed class StubHttpMessageHandler(Func> send) - : HttpMessageHandler - { - protected override Task SendAsync( - HttpRequestMessage request, - CancellationToken cancellationToken) - { - return send(request, cancellationToken); - } + using var reader = new StreamReader(request.Content.ReadAsStream()); + using var document = JsonDocument.Parse(reader.ReadToEnd()); + assert(document.RootElement); } } diff --git a/tests/qyl.mcp.tests/Tools/SummaryCredentialRedactorTests.cs b/tests/qyl.mcp.tests/Tools/SummaryCredentialRedactorTests.cs index de50bc782..844866d54 100644 --- a/tests/qyl.mcp.tests/Tools/SummaryCredentialRedactorTests.cs +++ b/tests/qyl.mcp.tests/Tools/SummaryCredentialRedactorTests.cs @@ -4,98 +4,65 @@ namespace Qyl.Mcp.Tests.Tools; public sealed class SummaryCredentialRedactorTests { - [Fact] - public void Redact_RemovesForgejoAndHttpCredentials() + [Theory] + [InlineData("Authorization: Bearer abc.def-123", "abc.def-123")] + [InlineData("Authorization: Basic dXNlcjpwYXNz", "dXNlcjpwYXNz")] + [InlineData("export FORGEJO_API_TOKEN=\"secret-token\"", "secret-token")] + [InlineData("export FORGEJO_RUNNER_TOKEN=\"runner-env-token\"", "runner-env-token")] + [InlineData("export FORGEJO_RUNNER_SECRET='runner-env-secret'", "runner-env-secret")] + [InlineData("INPUT_TOKEN=github-action-input-token", "github-action-input-token")] + [InlineData("curl --user root:admin1234 https://example.test", "root:admin1234")] + [InlineData("curl -u start:secret https://example.test", "start:secret")] + [InlineData("curl -u \"alice:password\" https://example.test", "alice:password")] + [InlineData("curl --user=start-option:secret https://example.test", "start-option:secret")] + [InlineData("https://root:admin1234@example.test/root/repo", "root:admin1234")] + [InlineData("forgejo actions register --secret \"shared-secret-value\"", "shared-secret-value")] + [InlineData("forgejo-runner register --token runner-registration-token", "runner-registration-token")] + [InlineData("forgejo admin user create --password admin-password", "admin-password")] + [InlineData("forgejo dump-repo --auth_token \"cli-personal-token\"", "cli-personal-token")] + [InlineData("forgejo dump-repo --auth_password=cli-password", "cli-password")] + [InlineData("forgejo dump-repo --auth_username cli-user", "cli-user")] + [InlineData("GET /api/v1/repos/a/b/actions/runners?token=abc123&other=1", "abc123")] + [InlineData("GET /api/v1/repos/migrate?auth_token=abc456&other=1", "abc456")] + [InlineData("PUT /api/v1/repos/a/b/actions/secrets/MY_SECRET {\"data\":\"repo-secret-value\"}", "repo-secret-value")] + [InlineData("{\"token\":\"runner-secret\"}", "runner-secret")] + [InlineData("{\"access_token\":\"api-secret\"}", "api-secret")] + [InlineData("{\"auth_token\":\"migrate-token\"}", "migrate-token")] + [InlineData("{\"auth_password\":\"migrate-password\"}", "migrate-password")] + [InlineData("{\"auth_username\":\"migrate-user\"}", "migrate-user")] + [InlineData("{\"authorization_header\":\"Bearer webhook-secret\"}", "webhook-secret")] + [InlineData("{\"client_secret\":\"oauth-client-secret\"}", "oauth-client-secret")] + [InlineData("{\"remote_password\":\"mirror-password\"}", "mirror-password")] + [InlineData("{\"password\":\"user-password\"}", "user-password")] + [InlineData("X-Forgejo-OTP: 123456", "123456")] + [InlineData("X-Gitea-OTP: 123456", "123456")] + public void Redact_StripsSecretFromInput(string input, string secret) { - var syntheticRunnerToken = new string('a', 40); - var quotedSyntheticRunnerToken = new string('b', 40); - var configSyntheticRunnerToken = new string('c', 40); - var literalSyntheticRunnerToken = new string('d', 40); - var uppercaseConfigSyntheticRunnerToken = new string('e', 40); - const string nonHexRunnerToken = "Sk9wHjBHelH4n1ckQy-mo3KVYRdoaPZ_aaH1ATfgI05"; - var input = $$""" - -u start:secret - Authorization: Bearer abc.def-123 - Authorization: Basic dXNlcjpwYXNz - export FORGEJO_API_TOKEN="secret-token" - export FORGEJO_RUNNER_TOKEN="runner-env-token" - export FORGEJO_RUNNER_SECRET='runner-env-secret' - INPUT_TOKEN=github-action-input-token - -u start:secret - --user=start-option:secret - curl --user root:admin1234 https://example.test - curl -u "alice:password" https://example.test - forgejo actions register --secret "shared-secret-value" - forgejo-runner register --token runner-registration-token - forgejo admin user create --username root --password admin-password --email root@example.test - forgejo dump-repo --auth_token "cli-personal-token" --auth_password=cli-password --auth_username cli-user - https://root:admin1234@example.test/root/repo - GET /api/v1/repos/a/b/actions/runners?token=abc123&other=1 - GET /api/v1/repos/migrate?auth_token=abc456&other=1 - PUT /api/v1/repos/a/b/actions/secrets/MY_SECRET {"data":"repo-secret-value"} - {"token":"runner-secret","access_token":"api-secret"} - {"auth_token":"migrate-token","auth_password":"migrate-password","auth_username":"migrate-user"} - {"authorization_header":"Bearer webhook-secret","client_secret":"oauth-client-secret","remote_password":"mirror-password","password":"user-password"} - TOKEN: {{syntheticRunnerToken}} - Token: "{{quotedSyntheticRunnerToken}}" - token: {{syntheticRunnerToken}} - token: "{{quotedSyntheticRunnerToken}}" - token = "{{configSyntheticRunnerToken}}"; - TOKEN: "{{uppercaseConfigSyntheticRunnerToken}}" - kubectl create secret generic forgejo-registration --from-literal=token={{literalSyntheticRunnerToken}} - token: "{{nonHexRunnerToken}}" - X-Forgejo-OTP: 123456 - X-Gitea-OTP: 123456 - """; - var redacted = SummaryCredentialRedactor.Redact(input); - Assert.DoesNotContain("abc.def-123", redacted, StringComparison.Ordinal); - Assert.DoesNotContain("start:secret", redacted, StringComparison.Ordinal); - Assert.DoesNotContain("dXNlcjpwYXNz", redacted, StringComparison.Ordinal); - Assert.DoesNotContain("secret-token", redacted, StringComparison.Ordinal); - Assert.DoesNotContain("runner-env-token", redacted, StringComparison.Ordinal); - Assert.DoesNotContain("runner-env-secret", redacted, StringComparison.Ordinal); - Assert.DoesNotContain("github-action-input-token", redacted, StringComparison.Ordinal); - Assert.DoesNotContain("root:admin1234", redacted, StringComparison.Ordinal); - Assert.DoesNotContain("start:secret", redacted, StringComparison.Ordinal); - Assert.DoesNotContain("start-option:secret", redacted, StringComparison.Ordinal); - Assert.DoesNotContain("alice:password", redacted, StringComparison.Ordinal); - Assert.DoesNotContain("shared-secret-value", redacted, StringComparison.Ordinal); - Assert.DoesNotContain("runner-registration-token", redacted, StringComparison.Ordinal); - Assert.DoesNotContain("admin-password", redacted, StringComparison.Ordinal); - Assert.DoesNotContain("abc123", redacted, StringComparison.Ordinal); - Assert.DoesNotContain("abc456", redacted, StringComparison.Ordinal); - Assert.DoesNotContain("repo-secret-value", redacted, StringComparison.Ordinal); - Assert.DoesNotContain("runner-secret", redacted, StringComparison.Ordinal); - Assert.DoesNotContain("api-secret", redacted, StringComparison.Ordinal); - Assert.DoesNotContain("migrate-token", redacted, StringComparison.Ordinal); - Assert.DoesNotContain("migrate-password", redacted, StringComparison.Ordinal); - Assert.DoesNotContain("migrate-user", redacted, StringComparison.Ordinal); - Assert.DoesNotContain("cli-personal-token", redacted, StringComparison.Ordinal); - Assert.DoesNotContain("cli-password", redacted, StringComparison.Ordinal); - Assert.DoesNotContain("cli-user", redacted, StringComparison.Ordinal); - Assert.DoesNotContain("webhook-secret", redacted, StringComparison.Ordinal); - Assert.DoesNotContain("oauth-client-secret", redacted, StringComparison.Ordinal); - Assert.DoesNotContain("mirror-password", redacted, StringComparison.Ordinal); - Assert.DoesNotContain("user-password", redacted, StringComparison.Ordinal); - Assert.DoesNotContain(syntheticRunnerToken, redacted, StringComparison.Ordinal); - Assert.DoesNotContain(quotedSyntheticRunnerToken, redacted, StringComparison.Ordinal); - Assert.DoesNotContain(configSyntheticRunnerToken, redacted, StringComparison.Ordinal); - Assert.DoesNotContain(uppercaseConfigSyntheticRunnerToken, redacted, StringComparison.Ordinal); - Assert.DoesNotContain(literalSyntheticRunnerToken, redacted, StringComparison.Ordinal); - Assert.DoesNotContain(nonHexRunnerToken, redacted, StringComparison.Ordinal); - Assert.DoesNotContain("123456", redacted, StringComparison.Ordinal); - Assert.Contains("", redacted, StringComparison.Ordinal); + redacted.Should().NotContain(secret); + redacted.Should().Contain(""); } - [Fact] - public void Redact_KeepsNonCredentialSummaryText() + [Theory] + [InlineData("TOKEN: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa")] + [InlineData("Token: \"bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb\"")] + [InlineData("token = \"cccccccccccccccccccccccccccccccccccccccc\";")] + [InlineData("token: \"Sk9wHjBHelH4n1ckQy-mo3KVYRdoaPZ_aaH1ATfgI05\"")] + [InlineData("--from-literal=token=dddddddddddddddddddddddddddddddddddddddd")] + public void Redact_StripsBareRunnerTokenLiterals(string input) { - const string input = "Trace ID: xyz789\nSpan Count: 3\nGET /api/v1/repos/owner/repo/actions/runners"; - var redacted = SummaryCredentialRedactor.Redact(input); - Assert.Equal(input, redacted); + redacted.Should().Contain(""); + redacted.Should().NotContain("aaaa").And.NotContain("bbbb").And.NotContain("cccc") + .And.NotContain("dddd").And.NotContain("Sk9wHjBH"); } + + [Theory] + [InlineData("Trace ID: xyz789")] + [InlineData("Span Count: 3")] + [InlineData("GET /api/v1/repos/owner/repo/actions/runners")] + public void Redact_KeepsNonCredentialSummaryText(string input) => + SummaryCredentialRedactor.Redact(input).Should().Be(input); } diff --git a/tests/qyl.mcp.tests/qyl.mcp.tests.csproj b/tests/qyl.mcp.tests/qyl.mcp.tests.csproj index c25e4db59..458d21c26 100644 --- a/tests/qyl.mcp.tests/qyl.mcp.tests.csproj +++ b/tests/qyl.mcp.tests/qyl.mcp.tests.csproj @@ -8,6 +8,7 @@ + diff --git a/tests/qyl.opentelemetry.extensions.tests/QylOpenTelemetryServiceCollectionExtensionsTests.cs b/tests/qyl.opentelemetry.extensions.tests/QylOpenTelemetryServiceCollectionExtensionsTests.cs index a9af99d83..c1316a100 100644 --- a/tests/qyl.opentelemetry.extensions.tests/QylOpenTelemetryServiceCollectionExtensionsTests.cs +++ b/tests/qyl.opentelemetry.extensions.tests/QylOpenTelemetryServiceCollectionExtensionsTests.cs @@ -3,7 +3,6 @@ using Microsoft.Extensions.Hosting; using OpenTelemetry; using OpenTelemetry.Metrics; -using Xunit; namespace Qyl.OpenTelemetry.Extensions.Tests; @@ -11,77 +10,43 @@ public sealed class QylOpenTelemetryServiceCollectionExtensionsTests { private static readonly Uri s_traceEndpoint = new("http://localhost:4318/v1/traces"); - [Fact] - public void AddQylOpenTelemetry_Allows_Metrics_Pipeline_With_Meter_Name_And_Callback() - { - var services = new ServiceCollection(); - var metricsConfigured = false; - - services.AddQylOpenTelemetry(o => - { - o.Endpoint = s_traceEndpoint; - o.ServiceName = "orders-api"; - o.MeterNames.Add("orders-api"); - o.ConfigureMetrics = _ => metricsConfigured = true; - }); - - Assert.True(metricsConfigured); - Assert.NotEmpty(services); - } - - [Fact] - public void AddQylOpenTelemetry_Allows_Metrics_Only_Without_Trace_Endpoint() + public static TheoryData> HappyPathConfigurations() => new() { - var services = new ServiceCollection(); - - services.AddQylOpenTelemetry(static o => - { - o.EnableTracing = false; - o.EnableMetrics = true; - o.ServiceName = "orders-api"; - }); - - Assert.NotEmpty(services); - } + static o => { o.Endpoint = s_traceEndpoint; o.ServiceName = "orders-api"; o.MeterNames.Add("orders-api"); }, + static o => { o.EnableTracing = false; o.EnableMetrics = true; o.ServiceName = "orders-api"; }, + static o => { o.EnableTracing = false; o.ServiceName = "orders-api"; o.MeterNames.Add("orders-api"); }, + static o => { o.EnableTracing = false; o.ServiceName = "orders-api"; o.ConfigureMetrics = static m => m.AddMeter("orders-api"); }, + }; - [Fact] - public void AddQylOpenTelemetry_Allows_MeterNames_Without_Trace_Endpoint() + [Theory] + [MemberData(nameof(HappyPathConfigurations))] + public void AddQylOpenTelemetry_RegistersServices_ForValidConfigurations(Action configure) { var services = new ServiceCollection(); - services.AddQylOpenTelemetry(static o => - { - o.EnableTracing = false; - o.ServiceName = "orders-api"; - o.MeterNames.Add("orders-api"); - }); + services.AddQylOpenTelemetry(configure); - Assert.NotEmpty(services); + services.Should().NotBeEmpty(); } [Fact] - public void AddQylOpenTelemetry_Allows_ConfigureMetrics_Without_Qyl_Metric_Exporter_Endpoint() + public void AddQylOpenTelemetry_InvokesConfigureMetricsCallback() { var services = new ServiceCollection(); - var metricsConfigured = false; + var configured = false; services.AddQylOpenTelemetry(o => { o.EnableTracing = false; o.ServiceName = "orders-api"; - o.ConfigureMetrics = metrics => - { - metricsConfigured = true; - metrics.AddMeter("orders-api"); - }; + o.ConfigureMetrics = _ => configured = true; }); - Assert.True(metricsConfigured); - Assert.NotEmpty(services); + configured.Should().BeTrue(); } [Fact] - public async Task AddQylOpenTelemetry_Collects_Configured_Meter_Name_Through_OpenTelemetry_Reader() + public async Task AddQylOpenTelemetry_CollectsConfiguredMeter_ThroughOpenTelemetryReader() { var services = new ServiceCollection(); var exporter = new CapturingMetricExporter(); @@ -93,111 +58,58 @@ public async Task AddQylOpenTelemetry_Collects_Configured_Meter_Name_Through_Ope o.ServiceName = "orders-api"; o.MeterNames.Add(" orders-api "); o.MeterNames.Add("orders-api"); - o.ConfigureMetrics = metrics => metrics.AddReader(reader); + o.ConfigureMetrics = m => m.AddReader(reader); }); await using var provider = services.BuildServiceProvider(); - List hostedServices = []; - foreach (var hostedService in provider.GetServices()) - { - hostedServices.Add(hostedService); - } - - foreach (var hostedService in hostedServices) - await hostedService.StartAsync(CancellationToken.None); + var hosted = provider.GetServices().ToList(); + foreach (var h in hosted) await h.StartAsync(TestContext.Current.CancellationToken); try { using var meter = new Meter("orders-api"); - var counter = meter.CreateCounter( - name: "orders.processed", - unit: "{order}", - description: "Processed orders."); - - counter.Add(7); + meter.CreateCounter("orders.processed", "{order}", "Processed orders.").Add(7); - Assert.True(reader.Collect(timeoutMilliseconds: 10_000)); + reader.Collect(timeoutMilliseconds: 10_000).Should().BeTrue(); - var metric = Assert.Single(exporter.Metrics, static metric => metric.Name == "orders.processed"); - - Assert.Equal("orders-api", metric.MeterName); - Assert.Equal("{order}", metric.Unit); - Assert.Equal("Processed orders.", metric.Description); - Assert.Equal(7, metric.Value); + var metric = exporter.Metrics.Should().ContainSingle(m => m.Name == "orders.processed").Subject; + metric.MeterName.Should().Be("orders-api"); + metric.Unit.Should().Be("{order}"); + metric.Description.Should().Be("Processed orders."); + metric.Value.Should().Be(7); } finally { - for (var i = hostedServices.Count - 1; i >= 0; i--) - { - await hostedServices[i].StopAsync(CancellationToken.None); - } + for (var i = hosted.Count - 1; i >= 0; i--) + await hosted[i].StopAsync(TestContext.Current.CancellationToken); } } - [Fact] - public void AddQylOpenTelemetry_Requires_Endpoint_When_Tracing_Is_Enabled() - { - var services = new ServiceCollection(); - - var ex = Assert.Throws(() => services.AddQylOpenTelemetry(static o => - { - o.ServiceName = "orders-api"; - })); - - Assert.Contains(nameof(QylOtelOptions.Endpoint), ex.Message, StringComparison.Ordinal); - } - - [Theory] - [InlineData(null)] - [InlineData("")] - [InlineData(" ")] - public void AddQylOpenTelemetry_Rejects_Missing_Service_Name(string? serviceName) + public static TheoryData, string> RejectionCases() => new() { - var services = new ServiceCollection(); - - var ex = Assert.Throws(() => services.AddQylOpenTelemetry(o => - { - o.EnableTracing = false; - o.EnableMetrics = true; - o.ServiceName = serviceName; - })); - - Assert.Contains(nameof(QylOtelOptions.ServiceName), ex.Message, StringComparison.Ordinal); - } + // Tracing on but no endpoint → Endpoint required + { static o => { o.ServiceName = "orders-api"; }, nameof(QylOtelOptions.Endpoint) }, + // Missing service name variants + { static o => { o.EnableTracing = false; o.EnableMetrics = true; o.ServiceName = null; }, nameof(QylOtelOptions.ServiceName) }, + { static o => { o.EnableTracing = false; o.EnableMetrics = true; o.ServiceName = ""; }, nameof(QylOtelOptions.ServiceName) }, + { static o => { o.EnableTracing = false; o.EnableMetrics = true; o.ServiceName = " "; }, nameof(QylOtelOptions.ServiceName) }, + // Invalid sample rates + { static o => { o.EnableTracing = false; o.EnableMetrics = true; o.ServiceName = "orders-api"; o.SampleRate = -0.01; }, nameof(QylOtelOptions.SampleRate) }, + { static o => { o.EnableTracing = false; o.EnableMetrics = true; o.ServiceName = "orders-api"; o.SampleRate = 1.01; }, nameof(QylOtelOptions.SampleRate) }, + { static o => { o.EnableTracing = false; o.EnableMetrics = true; o.ServiceName = "orders-api"; o.SampleRate = double.NaN; }, nameof(QylOtelOptions.SampleRate) }, + { static o => { o.EnableTracing = false; o.EnableMetrics = true; o.ServiceName = "orders-api"; o.SampleRate = double.PositiveInfinity; }, nameof(QylOtelOptions.SampleRate) }, + // Whitespace meter name + { static o => { o.Endpoint = s_traceEndpoint; o.ServiceName = "orders-api"; o.MeterNames.Add(" "); }, nameof(QylOtelOptions.MeterNames) }, + }; [Theory] - [InlineData(-0.01)] - [InlineData(1.01)] - [InlineData(double.NaN)] - [InlineData(double.PositiveInfinity)] - public void AddQylOpenTelemetry_Rejects_Invalid_Sample_Rate(double sampleRate) - { - var services = new ServiceCollection(); - - var ex = Assert.Throws(() => services.AddQylOpenTelemetry(o => - { - o.EnableTracing = false; - o.EnableMetrics = true; - o.ServiceName = "orders-api"; - o.SampleRate = sampleRate; - })); - - Assert.Contains(nameof(QylOtelOptions.SampleRate), ex.Message, StringComparison.Ordinal); - } - - [Fact] - public void AddQylOpenTelemetry_Rejects_Empty_Meter_Name_During_Registration() + [MemberData(nameof(RejectionCases))] + public void AddQylOpenTelemetry_RejectsInvalidConfiguration(Action configure, string expectedFieldName) { - var services = new ServiceCollection(); + var ex = Assert.Throws(() => + new ServiceCollection().AddQylOpenTelemetry(configure)); - var ex = Assert.Throws(() => services.AddQylOpenTelemetry(static o => - { - o.Endpoint = s_traceEndpoint; - o.ServiceName = "orders-api"; - o.MeterNames.Add(" "); - })); - - Assert.Contains(nameof(QylOtelOptions.MeterNames), ex.Message, StringComparison.Ordinal); + ex.Message.Should().Contain(expectedFieldName); } private sealed class CapturingMetricExporter : BaseExporter @@ -209,26 +121,11 @@ private sealed class CapturingMetricExporter : BaseExporter public override ExportResult Export(in Batch batch) { foreach (var metric in batch) - { foreach (var point in metric.GetMetricPoints()) - { - _metrics.Add(new CapturedMetric( - metric.MeterName, - metric.Name, - metric.Unit, - metric.Description, - point.GetSumLong())); - } - } - + _metrics.Add(new CapturedMetric(metric.MeterName, metric.Name, metric.Unit, metric.Description, point.GetSumLong())); return ExportResult.Success; } } - private sealed record CapturedMetric( - string MeterName, - string Name, - string Unit, - string Description, - long Value); + private sealed record CapturedMetric(string MeterName, string Name, string Unit, string Description, long Value); }