Compile public documentation snippets in CI - #2874
Conversation
|
Warning Review limit reached
Next review available in: 15 minutes Limit details: You’ve used all 10 included reviews currently available. Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?Wait for the limit to reset, then comment An organization admin can change what happens after included review limits in Billing. How do review limits work?CodeRabbit enforces per-developer PR review limits within each organization. For paid Pro and Pro+ reviews, CodeRabbit uses a developer's included PR review attempts over the past 7 days to set the current hourly allowance. At typical activity levels, the full plan allowance applies. Higher sustained activity can lower the allowance until earlier attempts leave the 7-day window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (21)
Comment |
Greptile SummaryThe PR adds a CI gate that compiles public C# documentation fences against current Dekaf projects, validates selected shell commands, and fingerprints known legacy failures.
Confidence Score: 5/5The PR appears safe to merge. No blocking failure remains.
|
| Filename | Overview |
|---|---|
| tests/Dekaf.DocTests/Program.cs | Implements snippet extraction, Roslyn compilation, shell-sample validation, explicit exclusions, and deterministic legacy-failure fingerprinting. |
| tests/Dekaf.DocTests/DocContext.cs | Provides the shared application context intentionally used to validate fragment-level public API usage. |
| .github/workflows/ci.yml | Adds the documentation-snippet job to relevant path filtering and the required aggregate CI gate. |
| docs/docs/testing.md | Documents local execution, known-failure handling, explicit exclusions, and the gate’s non-standalone scope. |
Reviews (2): Last reviewed commit: "fix(docs): harden snippet validation" | Re-trigger Greptile
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
| private static readonly string Prelude = """ | ||
| using System; | ||
| using System.Buffers; | ||
| using System.Buffers.Binary; | ||
| using System.Collections.Concurrent; | ||
| using System.Collections.Generic; | ||
| using System.IO; | ||
| using System.Linq; | ||
| using System.Net; | ||
| using System.Net.Http; | ||
| using System.Net.Security; | ||
| using System.Security.Authentication; | ||
| using System.Security.Cryptography; | ||
| using System.Security.Cryptography.X509Certificates; | ||
| using System.Text; | ||
| using System.Text.Json; | ||
| using System.Text.Json.Serialization; | ||
| using System.Text.Json.Serialization.Metadata; | ||
| using System.Threading; | ||
| using System.Threading.Tasks; | ||
| using Amazon; | ||
| using Avro; | ||
| using Avro.Generic; | ||
| using Azure.Identity; | ||
| using Azure.Core; | ||
| using BenchmarkDotNet.Attributes; | ||
| using Dekaf; | ||
| using Dekaf.Admin; | ||
| using Dekaf.Compression; | ||
| using Dekaf.Compression.Brotli; | ||
| using Dekaf.Compression.Lz4; | ||
| using Dekaf.Compression.Snappy; | ||
| using Dekaf.Compression.Zstd; | ||
| using Dekaf.Consumer; | ||
| using Dekaf.Consumer.DeadLetter; | ||
| using Dekaf.Errors; | ||
| using Dekaf.Extensions.DependencyInjection; | ||
| using Dekaf.Extensions.Hosting; | ||
| using Dekaf.OpenTelemetry; | ||
| using Dekaf.Networking; | ||
| using Dekaf.Outbox; | ||
| using Dekaf.Outbox.EntityFrameworkCore; | ||
| using Dekaf.Producer; | ||
| using Dekaf.Protocol; | ||
| using Dekaf.Protocol.Messages; | ||
| using Dekaf.Protocol.Records; | ||
| using Dekaf.Retry; | ||
| using Dekaf.SchemaRegistry; | ||
| using Dekaf.SchemaRegistry.Avro; | ||
| using Dekaf.SchemaRegistry.Json; | ||
| using Dekaf.SchemaRegistry.Jsonata; | ||
| using Dekaf.SchemaRegistry.Kms.Aws; | ||
| using Dekaf.SchemaRegistry.Kms.Azure; | ||
| using Dekaf.SchemaRegistry.Kms.Gcp; | ||
| using Dekaf.SchemaRegistry.Kms.Vault; | ||
| using Dekaf.SchemaRegistry.Protobuf; | ||
| using Dekaf.Security; | ||
| using Dekaf.Security.Sasl; | ||
| using Dekaf.ShareConsumer; | ||
| using Dekaf.Serialization; | ||
| using Dekaf.Serialization.Json; | ||
| using Dekaf.Serialization.Routing; | ||
| using Dekaf.Testing; | ||
| using Google.Cloud.Kms.V1; | ||
| using Microsoft.AspNetCore.Builder; | ||
| using Microsoft.Extensions.Configuration; | ||
| using Microsoft.Extensions.DependencyInjection; | ||
| using Microsoft.Extensions.Hosting; | ||
| using Microsoft.Extensions.Logging; | ||
| using Microsoft.Extensions.Logging.Abstractions; | ||
| using Microsoft.EntityFrameworkCore; | ||
| using Dekaf.DocTests; | ||
| using static Dekaf.DocTests.DocContext; | ||
| """; |
There was a problem hiding this comment.
The harness injects every Dekaf namespace, references nearly every source project, and exposes synthetic context to each fence. This lets examples with omitted imports, package dependencies, models, or variables pass CI even though the prerequisites shown to readers are insufficient, reducing the gate's ability to catch incomplete snippets.
Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 16db679ceb
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| source = BuildBuilderChain(snippet, source); | ||
| } | ||
|
|
||
| source = MoveTopLevelStatementsBeforeDeclarations(source); |
There was a problem hiding this comment.
For fences that place a type declaration before example usage, this rewrite moves the global statements ahead of the declaration and compiles code different from what readers see. Existing examples such as the custom preset fence in docs/docs/configuration/presets.md contain a class followed by var producer = ...; copied verbatim, that is rejected with CS8803, but this gate reports it as compilable. The documentation check should compile the original ordering and require these examples to be rearranged or explicitly excluded.
Useful? React with 👍 / 👎.
| docs: | ||
| - 'README.md' | ||
| - 'docs/docs/**' | ||
| - 'src/**' | ||
| - 'tests/Dekaf.DocTests/**' | ||
| - 'Directory.Build.props' | ||
| - 'Directory.Build.targets' | ||
| - 'Directory.Packages.props' |
There was a problem hiding this comment.
The inspected doc-snippets job is controlled by this docs output, but .github/workflows/ci.yml is absent from the allowlist. Consequently, a PR that changes only the job's command or setup marks doc-snippets as skipped; the separate code filter runs the full solution, which does not include Dekaf.DocTests, and ci-passed explicitly accepts skipped jobs. Include this workflow among the docs inputs so changes to the gate itself are exercised.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 56c3a3174e
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| - 'Directory.Build.props' | ||
| - 'Directory.Build.targets' | ||
| - 'Directory.Packages.props' | ||
| - '.github/workflows/ci.yml' |
There was a problem hiding this comment.
When a PR adds or changes only global.json, the code filter runs but this docs filter remains false. The full-solution build does not include Dekaf.DocTests, and ci-passed accepts the skipped snippet job, so an SDK/compiler change can break this new project without being detected until a later documentation or source change. Include global.json in this allowlist.
Useful? React with 👍 / 👎.
1 participant
Summary
Adapted from thomhurst/Kevlar#69.
Validation
dotnet build tests/Dekaf.DocTests --configuration Release --framework net10.0dotnet run --no-build --project tests/Dekaf.DocTests --configuration Release --framework net10.0 -- --repository-root .dotnet format tests/Dekaf.DocTests/Dekaf.DocTests.csproj --verify-no-changes --no-restore --verbosity minimalnpm run buildfromdocs/git diff --checkNo
src/changes; performance benchmarks are not applicable.