Skip to content

docs: Add Benchmarks landing page and update CI triggers#931

Merged
vbreuss merged 7 commits into
mainfrom
feat/benchmarks-page
May 3, 2026
Merged

docs: Add Benchmarks landing page and update CI triggers#931
vbreuss merged 7 commits into
mainfrom
feat/benchmarks-page

Conversation

@vbreuss

@vbreuss vbreuss commented May 3, 2026

Copy link
Copy Markdown
Member

This pull request introduces a new benchmarks documentation page and updates the workflow that notifies the documentation site of changes. The most significant changes are the addition of a comprehensive benchmarks comparison between aweXpect and FluentAssertions, and improvements to the automation for triggering documentation site rebuilds.

Documentation: Benchmarks

  • Added a new page 17-benchmarks.mdx to the documentation, providing detailed benchmarks comparing aweXpect's performance to FluentAssertions across multiple assertion scenarios. The page uses interactive tabs, code samples, and links to source files to help users understand performance characteristics.

CI/CD Workflow Improvements

  • Changed the workflow trigger from a direct push on specific docs paths to running after the "Build" workflow completes successfully, or via manual dispatch. This ensures the documentation site is only notified after a successful build.
  • Enhanced the payload sent to the documentation site to use the correct commit SHA from the triggering workflow, improving traceability and accuracy of site updates.

vbreuss and others added 3 commits May 3, 2026 09:44
Adds a new "Benchmarks" page slotted between "Advanced" (sidebar position 16)
and "Write your own extension" (renumbered 17 -> 18). The page hosts one tab
per HappyCaseBenchmark, each showing a short description, the latest aweXpect
vs FluentAssertions snapshot rendered by the BenchmarkResult component (lives
in Testably.Server, imported via @site/src/components/BenchmarkResult), the
two source snippets, and a link to the benchmark file.

The component, the snapshot data, and the build-time fetch from this repo's
benchmarks branch are introduced in the matching Testably.Server change.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two fixes folded together:

1. The path filter was matched against the pre-flatten layout
   (Docs/pages/docs/expectations/**, Docs/pages/docs/extensions/**) which
   was removed in PR #930. After PR #930 the docs sit directly under
   Docs/pages/, so the workflow has been silently failing to fire on doc
   updates. Replaced with a single Docs/pages/** filter that also matches
   the benchmark data files on the long-lived benchmarks branch.

2. Added the benchmarks branch to the trigger so updates to the
   limited-data.js file used by Testably.Server's BenchmarkResult
   component also kick off a site rebuild.

Note: pushes to the benchmarks branch performed by the Build workflow's
GITHUB_TOKEN don't actually fire push events (a documented GitHub
limitation to avoid recursive workflows). To make the auto-rebuild work
end-to-end, the benchmarks job in build.yml will need to either switch
to a PAT for the Contents API PUT or fire a repository_dispatch directly.
That is left as a follow-up.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Replaces the push + path filter with a workflow_run trigger gated on the
"Build" workflow finishing successfully on main. The benchmarks job is part
of every main build and updates the long-lived benchmarks branch, so a
single signal per main build covers both doc updates and benchmark refreshes
without trying to express both via path filters (which couldn't anyway --
the benchmarks-branch push happens via GITHUB_TOKEN, which doesn't fire
push events).

The dispatched payload now uses workflow_run.head_sha so the receiver gets
the sha of the commit that was actually built, not the sha of main at the
time the notify workflow itself runs (those can differ briefly).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@vbreuss vbreuss self-assigned this May 3, 2026
Copilot AI review requested due to automatic review settings May 3, 2026 08:57
@vbreuss vbreuss added the documentation Improvements or additions to documentation label May 3, 2026
@vbreuss vbreuss enabled auto-merge (squash) May 3, 2026 09:00

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR expands the documentation set with a new benchmarks landing page and an extension-authoring guide, and changes the docs-site notification workflow so site rebuilds are triggered after a successful main-branch build. In the broader codebase, these changes are aimed at making benchmark data visible in the published docs and improving guidance for consumers extending aweXpect.

Changes:

  • Add a new Benchmarks docs page with methodology, benchmark tabs, and source links.
  • Add a new Write your own extension guide covering custom expectations, constraints, and customization APIs.
  • Switch docs-site notifications from path-based push triggers to workflow_run on the Build workflow.

Reviewed changes

Copilot reviewed 2 out of 3 changed files in this pull request and generated 3 comments.

File Description
Docs/pages/18-write-extension.md Adds a new extension authoring guide with code samples for custom expectations and customization.
Docs/pages/17-benchmarks.mdx Adds a benchmark landing page with tabs, methodology, and links to benchmark implementations.
.github/workflows/notify-docs-site.yml Changes docs-site rebuild triggering from direct pushes to completed Build workflow runs.

Comment thread Docs/pages/17-benchmarks.mdx
Comment thread .github/workflows/notify-docs-site.yml Outdated
Comment thread Docs/pages/17-benchmarks.mdx
@github-actions

github-actions Bot commented May 3, 2026

Copy link
Copy Markdown
Contributor

Test Results

     23 files   - 27       23 suites   - 27   9m 15s ⏱️ + 1m 37s
 19 519 tests  - 29   19 518 ✅  - 29  1 💤 ±0  0 ❌ ±0 
101 024 runs   - 81  101 023 ✅  - 81  1 💤 ±0  0 ❌ ±0 

Results for commit 21156da. ± Comparison against base commit 4aaf522.

This pull request removes 3112 and adds 3083 tests. Note that renamed tests count towards both.
aweXpect.Core.Tests.Core.Exceptions.FailExceptionTests ‑ Message_ShouldBeSet(message: "message11ab2512-398c-4266-af19-bd5f739058a5")
aweXpect.Core.Tests.Core.Exceptions.FailExceptionTests ‑ Message_ShouldBeSet(message: "message987a43de-582e-419d-b0c4-9ac07c69149a")
aweXpect.Core.Tests.Core.Exceptions.FailExceptionTests ‑ Message_ShouldBeSet(message: "messagec6af4774-ffcf-4767-8c95-c03f34cd9da1")
aweXpect.Core.Tests.Core.Exceptions.FailExceptionTests ‑ Message_ShouldBeSet(message: "messagedd32f8c5-fb91-4afe-bbcf-d7909953cebf")
aweXpect.Core.Tests.Core.Exceptions.FailExceptionTests ‑ Message_ShouldBeSet(message: "messagee4f44a53-2d87-41a6-9768-03379cb9709e")
aweXpect.Core.Tests.Core.Exceptions.FailExceptionTests ‑ Message_ShouldBeSet(message: "messagefcba0ef3-cf61-4dc9-bf23-c2a268afbff7")
aweXpect.Core.Tests.Core.Exceptions.SkipExceptionTests ‑ Message_ShouldBeSet(message: "message10e606ec-93df-4c7c-be98-b2a5f9ac7d42")
aweXpect.Core.Tests.Core.Exceptions.SkipExceptionTests ‑ Message_ShouldBeSet(message: "message31cf8c66-a789-4d73-aea3-f3a6d1b444e3")
aweXpect.Core.Tests.Core.Exceptions.SkipExceptionTests ‑ Message_ShouldBeSet(message: "message37ad75c9-03d2-4e24-965c-04eb9ec5e26e")
aweXpect.Core.Tests.Core.Exceptions.SkipExceptionTests ‑ Message_ShouldBeSet(message: "message5913bcc6-a58e-4130-8ef9-e4fe0d662ace")
…
aweXpect.Core.Tests.Core.Exceptions.FailExceptionTests ‑ Message_ShouldBeSet(message: "message0e1500ad-1370-456f-9564-03b1a9831228")
aweXpect.Core.Tests.Core.Exceptions.FailExceptionTests ‑ Message_ShouldBeSet(message: "message16eb8ff1-293c-4a74-989e-5e6e2b515a50")
aweXpect.Core.Tests.Core.Exceptions.FailExceptionTests ‑ Message_ShouldBeSet(message: "message1fb3129f-639d-461a-8096-58c69f9dc549")
aweXpect.Core.Tests.Core.Exceptions.FailExceptionTests ‑ Message_ShouldBeSet(message: "message3de9235b-fd04-40b7-8c13-c6311d78e371")
aweXpect.Core.Tests.Core.Exceptions.FailExceptionTests ‑ Message_ShouldBeSet(message: "message9044f742-0932-46fa-a2c5-f0062162e9f6")
aweXpect.Core.Tests.Core.Exceptions.FailExceptionTests ‑ Message_ShouldBeSet(message: "messageebb25351-a28a-46eb-82a4-43ae15e7046a")
aweXpect.Core.Tests.Core.Exceptions.SkipExceptionTests ‑ Message_ShouldBeSet(message: "message590af7f8-ea39-4484-ac4e-b1eb1d6eab33")
aweXpect.Core.Tests.Core.Exceptions.SkipExceptionTests ‑ Message_ShouldBeSet(message: "message624adccc-c8f8-4154-a165-c918337e6cd6")
aweXpect.Core.Tests.Core.Exceptions.SkipExceptionTests ‑ Message_ShouldBeSet(message: "message97791cf0-cfb7-403d-803f-ef0e2074f3c4")
aweXpect.Core.Tests.Core.Exceptions.SkipExceptionTests ‑ Message_ShouldBeSet(message: "messagea48aa519-aee5-4b5b-9e1b-4705c01ae77a")
…

♻️ This comment has been updated with latest results.

vbreuss and others added 3 commits May 3, 2026 11:21
Per code review: the previous condition allowed workflow_dispatch from any
branch, so a feature branch could trigger a production docs-site rebuild
with an unmerged SHA. Re-add the main-only guard for the manual case while
keeping the workflow_run-driven automatic case unchanged.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Sweep the canonical org rename and the docs hand-off into one update:

- github.com/aweXpect/aweXpect -> github.com/Testably/aweXpect across
  README badges, RepositoryUrl in Directory.Build.props (which flows
  into the API surface .txt fixtures), build pipeline URLs, Octokit
  owner args, and the Stryker dashboard project name.
- PackageProjectUrl and the README/landing-page doc links move from
  https://aweXpect.com to https://docs.testably.org/awexpect, matching
  the path layout after the docs flatten in PR #930.

Sibling-repo links (aweXpect.Json/.Web/.Reflection/.Testably/.Mockolate)
are intentionally left alone -- those repos need separate confirmation
before the rename.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings May 3, 2026 09:35

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 15 out of 16 changed files in this pull request and generated 2 comments.

Comment thread Pipeline/Build.MutationTests.cs
Comment thread .github/workflows/notify-docs-site.yml
@vbreuss vbreuss disabled auto-merge May 3, 2026 09:49
@vbreuss vbreuss merged commit 6dfb897 into main May 3, 2026
9 checks passed
@vbreuss vbreuss deleted the feat/benchmarks-page branch May 3, 2026 09:49
github-actions Bot added a commit that referenced this pull request May 3, 2026
github-actions Bot added a commit that referenced this pull request May 3, 2026
@github-actions

github-actions Bot commented May 3, 2026

Copy link
Copy Markdown
Contributor

🚀 Benchmark Results

Details

BenchmarkDotNet v0.15.8, Linux Ubuntu 24.04.4 LTS (Noble Numbat)
AMD EPYC 7763 2.90GHz, 1 CPU, 4 logical and 2 physical cores
.NET SDK 10.0.203
[Host] : .NET 8.0.26 (8.0.26, 8.0.2626.16921), X64 RyuJIT x86-64-v3

Job=InProcess Toolchain=InProcessEmitToolchain IterationCount=15
LaunchCount=1 WarmupCount=10

Method Mean Error StdDev Gen0 Gen1 Allocated
Bool_aweXpect 246.4 ns 1.48 ns 1.38 ns 0.0415 - 696 B
Bool_FluentAssertions 231.7 ns 2.69 ns 2.51 ns 0.0567 - 952 B
Equivalency_aweXpect 298,487.2 ns 2,055.18 ns 1,821.87 ns 20.0195 0.4883 335444 B
Equivalency_FluentAssertions 2,498,435.0 ns 12,798.61 ns 10,687.42 ns 285.1563 46.8750 4804906 B
Int_GreaterThan_aweXpect 252.5 ns 7.21 ns 6.75 ns 0.0482 - 808 B
Int_GreaterThan_FluentAssertions 246.2 ns 6.87 ns 6.43 ns 0.0730 - 1224 B
ItemsCount_AtLeast_aweXpect 484.1 ns 7.02 ns 6.22 ns 0.0811 - 1360 B
ItemsCount_AtLeast_FluentAssertions 463.8 ns 4.53 ns 3.78 ns 0.1197 - 2008 B
String_aweXpect 481.7 ns 5.58 ns 5.22 ns 0.0668 - 1128 B
String_FluentAssertions 1,165.0 ns 8.37 ns 7.42 ns 0.2346 - 3944 B
StringArray_aweXpect 1,947.9 ns 14.96 ns 14.00 ns 0.1564 - 2624 B
StringArray_FluentAssertions 1,334.5 ns 27.84 ns 26.04 ns 0.2480 - 4152 B
StringArrayInAnyOrder_aweXpect 2,619.9 ns 9.59 ns 8.51 ns 0.1678 - 2816 B
StringArrayInAnyOrder_FluentAssertions 90,630.9 ns 660.02 ns 585.09 ns 3.4180 - 57481 B

@github-actions

Copy link
Copy Markdown
Contributor

This is addressed in release v2.32.0.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation state: released The issue is released

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants