Skip to content

feat: add Criterion benchmarks for safety layer hot paths #836

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
ilblackdragon merged 11 commits into from
Mar 15, 2026
Merged

feat: add Criterion benchmarks for safety layer hot paths #836

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
5a2ec50
feat: add Criterion benchmarks for safety layer hot paths
ilblackdragon Mar 10, 2026
70c80e2
fix: add bench-compile to CI roll-up job
ilblackdragon Mar 10, 2026
e9d317f
fix: add black_box to benchmarks, use real SafetyLayer pipeline
ilblackdragon Mar 10, 2026
5a5a966
fix: apply cargo fmt to benchmark files
ilblackdragon Mar 10, 2026
edae346
fix: copy benches/ in Dockerfile to fix manifest parse error
ilblackdragon Mar 10, 2026
446dd31
chore: re-trigger CI after adding skip-regression-check label
ilblackdragon Mar 10, 2026
219a2ce
fix: address PR review comments on criterion benchmarks
ilblackdragon Mar 10, 2026
7f3fe8b
chore: update Cargo.lock with criterion dependencies
ilblackdragon Mar 10, 2026
ae2a7d0
fix(bench): build secret-like strings at runtime to avoid CI secret s…
ilblackdragon Mar 10, 2026
0fd8bad
fix: rename tool_dispatch bench, drop async_tokio, replace JSON bench…
ilblackdragon Mar 12, 2026
9722a8c
Merge remote-tracking branch 'origin/staging' into feat/criterion-ben…
ilblackdragon Mar 14, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 17 additions & 2 deletions .github/workflows/test.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -104,6 +104,20 @@ jobs:
- name: Instantiation test (host linker compatibility)
run: cargo test --all-features wit_compat -- --nocapture

bench-compile:
name: Benchmark Compilation
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@v6
- name: Install Rust
uses: dtolnay/rust-toolchain@stable
- uses: Swatinem/rust-cache@v2
with:
key: bench
- name: Compile benchmarks
run: cargo bench --all-features --no-run

docker-build:
name: Docker Build
if: >
Expand Down Expand Up @@ -135,7 +149,7 @@ jobs:
name: Run Tests
runs-on: ubuntu-latest
if: always()
needs: [tests, telegram-tests, wasm-wit-compat, docker-build, windows-build, version-check]
needs: [tests, telegram-tests, wasm-wit-compat, docker-build, windows-build, version-check, bench-compile]
steps:
- run: |
# Unit tests must always pass
Expand All @@ -144,13 +158,14 @@ jobs:
exit 1
fi
# Gated jobs: must pass on promotion PRs / push, skipped on developer PRs
for job in telegram-tests wasm-wit-compat docker-build windows-build version-check; do
for job in telegram-tests wasm-wit-compat docker-build windows-build version-check bench-compile; do
case "$job" in
telegram-tests) result="${{ needs.telegram-tests.result }}" ;;
wasm-wit-compat) result="${{ needs.wasm-wit-compat.result }}" ;;
docker-build) result="${{ needs.docker-build.result }}" ;;
windows-build) result="${{ needs.windows-build.result }}" ;;
version-check) result="${{ needs.version-check.result }}" ;;
bench-compile) result="${{ needs.bench-compile.result }}" ;;
esac
if [[ "$result" == "failure" || "$result" == "cancelled" ]]; then
echo "$job failed"
Expand Down
155 changes: 153 additions & 2 deletions Cargo.lock
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

9 changes: 9 additions & 0 deletions Cargo.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -197,6 +197,15 @@ testcontainers-modules = { version = "0.11", features = ["postgres"] }
pretty_assertions = "1"
tempfile = "3"
insta = "1.46.3"
criterion = "0.5"

[[bench]]
name = "safety_check"
harness = false

[[bench]]
name = "safety_pipeline"
harness = false

[features]
default = ["postgres", "libsql", "html-to-markdown"]
Expand Down
2 changes: 2 additions & 0 deletions Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,8 @@ COPY registry/ registry/
COPY channels-src/ channels-src/
COPY wit/ wit/
COPY providers.json providers.json
# [[bench]] entries in Cargo.toml require bench sources to exist for cargo to parse the manifest
COPY benches/ benches/

Copilot AI Mar 10, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

COPY benches/ benches/ isn’t needed for cargo build --release --bin ironclaw and will invalidate Docker layer caching whenever benchmarks change, increasing image build time. Consider dropping this copy (or moving it behind a build arg / separate stage) unless the image build needs to compile/run benches.

Suggested change
COPY benches/ benches/

Copilot uses AI. Check for mistakes.

@ilblackdragon ilblackdragon Mar 10, 2026

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Kept with comment — [[bench]] entries require source files for manifest parsing, so removing COPY breaks the Docker build.


RUN cargo build --release --bin ironclaw

Expand Down
Loading
Loading