Skip to content

ci: breakdown benchmarks #7641

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
ematipico merged 2 commits into from
Oct 2, 2025
Merged

ci: breakdown benchmarks #7641

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
d321545
ci: breakdown benchmarks
ematipico Oct 2, 2025
b7e6dd1
chore: apply feedback
ematipico Oct 2, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 2 additions & 29 deletions .github/workflows/benchmark.yml → ...hub/workflows/benchmark_configuration.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
name: Benchmarks
name: Benchmarks Configuration

on:
workflow_dispatch:
Expand All @@ -9,27 +9,15 @@ on:
- next
paths:
- 'Cargo.lock'
- 'crates/**_analyze/**/*.rs'
- 'crates/**_formatter/**/*.rs'
- 'crates/**_parser/**/*.rs'
- 'crates/biome_configuration/**/*.rs'
- 'crates/biome_grit_patterns/**/*.rs'
- 'crates/biome_module_graph/**/*.rs'
- 'crates/biome_package/**/*.rs'
- 'crates/biome_rowan/**/*.rs'
push:
branches:
- main
- next
paths:
- 'Cargo.lock'
- 'crates/**_analyze/**/*.rs'
- 'crates/**_formatter/**/*.rs'
- 'crates/**_parser/**/*.rs'
- 'crates/biome_configuration/**/*.rs'
- 'crates/biome_grit_patterns/**/*.rs'
- 'crates/biome_module_graph/**/*.rs'
- 'crates/biome_package/**/*.rs'
- 'crates/biome_rowan/**/*.rs'

env:
Expand All @@ -44,29 +32,14 @@ jobs:
strategy:
matrix:
package:
- biome_js_parser
- biome_js_formatter
- biome_js_analyze
- biome_css_parser
- biome_css_formatter
- biome_css_analyze
- biome_json_parser
- biome_json_formatter
- biome_json_analyze
- biome_graphql_parser
- biome_graphql_formatter
- biome_html_parser
- biome_html_formatter
- biome_module_graph
- biome_package
- biome_configuration

steps:

- name: Checkout PR Branch
uses: actions/checkout@08eba0b27e820071cde6df949e0beb9ba4906955 # v4.3.0
with:
ref: ${{ steps.sha.outputs.result }}
ref: ${{ github.event.pull_request.head.sha || github.sha }}

- name: Install toolchain
uses: moonrepo/setup-rust@ede6de059f8046a5e236c94046823e2af11ca670 # v1.2.2
Expand Down
75 changes: 75 additions & 0 deletions .github/workflows/benchmark_css.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,75 @@
name: Benchmarks CSS

on:
workflow_dispatch:
pull_request:
types: [ opened, synchronize ]
branches:
- main
- next
paths:
- 'Cargo.lock'
- 'crates/biome_css_analyze/**/*.rs'
- 'crates/biome_css_formatter/**/*.rs'
- 'crates/biome_css_parser/**/*.rs'
- 'crates/biome_css_*/**/*.rs'
- 'crates/biome_rowan/**/*.rs'
- 'crates/biome_parser/**/*.rs'
push:
branches:
- main
- next
paths:
- 'Cargo.lock'
- 'crates/biome_css_analyze/**/*.rs'
- 'crates/biome_css_formatter/**/*.rs'
- 'crates/biome_css_parser/**/*.rs'
- 'crates/biome_css_*/**/*.rs'
- 'crates/biome_rowan/**/*.rs'
- 'crates/biome_parser/**/*.rs'

env:
RUST_LOG: info

jobs:
bench:
permissions:
pull-requests: write
Comment on lines +36 to +37
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai bot Oct 2, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🔴 Critical

Don’t forget contents: read.

The checkout step needs that scope; otherwise every push run falls over instantly.

Same fix as elsewhere:

     permissions:
-      pull-requests: write
+      pull-requests: write
+      contents: read
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
permissions:
pull-requests: write
permissions:
pull-requests: write
contents: read
🤖 Prompt for AI Agents 
In .github/workflows/benchmark_css.yml around lines 36 to 37, the workflow
grants pull-requests: write but is missing the required contents: read
permission for the checkout action; update the permissions block to include
contents: read alongside the existing pull-requests permission so the checkout
step can access the repository contents and workflow runs won't fail.

name: Bench
runs-on: depot-ubuntu-24.04-arm-16
strategy:
matrix:
package:
- biome_css_parser
- biome_css_formatter
- biome_css_analyze

steps:

- name: Checkout PR Branch
uses: actions/checkout@08eba0b27e820071cde6df949e0beb9ba4906955 # v4.3.0
with:
ref: ${{ github.event.pull_request.head.sha || github.sha }}

- name: Install toolchain
uses: moonrepo/setup-rust@ede6de059f8046a5e236c94046823e2af11ca670 # v1.2.2
with:
channel: stable
cache-target: release
bins: cargo-codspeed
cache-base: main
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

- name: Compile
timeout-minutes: 20
run: cargo codspeed build -p ${{ matrix.package }}
env:
CARGO_BUILD_JOBS: 3 # Default is 4 (equals to the vCPU count of the runner), which leads OOM on cargo build

- name: Run the benchmarks
uses: CodSpeedHQ/action@76578c2a7ddd928664caa737f0e962e3085d4e7c # v3.8.1
timeout-minutes: 50
with:
run: cargo codspeed run
token: ${{ secrets.CODSPEED_TOKEN }}
72 changes: 72 additions & 0 deletions .github/workflows/benchmark_graphql.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,72 @@
name: Benchmarks GraphQL

on:
workflow_dispatch:
pull_request:
types: [ opened, synchronize ]
branches:
- main
- next
paths:
- 'Cargo.lock'
- 'crates/biome_graphql_formatter/**/*.rs'
- 'crates/biome_graphql_parser/**/*.rs'
- 'crates/biome_graphql_*/**/*.rs'
- 'crates/biome_rowan/**/*.rs'
- 'crates/biome_parser/**/*.rs'
push:
branches:
- main
- next
paths:
- 'Cargo.lock'
- 'crates/biome_graphql_formatter/**/*.rs'
- 'crates/biome_graphql_parser/**/*.rs'
- 'crates/biome_graphql_*/**/*.rs'
- 'crates/biome_rowan/**/*.rs'
- 'crates/biome_parser/**/*.rs'

env:
RUST_LOG: info

jobs:
bench:
permissions:
pull-requests: write
Comment on lines +34 to +35
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai bot Oct 2, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🔴 Critical

Restore contents read scope.

Checkout needs contents: read; without it, pushes to main/next will keep failing right at the first step.

Suggested tweak:

     permissions:
-      pull-requests: write
+      pull-requests: write
+      contents: read
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
permissions:
pull-requests: write
permissions:
pull-requests: write
contents: read
🤖 Prompt for AI Agents 
In .github/workflows/benchmark_graphql.yml around lines 34 to 35, the workflow
permissions only grant pull-requests: write but lack the required contents: read
permission needed by actions/checkout; update the permissions block to include
contents: read (in addition to pull-requests: write) so the checkout step can
access the repository contents and subsequent pushes to main/next won't fail.

name: Bench
runs-on: depot-ubuntu-24.04-arm-16
strategy:
matrix:
package:
- biome_graphql_parser
- biome_graphql_formatter

steps:

- name: Checkout PR Branch
uses: actions/checkout@08eba0b27e820071cde6df949e0beb9ba4906955 # v4.3.0
with:
ref: ${{ github.event.pull_request.head.sha || github.sha }}

- name: Install toolchain
uses: moonrepo/setup-rust@ede6de059f8046a5e236c94046823e2af11ca670 # v1.2.2
with:
channel: stable
cache-target: release
bins: cargo-codspeed
cache-base: main
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

- name: Compile
timeout-minutes: 20
run: cargo codspeed build -p ${{ matrix.package }}
env:
CARGO_BUILD_JOBS: 3 # Default is 4 (equals to the vCPU count of the runner), which leads OOM on cargo build

- name: Run the benchmarks
uses: CodSpeedHQ/action@76578c2a7ddd928664caa737f0e962e3085d4e7c # v3.8.1
timeout-minutes: 50
with:
run: cargo codspeed run
token: ${{ secrets.CODSPEED_TOKEN }}
75 changes: 75 additions & 0 deletions .github/workflows/benchmark_js.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,75 @@
name: Benchmarks JS

on:
workflow_dispatch:
pull_request:
types: [ opened, synchronize ]
branches:
- main
- next
paths:
- 'Cargo.lock'
- 'crates/biome_js_analyze/**/*.rs'
- 'crates/biome_js_formatter/**/*.rs'
- 'crates/biome_js_parser/**/*.rs'
- 'crates/biome_js_*/**/*.rs'
- 'crates/biome_rowan/**/*.rs'
- 'crates/biome_parser/**/*.rs'
push:
branches:
- main
- next
paths:
- 'Cargo.lock'
- 'crates/biome_js_analyze/**/*.rs'
- 'crates/biome_js_formatter/**/*.rs'
- 'crates/biome_js_parser/**/*.rs'
- 'crates/biome_js_*/**/*.rs'
- 'crates/biome_rowan/**/*.rs'
- 'crates/biome_parser/**/*.rs'

env:
RUST_LOG: info

jobs:
bench:
permissions:
pull-requests: write
Comment on lines +36 to +37
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai bot Oct 2, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🔴 Critical

Needs contents scope for checkout.

Right now the token can’t read the repo on push events, so the workflow will fail before building.

Patch proposal:

     permissions:
-      pull-requests: write
+      pull-requests: write
+      contents: read
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
permissions:
pull-requests: write
permissions:
pull-requests: write
contents: read
🤖 Prompt for AI Agents 
.github/workflows/benchmark_js.yml around lines 36 to 37: the workflow only
grants pull-requests: write, but checkout on push requires repository contents
permission, so add a contents permission (at least read) under permissions;
update the permissions block to include contents: read (or contents: write if
the job needs push access) alongside the existing pull-requests: write so the
GITHUB_TOKEN can access the repo during checkout.

name: Bench
runs-on: depot-ubuntu-24.04-arm-16
strategy:
matrix:
package:
- biome_js_parser
- biome_js_formatter
- biome_js_analyze

steps:

- name: Checkout PR Branch
uses: actions/checkout@08eba0b27e820071cde6df949e0beb9ba4906955 # v4.3.0
with:
ref: ${{ github.event.pull_request.head.sha || github.sha }}

- name: Install toolchain
uses: moonrepo/setup-rust@ede6de059f8046a5e236c94046823e2af11ca670 # v1.2.2
with:
channel: stable
cache-target: release
bins: cargo-codspeed
cache-base: main
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

- name: Compile
timeout-minutes: 20
run: cargo codspeed build -p ${{ matrix.package }}
env:
CARGO_BUILD_JOBS: 3 # Default is 4 (equals to the vCPU count of the runner), which leads OOM on cargo build

- name: Run the benchmarks
uses: CodSpeedHQ/action@76578c2a7ddd928664caa737f0e962e3085d4e7c # v3.8.1
timeout-minutes: 50
with:
run: cargo codspeed run
token: ${{ secrets.CODSPEED_TOKEN }}
75 changes: 75 additions & 0 deletions .github/workflows/benchmark_json.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,75 @@
name: Benchmarks JSON

on:
workflow_dispatch:
pull_request:
types: [ opened, synchronize ]
branches:
- main
- next
paths:
- 'Cargo.lock'
- 'crates/biome_json_analyze/**/*.rs'
- 'crates/biome_json_formatter/**/*.rs'
- 'crates/biome_json_parser/**/*.rs'
- 'crates/biome_json_*/**/*.rs'
- 'crates/biome_rowan/**/*.rs'
- 'crates/biome_parser/**/*.rs'
push:
branches:
- main
- next
paths:
- 'Cargo.lock'
- 'crates/biome_json_analyze/**/*.rs'
- 'crates/biome_json_formatter/**/*.rs'
- 'crates/biome_json_parser/**/*.rs'
- 'crates/biome_json_*/**/*.rs'
- 'crates/biome_rowan/**/*.rs'
- 'crates/biome_parser/**/*.rs'

env:
RUST_LOG: info

jobs:
bench:
permissions:
pull-requests: write
Comment on lines +36 to +37
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai bot Oct 2, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🔴 Critical

Add contents read permission.

With only pull-requests: write, the token cannot fetch the repo on push events, so checkout will bomb out. Please include contents: read alongside it.

Apply this diff:

     permissions:
-      pull-requests: write
+      pull-requests: write
+      contents: read
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
permissions:
pull-requests: write
permissions:
pull-requests: write
contents: read
🤖 Prompt for AI Agents 
.github/workflows/benchmark_json.yml around lines 36 to 37: the workflow only
grants pull-requests: write which prevents the checkout from fetching repo
contents on push events; update the permissions block to include contents: read
alongside pull-requests: write (i.e., add a contents: read entry under
permissions) so the token can read repository contents during checkout.

name: Bench
runs-on: depot-ubuntu-24.04-arm-16
strategy:
matrix:
package:
- biome_json_parser
- biome_json_formatter
- biome_json_analyze

steps:

- name: Checkout PR Branch
uses: actions/checkout@08eba0b27e820071cde6df949e0beb9ba4906955 # v4.3.0
with:
ref: ${{ github.event.pull_request.head.sha || github.sha }}

- name: Install toolchain
uses: moonrepo/setup-rust@ede6de059f8046a5e236c94046823e2af11ca670 # v1.2.2
with:
channel: stable
cache-target: release
bins: cargo-codspeed
cache-base: main
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

- name: Compile
timeout-minutes: 20
run: cargo codspeed build -p ${{ matrix.package }}
env:
CARGO_BUILD_JOBS: 3 # Default is 4 (equals to the vCPU count of the runner), which leads OOM on cargo build

- name: Run the benchmarks
uses: CodSpeedHQ/action@76578c2a7ddd928664caa737f0e962e3085d4e7c # v3.8.1
timeout-minutes: 50
with:
run: cargo codspeed run
token: ${{ secrets.CODSPEED_TOKEN }}
Loading