Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[CI][UR] Unify UR and SYCL benchmarking CI into a single benchmarking CI #17229

Draft
wants to merge 58 commits into
base: sycl
Choose a base branch
from
Draft
Changes from 1 commit
Commits
Show all changes
58 commits
Select commit Hold shift + click to select a range
e6ca992
Move UR devops scripts to devops folder
ianayl Feb 27, 2025
3d42db2
Restrict number of cores used
ianayl Feb 28, 2025
fc70520
Merge branch 'sycl' of https://github.com/intel/llvm into unify-bench…
ianayl Mar 4, 2025
4f08dd6
Restore ur-benchmark*.yml
ianayl Mar 4, 2025
497dcce
[benchmarks] improve HTML and Markdown output
pbalcer Mar 5, 2025
3cbed5e
Test UR benchmarking suite
ianayl Mar 5, 2025
1936207
Merge branch 'unify-benchmark-ci' of https://github.com/intel/llvm in…
ianayl Mar 5, 2025
f79bbbf
Bump tolerance to 7%
ianayl Mar 5, 2025
ffc8139
Revert "Bump tolerance to 7%"
ianayl Mar 5, 2025
0a34e0d
[benchmarks] fix failing benchmarks, improve html output
pbalcer Mar 6, 2025
3f42420
[benchmarks] fix python formatting with black
pbalcer Mar 6, 2025
1c7b189
update driver version
pbalcer Mar 6, 2025
ad13e93
simplify preset implementation and fix normal preset
pbalcer Mar 6, 2025
68ed0c4
Add PVC and BMG as runners
ianayl Mar 6, 2025
18fff93
Merge branch 'unify-benchmark-ci' of https://github.com/intel/llvm in…
ianayl Mar 6, 2025
3a65b98
Install dependencies before running UR script
ianayl Mar 6, 2025
220121a
Use venv for python packages
ianayl Mar 6, 2025
37d361c
Install venv before using venv
ianayl Mar 6, 2025
07f1e10
[benchmarks] allow specifying custom results directories
pbalcer Mar 7, 2025
64cf79c
[benchmarks] sort runs by date for html output
pbalcer Mar 7, 2025
6c28d33
simplify presets, remove suites if all set
pbalcer Mar 10, 2025
e15b94f
[benchmarks] use python venv for scripts
pbalcer Mar 10, 2025
78fd037
Run apt with sudo
ianayl Mar 10, 2025
0ed1599
Merge branch 'unify-benchmark-ci' of https://github.com/intel/llvm in…
ianayl Mar 10, 2025
82b6e55
Ignore "missing" apt packages in workflow
ianayl Mar 10, 2025
162cba0
Change pip to install to user
ianayl Mar 10, 2025
848f741
Ignore system controlled python env
ianayl Mar 10, 2025
918604e
[CI] use realpaths when referring to SYCL
ianayl Mar 10, 2025
72d8730
[CI] use minimal preset when running benchmarks
ianayl Mar 10, 2025
066f5a6
[CI] Allow 2 bench scripts locations (#17394)
lukaszstolarczuk Mar 12, 2025
18e5291
add ulls compute benchmarks
pbalcer Mar 12, 2025
237750e
[CI][Benchmark] Decouple results from existing file structure, fetch …
ianayl Mar 11, 2025
ba1297f
[benchmark] Disabling UR test suites
ianayl Mar 12, 2025
cd6097f
update compute benchmarks and fix requirements
pbalcer Mar 13, 2025
c4e92c6
fix url updates
pbalcer Mar 13, 2025
ed8eecc
use timestamps in result file names
pbalcer Mar 13, 2025
130212d
add hostname to benchmark run
pbalcer Mar 13, 2025
a884df8
Merge branch 'sycl' of https://github.com/intel/llvm into unify-bench…
ianayl Mar 13, 2025
5323386
add SubmitGraph benchmark
pbalcer Mar 13, 2025
5bd1d56
Restore sycl-linux-run-tests benchmarking action
ianayl Mar 13, 2025
e9b1375
Restore old SYCL benchmarking CI
ianayl Mar 13, 2025
a3edf7a
Add benchmarking results to sycl-docs.yml
ianayl Mar 13, 2025
6620e4a
[CI] Bump compute bench (#17431)
lukaszstolarczuk Mar 13, 2025
f4a2e39
Initial implementation of unified benchmark workflow
ianayl Mar 13, 2025
5d3b0d9
Merge branch 'unify-benchmark-ci' of https://github.com/intel/llvm in…
ianayl Mar 13, 2025
38394bb
[CI] Use commit hash instead, fix issues with run
ianayl Mar 13, 2025
f232b93
add benchmark metadata
pbalcer Mar 14, 2025
30cd308
apply formatting
pbalcer Mar 14, 2025
5e0539a
fix multiple descriptions/notes
pbalcer Mar 14, 2025
137407a
fix benchmark descriptions
pbalcer Mar 14, 2025
e0f5ca6
fix remote html output
pbalcer Mar 14, 2025
1041db6
fix metadata collection with dry run
pbalcer Mar 14, 2025
fae04f4
cleanup compute bench, fix readme, use newer sycl-bench
pbalcer Mar 14, 2025
cfa4a9c
[CI] configure upload results
ianayl Mar 14, 2025
ca963e6
[CI] Change config to update during workflow run instead
ianayl Mar 14, 2025
45a02e1
[CI] Change save name depending on build
ianayl Mar 14, 2025
98f9d38
bump to 2024-2025
ianayl Mar 14, 2025
ef88ea0
[CI] Enforce commit hash to be string regardless
ianayl Mar 14, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
[CI] Allow 2 bench scripts locations (#17394)
On PRs based on main, the scripts location is "old" and not accesible.
Pick location based on the dir existance. Step 'gather info' is in
a 'weird' location, so solve it with 2 tries to execute the script.
lukaszstolarczuk authored Mar 12, 2025

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
commit 066f5a60c102669878188cc76532c0f57e2e55bc
23 changes: 20 additions & 3 deletions .github/workflows/ur-benchmarks-reusable.yml
Original file line number Diff line number Diff line change
@@ -80,13 +80,27 @@ jobs:
git checkout origin/pr/${{ inputs.pr_no }}/merge
git rev-parse origin/pr/${{ inputs.pr_no }}/merge
# TODO: As long as we didn't merge this workflow into main, we should allow both scripts location
- name: Establish bench scripts location
run: |
if [ -d "${{github.workspace}}/sycl-repo/devops/scripts/benchmarks" ]; then
echo "Bench scripts are in devops/scripts"
echo "BENCH_SCRIPTS_DIR=${{github.workspace}}/sycl-repo/devops/scripts/benchmarks" >> $GITHUB_ENV
elif [ -d "${{github.workspace}}/sycl-repo/unified-runtime/scripts/benchmarks" ]; then
echo "Bench scripts are in unified-runtime/scripts"
echo "BENCH_SCRIPTS_DIR=${{github.workspace}}/sycl-repo/unified-runtime/scripts/benchmarks" >> $GITHUB_ENV
else
echo "Bench scripts are absent...?"
exit 1
fi
- name: Create virtual environment
run: python -m venv .venv

- name: Activate virtual environment and install pip packages
run: |
source .venv/bin/activate
pip install -r ${{github.workspace}}/sycl-repo/devops/scripts/benchmarks/requirements.txt
pip install -r ${BENCH_SCRIPTS_DIR}/requirements.txt
- name: Configure SYCL
run: >
@@ -144,7 +158,7 @@ jobs:
id: benchmarks
run: >
source .venv/bin/activate &&
taskset -c "${{ env.CORES }}" ${{ github.workspace }}/sycl-repo/devops/scripts/benchmarks/main.py
taskset -c "${{ env.CORES }}" ${BENCH_SCRIPTS_DIR}/main.py
~/llvm_bench_workdir
--sycl ${{ github.workspace }}/sycl_build
--ur ${{ github.workspace }}/ur_install
@@ -198,6 +212,9 @@ jobs:
path: benchmark_results_${{ inputs.pr_no }}.html
key: benchmark-results-${{ inputs.pr_no }}-${{ matrix.adapter.str_name }}-${{ github.run_id }}

# TODO: As long as we didn't merge this workflow into main, we should allow both scripts location
- name: Get information about platform
if: ${{ always() }}
run: ${{github.workspace}}/sycl-repo/devops/scripts/get_system_info.sh
run: |
${{github.workspace}}/sycl-repo/devops/scripts/get_system_info.sh || true
${{github.workspace}}/sycl-repo/unified-runtime/.github/scripts/get_system_info.sh || true