Skip to content

Commit

Permalink
Github Action to prevent large files (#1478)
Browse files Browse the repository at this point in the history
Add CI step to check for too-large files and lfs files

---------

Co-authored-by: Emil Ernerfeldt <[email protected]>
  • Loading branch information
Wumpf and emilk authored Mar 2, 2023
1 parent ded471b commit a0fb0ca
Show file tree
Hide file tree
Showing 4 changed files with 29 additions and 0 deletions.
5 changes: 5 additions & 0 deletions .github/workflows/misc.yml
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
name: CI (Misc)

on:
workflow_dispatch:
pull_request:
push:
branches:
Expand All @@ -25,3 +26,7 @@ jobs:
- name: Rerun lints
run: |
./scripts/lint.py
- name: Check for too large files
run: |
./scripts/check_large_files.sh
Binary file removed rerun_py/rerun_sdk/rerun_demo/demo.rrd
Binary file not shown.
22 changes: 22 additions & 0 deletions scripts/check_large_files.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
#!/usr/bin/env bash
# Check for files that are too large to be checked into the repository.
# Whenever we want to make an exception, we add it to `check_large_files_allow_list.txt`
set -eu

# Maximum file size, unless found in `check_large_files_allow_list.txt`
maximum_size=$((100 * 1024))

result=0
while read -d '' -r file; do
if [[ -f "$file" ]]; then
actualsize=$(wc -c <"$file")
if [ $actualsize -ge $maximum_size ]; then
if ! grep -qx "$file" ./scripts/check_large_files_allow_list.txt; then
echo "$file is $actualsize byte (max allowed is $maximum_size bytes)"
result=1
fi
fi
fi
done < <(git ls-files -z --empty-directory)

exit $result
2 changes: 2 additions & 0 deletions scripts/check_large_files_allow_list.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
Cargo.lock
crates/re_ui/data/Inter-Medium.otf

1 comment on commit a0fb0ca

@github-actions
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rust Benchmark

Benchmark suite Current: a0fb0ca Previous: ded471b Ratio
datastore/insert/batch/rects/insert 564249 ns/iter (± 4210) 560153 ns/iter (± 2493) 1.01
datastore/latest_at/batch/rects/query 1819 ns/iter (± 5) 1813 ns/iter (± 5) 1.00
datastore/latest_at/missing_components/primary 358 ns/iter (± 0) 358 ns/iter (± 0) 1
datastore/latest_at/missing_components/secondaries 425 ns/iter (± 0) 432 ns/iter (± 0) 0.98
datastore/range/batch/rects/query 153310 ns/iter (± 706) 152588 ns/iter (± 690) 1.00
mono_points_arrow/generate_message_bundles 52568854 ns/iter (± 926249) 49007145 ns/iter (± 428359) 1.07
mono_points_arrow/generate_messages 137612222 ns/iter (± 1352913) 125621023 ns/iter (± 945503) 1.10
mono_points_arrow/encode_log_msg 167268801 ns/iter (± 1330200) 154906616 ns/iter (± 3041639) 1.08
mono_points_arrow/encode_total 356435522 ns/iter (± 1532350) 331984683 ns/iter (± 3924046) 1.07
mono_points_arrow/decode_log_msg 190792695 ns/iter (± 1147469) 178070782 ns/iter (± 826853) 1.07
mono_points_arrow/decode_message_bundles 74630080 ns/iter (± 1440190) 65504074 ns/iter (± 916827) 1.14
mono_points_arrow/decode_total 264594093 ns/iter (± 1962100) 240400412 ns/iter (± 2178358) 1.10
batch_points_arrow/generate_message_bundles 334635 ns/iter (± 888) 331360 ns/iter (± 534) 1.01
batch_points_arrow/generate_messages 6321 ns/iter (± 14) 6208 ns/iter (± 15) 1.02
batch_points_arrow/encode_log_msg 359055 ns/iter (± 2909) 358771 ns/iter (± 1082) 1.00
batch_points_arrow/encode_total 721966 ns/iter (± 5000) 711574 ns/iter (± 5239) 1.01
batch_points_arrow/decode_log_msg 348045 ns/iter (± 989) 346432 ns/iter (± 926) 1.00
batch_points_arrow/decode_message_bundles 2106 ns/iter (± 11) 2098 ns/iter (± 7) 1.00
batch_points_arrow/decode_total 356364 ns/iter (± 2581) 351904 ns/iter (± 791) 1.01
arrow_mono_points/insert 7133946785 ns/iter (± 87417295) 6010499935 ns/iter (± 15452527) 1.19
arrow_mono_points/query 1749226 ns/iter (± 36986) 1733052 ns/iter (± 12358) 1.01
arrow_batch_points/insert 2677925 ns/iter (± 11884) 2622047 ns/iter (± 19730) 1.02
arrow_batch_points/query 16002 ns/iter (± 14) 16010 ns/iter (± 9) 1.00
arrow_batch_vecs/insert 41883 ns/iter (± 111) 41815 ns/iter (± 142) 1.00
arrow_batch_vecs/query 506006 ns/iter (± 1402) 505516 ns/iter (± 378) 1.00
tuid/Tuid::random 34 ns/iter (± 0) 34 ns/iter (± 0) 1

This comment was automatically generated by workflow using github-action-benchmark.

Please sign in to comment.