Skip to content

ci: add per-script timing to script-test target #2530

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
ralphbean merged 2 commits into from
Jun 24, 2026
Merged

ci: add per-script timing to script-test target #2530

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
1facad7
ci: add per-script timing to script-test target
ralphbean Jun 22, 2026
048f53d
fix(ci): propagate test exit code in run-timed macro
ralphbean Jun 24, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 19 additions & 11 deletions Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -105,18 +105,26 @@ go-tidy:
lint-md-links:
lychee --offline --no-progress --include-fragments --exclude-path node_modules --exclude-path experiments '**/*.md'

define run-timed

@fullsend-ai-review fullsend-ai-review Bot Jun 22, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[critical] error handling gap

The run-timed macro silently swallows test failures. The expanded shell command joins the test invocation ($(1)) and the timing bookkeeping with ; (semicolons), so the exit code of the entire recipe line is that of the last command (printf), which always succeeds. If a test script exits non-zero, Make will not see the failure, will not abort, and CI will pass green. In the original code, each test was its own recipe line whose exit code Make checked directly.

Suggested fix: Capture the exit code and re-exit with it after printing timing. For example: $(1); rc=$$?; elapsed=...; printf ...; exit $$rc

@start=$$(date +%s); \
rc=0; $(1) || rc=$$?; \
elapsed=$$(($$(date +%s) - $$start)); \
printf '::debug::script-test timing: %s completed in %ds\n' '$(1)' "$$elapsed"; \
exit $$rc
endef

script-test:
bash scripts/check-e2e-authorization-test.sh
bash internal/scaffold/fullsend-repo/scripts/post-triage-test.sh
bash internal/scaffold/fullsend-repo/scripts/post-prioritize-test.sh
bash internal/scaffold/fullsend-repo/scripts/post-code-test.sh
bash internal/scaffold/fullsend-repo/scripts/post-review-test.sh
bash internal/scaffold/fullsend-repo/scripts/reconcile-repos-test.sh
bash internal/scaffold/fullsend-repo/scripts/validate-output-schema-test.sh
bash internal/scaffold/fullsend-repo/scripts/pre-code-test.sh
bash internal/scaffold/fullsend-repo/scripts/pre-fetch-prior-review-test.sh
python3 internal/scaffold/fullsend-repo/scripts/process-fix-result-test.py
python3 skills/topissues/scripts/topissues_test.py
$(call run-timed,bash scripts/check-e2e-authorization-test.sh)
$(call run-timed,bash internal/scaffold/fullsend-repo/scripts/post-triage-test.sh)
$(call run-timed,bash internal/scaffold/fullsend-repo/scripts/post-prioritize-test.sh)
$(call run-timed,bash internal/scaffold/fullsend-repo/scripts/post-code-test.sh)
$(call run-timed,bash internal/scaffold/fullsend-repo/scripts/post-review-test.sh)
$(call run-timed,bash internal/scaffold/fullsend-repo/scripts/reconcile-repos-test.sh)
$(call run-timed,bash internal/scaffold/fullsend-repo/scripts/validate-output-schema-test.sh)
$(call run-timed,bash internal/scaffold/fullsend-repo/scripts/pre-code-test.sh)
$(call run-timed,bash internal/scaffold/fullsend-repo/scripts/pre-fetch-prior-review-test.sh)
$(call run-timed,python3 internal/scaffold/fullsend-repo/scripts/process-fix-result-test.py)
$(call run-timed,python3 skills/topissues/scripts/topissues_test.py)

test: lint-all go-test script-test

Expand Down
Loading