Repository navigation
ci: start the agent notification lane only for the suites it runs #13067
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
f30f2e1
73b9c5d
50299a8
5d05be4
b4f38c8
34ac86c
985c2a2
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -1292,6 +1292,62 @@ check_signing_intermediate_imports | |
| check_signing_intermediate_helper_behavior | ||
| check_sentry_cli_install_portability | ||
| check_sentry_cli_helper_behavior | ||
| check_agent_notification_paths_cover_its_suites() { | ||
| # The workflow reruns suites that ci.yml's shards already run, so it should | ||
| # start only for changes that can affect them: every file that defines one of | ||
| # its suites, and every helper file those name, must match a path trigger, | ||
| # and no cmuxTests trigger may match any other file. | ||
| ROOT_DIR="$ROOT_DIR" python3 - <<'PY' | ||
| import fnmatch, os, re, sys | ||
| from pathlib import Path | ||
|
|
||
| root = Path(os.environ["ROOT_DIR"]) | ||
| text = (root / ".github/workflows/agent-notification-tests.yml").read_text(encoding="utf-8") | ||
| paths = re.findall(r"^\s+- (cmuxTests/\S+)\s*$", text, flags=re.M) | ||
| suites = re.search(r"^\s*unit_test_suites:\s*(\S+)", text, flags=re.M).group(1).split(",") | ||
| errors = [] | ||
| if "cmuxTests/**" in paths: | ||
| errors.append("must not trigger on all of cmuxTests/**") | ||
| sources = {f: f.read_text(encoding="utf-8", errors="ignore") for f in sorted((root / "cmuxTests").glob("*.swift"))} | ||
| suite_files = set() | ||
| for suite in suites: | ||
| decl = re.compile(rf"^\s*(?:@\w+(?:\([^)]*\))?\s+)*(?:\w+\s+)*(?:class|struct|actor|extension)\s+{re.escape(suite)}\b", re.M) | ||
| files = [f for f, source in sources.items() if decl.search(source)] | ||
| if not files: | ||
| errors.append(f"runs {suite}, which no file in cmuxTests defines") | ||
| suite_files.update(files) | ||
| for f in files: | ||
| rel = f"cmuxTests/{f.name}" | ||
| if not any(fnmatch.fnmatchcase(rel, p) for p in paths): | ||
| errors.append(f"runs {suite} but {rel} matches no path trigger") | ||
| # A helper is a file whose top-level type the suite files name. Nested and | ||
| # private types are skipped: another file cannot reach them, and several test | ||
| # files declare a private type of the same name. | ||
| suite_text = "\n".join(sources[f] for f in suite_files) | ||
| helper_files = set() | ||
| top_level = re.compile(r"^(?:@\w+(?:\([^)]*\))?\s+)*(?:(?:final|internal|public|open)\s+)*(?:class|struct|enum|actor|protocol)\s+(\w+)", re.M) | ||
| for f, source in sources.items(): | ||
| if f in suite_files: | ||
| continue | ||
| used = sorted(n for n in set(top_level.findall(source)) if re.search(rf"\b{re.escape(n)}\b", suite_text)) | ||
|
Comment on lines
+1326
to
+1332
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🎯 Functional Correctness | 🟠 Major | 🏗️ Heavy lift 🔎 Supported by static analysis🏁 Script executed: sed -n '1280,1365p' tests/test_ci_self_hosted_guard.sh
sed -n '1,70p' .github/workflows/agent-notification-tests.yml
rg -n 'AgentHookTestNotificationPipeline|AgentJournalTestSupport|BundledCLILinkageTests|ClaudeHookLiveDeliveryTargetTestSupport|ClaudeHookSurfaceResolutionSwiftTests|ClaudeNotificationStatusLifecycleTests|TerminalControllingTTYWaiter|unit_test_suites' cmuxTests .github/workflows tests/test_ci_self_hosted_guard.shRepository: manaflow-ai/cmux Length of output: 18287 🏁 Script executed: set -eu
printf '%s\n' '--- workflow suite declaration and invocation ---'
sed -n '65,85p' .github/workflows/agent-notification-tests.yml
sed -n '145,175p' .github/workflows/test-depot.yml
printf '%s\n' '--- suite declarations and relevant support declarations/usages ---'
for f in \
cmuxTests/AgentNotificationLiveRetargetTests.swift \
cmuxTests/AgentJournalLifecycleCenterTests.swift \
cmuxTests/FeedWaiterRegistryTests.swift \
cmuxTests/ClaudeBackgroundWorkNotifyTests.swift \
cmuxTests/OpenCodeHookRegressionTests.swift \
cmuxTests/AgentHookTestNotificationPipeline.swift \
cmuxTests/AgentJournalTestSupport.swift \
cmuxTests/BundledCLILinkageTests.swift \
cmuxTests/ClaudeHookLiveDeliveryTargetTestSupport.swift \
cmuxTests/ClaudeHookSurfaceResolutionSwiftTests.swift \
cmuxTests/ClaudeNotificationStatusLifecycleTests.swift \
cmuxTests/TerminalControllingTTYWaiter.swift; do
echo "--- $f ---"
rg -n -C 2 '^(.*\b(class|struct|enum|actor|protocol|extension|typealias|func)\b)|AgentHookTestNotificationPipeline|AgentJournalTestSupport|BundledCLILinkageTests|ClaudeHookLiveDeliveryTargetTestSupport|ClaudeHookSurfaceResolutionSwiftTests|ClaudeNotificationStatusLifecycleTests|TerminalControllingTTYWaiter' "$f" || true
done
printf '%s\n' '--- possible support manifests/contracts ---'
rg -n -i 'agent-notification-tests|unit_test_suites|suite.*(support|helper)|support.*(suite|test)|dependency graph|manifest' .github tests cmuxTests Package.swift 2>/dev/null | head -250 || trueRepository: manaflow-ai/cmux Length of output: 50372 🏁 Script executed: set -eu
sed -n '65,85p' .github/workflows/agent-notification-tests.yml
sed -n '145,175p' .github/workflows/test-depot.yml
for f in cmuxTests/AgentNotificationLiveRetargetTests.swift cmuxTests/AgentJournalLifecycleCenterTests.swift cmuxTests/FeedWaiterRegistryTests.swift cmuxTests/ClaudeBackgroundWorkNotifyTests.swift cmuxTests/OpenCodeHookRegressionTests.swift cmuxTests/AgentHookTestNotificationPipeline.swift cmuxTests/AgentJournalTestSupport.swift cmuxTests/BundledCLILinkageTests.swift cmuxTests/ClaudeHookLiveDeliveryTargetTestSupport.swift cmuxTests/ClaudeHookSurfaceResolutionSwiftTests.swift cmuxTests/ClaudeNotificationStatusLifecycleTests.swift cmuxTests/TerminalControllingTTYWaiter.swift; do
echo "--- $f ---"
rg -n -C 2 '^(.*\b(class|struct|enum|actor|protocol|extension|typealias|func)\b)|AgentHookTestNotificationPipeline|AgentJournalTestSupport|BundledCLILinkageTests|ClaudeHookLiveDeliveryTargetTestSupport|ClaudeHookSurfaceResolutionSwiftTests|ClaudeNotificationStatusLifecycleTests|TerminalControllingTTYWaiter' "$f" || true
done
rg -n -i 'agent-notification-tests|unit_test_suites|suite.*(support|helper)|support.*(suite|test)|dependency graph|manifest' .github tests cmuxTests Package.swift 2>/dev/null | head -250 || trueRepository: manaflow-ai/cmux Length of output: 50372 🏁 Script executed: set -eu
for f in cmuxTests/ClaudeHookSurfaceSocketState.swift cmuxTests/ClaudeHookLiveDeliverySocketState.swift; do
echo "--- $f ---"
cat -n "$f"
done
printf '%s\n' '--- references to declarations from those files ---'
rg -n -C 2 'ClaudeHookSurfaceSocketState|ClaudeHookLiveDeliverySocketState|startClaudeSurfaceResolutionServer|ClaudeHookSurfaceResolutionSwiftTests\.' cmuxTests .github/workflows/agent-notification-tests.yml
printf '%s\n' '--- top-level declarations in candidate support files ---'
python3 - <<'PY'
from pathlib import Path
import re
files = [
Path("cmuxTests/ClaudeHookSurfaceSocketState.swift"),
Path("cmuxTests/ClaudeHookLiveDeliverySocketState.swift"),
Path("cmuxTests/AgentHookTestNotificationPipeline.swift"),
Path("cmuxTests/AgentJournalTestSupport.swift"),
Path("cmuxTests/ClaudeHookLiveDeliveryTargetTestSupport.swift"),
]
pat = re.compile(r"^(?:@\w+(?:\([^)]*\))?\s+)*(?:(?:final|internal|public|open)\s+)*(?:class|struct|enum|actor|protocol|extension|typealias|func)\b")
for p in files:
print(p)
for i, line in enumerate(p.read_text(encoding="utf-8", errors="ignore").splitlines(), 1):
if pat.search(line):
print(f"{i}:{line}")
PYRepository: manaflow-ai/cmux Length of output: 11725 Include extension and transitive helper files in coverage.
The same gap applies to transitive nominal dependencies such as 🤖 Prompt for AI Agents |
||
| rel = f"cmuxTests/{f.name}" | ||
| if used: | ||
| helper_files.add(f) | ||
| if used and not any(fnmatch.fnmatchcase(rel, p) for p in paths): | ||
| errors.append(f"suites use {', '.join(used)} from {rel}, which matches no path trigger") | ||
| # A trigger that also matches unrelated test files starts a second run of | ||
| # suites that ci.yml already ran. | ||
| for p in paths: | ||
| extra = sorted(f.name for f in sources if fnmatch.fnmatchcase(f"cmuxTests/{f.name}", p) and f not in suite_files | helper_files) | ||
| if extra: | ||
| errors.append(f"trigger {p} also matches unrelated files: {', '.join(extra[:5])}") | ||
| for e in errors: | ||
| print(f"FAIL: agent-notification-tests.yml {e}") | ||
| sys.exit(1 if errors else 0) | ||
| PY | ||
| echo "PASS: agent notification paths cover every suite file and helper the workflow runs" | ||
| } | ||
|
|
||
| pr_workflow_events() { | ||
| # Prints the pull request events a workflow triggers on, for the mapping, | ||
| # list and scalar forms of `on:`. | ||
|
|
@@ -1432,5 +1488,6 @@ check_no_ci_swift_package_skips | |
| check_web_db_behavior_tests | ||
| check_web_test_runner_behavior | ||
| check_tmux_terminal_nightly_isolation | ||
| check_agent_notification_paths_cover_its_suites | ||
| check_pr_macos_workflows_cancel_superseded_runs | ||
| check_no_paid_overflow_fallbacks | ||
Uh oh!
There was an error while loading. Please reload this page.