From the CI flakiness handoff (§4f), which proposed:
A guard that every Target CI* in build/CITargets.cs appears in some workflow, or is explicitly listed as manual.
The motivation is real and well earned: SlowTests existed, was maintained, was cited in issue write-ups, and ran in zero CI jobs — nothing anywhere reported that.
The catch
I prototyped the obvious version and it produces a false positive on day one:
$ comm -23 <targets defined> <targets named in .github/workflows/*.yml>
CIMessageRouting
CIMessageRouting appears in no workflow file, and it runs on every push — via build/build.cs:
Target CI => _ => _
.DependsOn(CoreTests, CIMessageRouting);
and dotnet.yml runs ./build.sh ci.
A guard that cries wolf on its first run is how a signal channel gets ignored — which is the exact failure this whole reporting effort exists to undo, and is already called out in flakiness-report.sh as the reason NO_LEDGER_EXPECTED exists.
What it actually needs
Reachability, not name matching:
- Collect every
Target CI* declared in the build.
- Collect the target names each workflow invokes (
./build.sh <target>).
- Expand those through
DependsOn / Triggers transitively.
- Report targets not in the transitive closure, minus an explicit allow-list of deliberately-manual ones (
SlowTests is the precedent — it is workflow_dispatch-only by design).
Nuke can enumerate this itself; a hand-rolled grep of DependsOn would have the same brittleness as the naive version.
Priority
Low. This is a guard against a rare mistake, and it is only worth adding if it is accurate — an inaccurate version is worse than none.
From the CI flakiness handoff (§4f), which proposed:
The motivation is real and well earned:
SlowTestsexisted, was maintained, was cited in issue write-ups, and ran in zero CI jobs — nothing anywhere reported that.The catch
I prototyped the obvious version and it produces a false positive on day one:
CIMessageRoutingappears in no workflow file, and it runs on every push — viabuild/build.cs:and
dotnet.ymlruns./build.sh ci.A guard that cries wolf on its first run is how a signal channel gets ignored — which is the exact failure this whole reporting effort exists to undo, and is already called out in
flakiness-report.shas the reasonNO_LEDGER_EXPECTEDexists.What it actually needs
Reachability, not name matching:
Target CI*declared in the build../build.sh <target>).DependsOn/Triggerstransitively.SlowTestsis the precedent — it isworkflow_dispatch-only by design).Nuke can enumerate this itself; a hand-rolled grep of
DependsOnwould have the same brittleness as the naive version.Priority
Low. This is a guard against a rare mistake, and it is only worth adding if it is accurate — an inaccurate version is worse than none.