Skip to content

ci: tolerate a SonarQube Cloud outage instead of failing the build - #395

Merged
matt-edmondson merged 1 commit into
mainfrom
claude/happy-rubin-w67sx1
Sep 13, 2026
Merged

matt-edmondson merged 1 commit into
mainfrom
claude/happy-rubin-w67sx1

Conversation

@matt-edmondson

Copy link
Copy Markdown
Contributor

SonarCloud is a third party, and its scanner fails in pre-processing — before a single project is compiled or a single test runs. A total outage therefore turns every pull request red while saying nothing at all about the change under review. That is a false signal, not a quality gate.

What changed

A Probe SonarQube Cloud step runs before the begin pass and asks sonarcloud.io whether it is there. Three attempts with backoff, because the point is to tell an outage from a blip — analysis is worth having, and one slow response should not cost a run its quality gate. Begin SonarQube and End SonarQube are both gated on the answer.

The end pass gets the same judgement for an outage that starts after the probe. If the upload fails, the server is re-checked: forgiven only when it is genuinely unreachable, so a malformed report, a bad token or a rejected analysis still fails the step.

Where the gate is blocking, an outage still fails

SONAR_BLOCKING_GATE opts a repository into sonar.qualitygate.wait=true. None of the tolerance above applies there:

  • The probe exits 1 rather than skipping. The Release step names no status function in its if:, so it is implicitly gated on the steps before it succeeding — and a skipped step is not a failed one. Forgiving an outage would release past the very gate the repository opted into.
  • The end pass is never forgiven either, because with qualitygate.wait=true a failed gate is one of the ways that command exits non-zero, and an unreachable server is not evidence about which one happened.

Skipping is loud, and never dressed up as a pass

No quality gate is produced when analysis is skipped, so the SonarCloud check simply does not report — it is never made to look as though it passed. A warning annotation and a job summary say why, and a branch rule that requires the check still holds the merge until a human decides otherwise.

How this was checked

Both step scripts were parsed with the PowerShell parser and then exercised under pwsh 7.4 outside CI, capturing the process exit code rather than a pipeline's:

case expected observed
probe, server reachable available=true, exit 0, no summary
probe, unreachable, advisory gate available=false, warning, summary, exit 0
probe, unreachable, blocking gate available=false, ::error, exit 1
end, scanner succeeds exit 0, silent
end, scanner fails, blocking gate exit 1, no annotation
end, scanner fails, server answering ::error, exit 1
end, scanner fails, server unreachable ::warning, summary, exit 0

One thing worth confirming, because it is the load-bearing assumption of the forgiving branch: a leftover non-zero $LASTEXITCODE from the scanner does not leak into the exit code of pwsh -Command ". 'script'", which is how a shell: pwsh step is invoked. Tested directly with a native command exiting 1 — the step exits 0.

Note on where this file comes from

.github/workflows/dotnet.yml is the centrally-synced workflow, so this change will be overwritten the next time it syncs unless the same edit is carried to the central source. The equivalent change for ImageGui's own dotnet-private.yaml is in ktsu-dev/ImageGui#29.

🤖 Generated with Claude Code

https://claude.ai/code/session_015qxqZVzN8CJcDxTb5gtWua


Generated by Claude Code

…atch]

SonarCloud is a third party and its scanner fails in pre-processing, before a
single project is compiled or a test runs. A total outage therefore turns every
pull request red while saying nothing at all about the change under review, which
is a false signal rather than a quality gate.

Probe sonarcloud.io before the begin pass — three attempts, so an outage is told
apart from a blip — and skip both Sonar steps when it does not answer. The end
pass gets the same judgement for an outage that starts after the probe: if the
upload fails, re-check the server and forgive the failure only when the server is
genuinely unreachable, so a malformed report, a bad token or a rejected analysis
still fails the step.

Where SONAR_BLOCKING_GATE is set, none of this applies and an outage still fails
the run. The Release step is implicitly gated on the steps before it succeeding
and a skipped step is not a failed one, so forgiving an outage in a repository
that opted into a blocking gate would release past the very gate it opted into.
For the same reason the end pass is never forgiven under a blocking gate: with
sonar.qualitygate.wait=true, a failed gate is one of the ways it exits non-zero.

Skipping is otherwise deliberately loud, and never dressed up as a pass: no
quality gate is produced, so the SonarCloud check does not report at all, a
warning annotation and a job summary say why, and a branch rule that requires the
check still holds the merge until a human decides otherwise.

Both step scripts were parsed and exercised under pwsh 7.4 outside CI, checking
the process exit code rather than a pipeline's: the probe reachable, unreachable
under an advisory gate and unreachable under a blocking gate; the end pass across
success, a failure under a blocking gate, a genuine analysis failure and a
mid-run outage.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015qxqZVzN8CJcDxTb5gtWua
@matt-edmondson
matt-edmondson merged commit 1b844b8 into main Sep 13, 2026
13 of 14 checks passed
@matt-edmondson
matt-edmondson deleted the claude/happy-rubin-w67sx1 branch September 13, 2026 08:06
matt-edmondson pushed a commit to ktsu-dev/Invoker that referenced this pull request Sep 13, 2026
…request [patch]

Observed live on ktsu-dev/ImGuiApp#395, where SONAR_BLOCKING_GATE is set: a
SonarCloud 503 failed the pull request on all three probe attempts. The probe
did exactly what it was written to do, but what it was written to do was wrong
-- that is precisely the false signal it was added to remove.

A pull request cannot publish, so failing it buys nothing. A required
SonarCloud check still holds the merge, because a skipped analysis reports no
gate at all. So an outage is now forgiven on a pull request whatever the gate
setting, and only a run that could publish is held to a blocking gate.

What must not happen is a release going out ungated, and that was previously
only prevented as a side effect of the whole job failing -- which does not
cover a gate that never happened, since a skipped step is not a failed one.
The Release step now names the two Sonar outputs explicitly: where the gate is
blocking, it needs a probe that found the server and an upload that was not
forgiven. End SonarQube publishes that second fact as a new `analysed` output.

The probe also moves ahead of the two Sonar caches and the scanner install, and
those three are gated on its answer, so an outage no longer pays for a cache
restore and a `dotnet tool install` it is about to throw away.

Re-exercised under pwsh 7.4 outside CI across nine cases, checking the process
exit code and the step outputs: the probe reachable, and unreachable under an
advisory gate, a blocking gate on a pull request, and a blocking gate on a
push; the end pass on success, a blocking-gate push failure, a blocking-gate
pull request, a genuine analysis failure and a mid-run outage.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015qxqZVzN8CJcDxTb5gtWua
matt-edmondson pushed a commit to ktsu-dev/KtsuBuild that referenced this pull request Sep 13, 2026
…request [patch]

Observed live on ktsu-dev/ImGuiApp#395, where SONAR_BLOCKING_GATE is set: a
SonarCloud 503 failed the pull request on all three probe attempts. The probe
did exactly what it was written to do, but what it was written to do was wrong
-- that is precisely the false signal it was added to remove.

A pull request cannot publish, so failing it buys nothing. A required
SonarCloud check still holds the merge, because a skipped analysis reports no
gate at all. So an outage is now forgiven on a pull request whatever the gate
setting, and only a run that could publish is held to a blocking gate.

What must not happen is a release going out ungated, and that was previously
only prevented as a side effect of the whole job failing -- which does not
cover a gate that never happened, since a skipped step is not a failed one.
The Release step now names the two Sonar outputs explicitly: where the gate is
blocking, it needs a probe that found the server and an upload that was not
forgiven. End SonarQube publishes that second fact as a new `analysed` output.

The probe also moves ahead of the two Sonar caches and the scanner install, and
those three are gated on its answer, so an outage no longer pays for a cache
restore and a `dotnet tool install` it is about to throw away.

Re-exercised under pwsh 7.4 outside CI across nine cases, checking the process
exit code and the step outputs: the probe reachable, and unreachable under an
advisory gate, a blocking gate on a pull request, and a blocking gate on a
push; the end pass on success, a blocking-gate push failure, a blocking-gate
pull request, a genuine analysis failure and a mid-run outage.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015qxqZVzN8CJcDxTb5gtWua
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants