ci: tolerate a SonarQube Cloud outage instead of failing the build - #395
Merged
Merged
Conversation
…atch] SonarCloud is a third party and its scanner fails in pre-processing, before a single project is compiled or a test runs. A total outage therefore turns every pull request red while saying nothing at all about the change under review, which is a false signal rather than a quality gate. Probe sonarcloud.io before the begin pass — three attempts, so an outage is told apart from a blip — and skip both Sonar steps when it does not answer. The end pass gets the same judgement for an outage that starts after the probe: if the upload fails, re-check the server and forgive the failure only when the server is genuinely unreachable, so a malformed report, a bad token or a rejected analysis still fails the step. Where SONAR_BLOCKING_GATE is set, none of this applies and an outage still fails the run. The Release step is implicitly gated on the steps before it succeeding and a skipped step is not a failed one, so forgiving an outage in a repository that opted into a blocking gate would release past the very gate it opted into. For the same reason the end pass is never forgiven under a blocking gate: with sonar.qualitygate.wait=true, a failed gate is one of the ways it exits non-zero. Skipping is otherwise deliberately loud, and never dressed up as a pass: no quality gate is produced, so the SonarCloud check does not report at all, a warning annotation and a job summary say why, and a branch rule that requires the check still holds the merge until a human decides otherwise. Both step scripts were parsed and exercised under pwsh 7.4 outside CI, checking the process exit code rather than a pipeline's: the probe reachable, unreachable under an advisory gate and unreachable under a blocking gate; the end pass across success, a failure under a blocking gate, a genuine analysis failure and a mid-run outage. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015qxqZVzN8CJcDxTb5gtWua
matt-edmondson
pushed a commit
to ktsu-dev/Invoker
that referenced
this pull request
Sep 13, 2026
…request [patch] Observed live on ktsu-dev/ImGuiApp#395, where SONAR_BLOCKING_GATE is set: a SonarCloud 503 failed the pull request on all three probe attempts. The probe did exactly what it was written to do, but what it was written to do was wrong -- that is precisely the false signal it was added to remove. A pull request cannot publish, so failing it buys nothing. A required SonarCloud check still holds the merge, because a skipped analysis reports no gate at all. So an outage is now forgiven on a pull request whatever the gate setting, and only a run that could publish is held to a blocking gate. What must not happen is a release going out ungated, and that was previously only prevented as a side effect of the whole job failing -- which does not cover a gate that never happened, since a skipped step is not a failed one. The Release step now names the two Sonar outputs explicitly: where the gate is blocking, it needs a probe that found the server and an upload that was not forgiven. End SonarQube publishes that second fact as a new `analysed` output. The probe also moves ahead of the two Sonar caches and the scanner install, and those three are gated on its answer, so an outage no longer pays for a cache restore and a `dotnet tool install` it is about to throw away. Re-exercised under pwsh 7.4 outside CI across nine cases, checking the process exit code and the step outputs: the probe reachable, and unreachable under an advisory gate, a blocking gate on a pull request, and a blocking gate on a push; the end pass on success, a blocking-gate push failure, a blocking-gate pull request, a genuine analysis failure and a mid-run outage. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015qxqZVzN8CJcDxTb5gtWua
matt-edmondson
pushed a commit
to ktsu-dev/KtsuBuild
that referenced
this pull request
Sep 13, 2026
…request [patch] Observed live on ktsu-dev/ImGuiApp#395, where SONAR_BLOCKING_GATE is set: a SonarCloud 503 failed the pull request on all three probe attempts. The probe did exactly what it was written to do, but what it was written to do was wrong -- that is precisely the false signal it was added to remove. A pull request cannot publish, so failing it buys nothing. A required SonarCloud check still holds the merge, because a skipped analysis reports no gate at all. So an outage is now forgiven on a pull request whatever the gate setting, and only a run that could publish is held to a blocking gate. What must not happen is a release going out ungated, and that was previously only prevented as a side effect of the whole job failing -- which does not cover a gate that never happened, since a skipped step is not a failed one. The Release step now names the two Sonar outputs explicitly: where the gate is blocking, it needs a probe that found the server and an upload that was not forgiven. End SonarQube publishes that second fact as a new `analysed` output. The probe also moves ahead of the two Sonar caches and the scanner install, and those three are gated on its answer, so an outage no longer pays for a cache restore and a `dotnet tool install` it is about to throw away. Re-exercised under pwsh 7.4 outside CI across nine cases, checking the process exit code and the step outputs: the probe reachable, and unreachable under an advisory gate, a blocking gate on a pull request, and a blocking gate on a push; the end pass on success, a blocking-gate push failure, a blocking-gate pull request, a genuine analysis failure and a mid-run outage. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015qxqZVzN8CJcDxTb5gtWua
This was referenced Sep 13, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
SonarCloud is a third party, and its scanner fails in pre-processing — before a single project is compiled or a single test runs. A total outage therefore turns every pull request red while saying nothing at all about the change under review. That is a false signal, not a quality gate.
What changed
A
Probe SonarQube Cloudstep runs before the begin pass and askssonarcloud.iowhether it is there. Three attempts with backoff, because the point is to tell an outage from a blip — analysis is worth having, and one slow response should not cost a run its quality gate.Begin SonarQubeandEnd SonarQubeare both gated on the answer.The end pass gets the same judgement for an outage that starts after the probe. If the upload fails, the server is re-checked: forgiven only when it is genuinely unreachable, so a malformed report, a bad token or a rejected analysis still fails the step.
Where the gate is blocking, an outage still fails
SONAR_BLOCKING_GATEopts a repository intosonar.qualitygate.wait=true. None of the tolerance above applies there:Releasestep names no status function in itsif:, so it is implicitly gated on the steps before it succeeding — and a skipped step is not a failed one. Forgiving an outage would release past the very gate the repository opted into.qualitygate.wait=truea failed gate is one of the ways that command exits non-zero, and an unreachable server is not evidence about which one happened.Skipping is loud, and never dressed up as a pass
No quality gate is produced when analysis is skipped, so the SonarCloud check simply does not report — it is never made to look as though it passed. A warning annotation and a job summary say why, and a branch rule that requires the check still holds the merge until a human decides otherwise.
How this was checked
Both step scripts were parsed with the PowerShell parser and then exercised under pwsh 7.4 outside CI, capturing the process exit code rather than a pipeline's:
available=true, exit 0, no summaryavailable=false, warning, summary, exit 0available=false,::error, exit 1::error, exit 1::warning, summary, exit 0One thing worth confirming, because it is the load-bearing assumption of the forgiving branch: a leftover non-zero
$LASTEXITCODEfrom the scanner does not leak into the exit code ofpwsh -Command ". 'script'", which is how ashell: pwshstep is invoked. Tested directly with a native command exiting 1 — the step exits 0.Note on where this file comes from
.github/workflows/dotnet.ymlis the centrally-synced workflow, so this change will be overwritten the next time it syncs unless the same edit is carried to the central source. The equivalent change for ImageGui's owndotnet-private.yamlis in ktsu-dev/ImageGui#29.🤖 Generated with Claude Code
https://claude.ai/code/session_015qxqZVzN8CJcDxTb5gtWua
Generated by Claude Code