Skip to content

Fix CI failures caused by docs build memory exhaustion#2580

Merged
pankajkoti merged 4 commits into
mainfrom
separate-out-docs-build
Apr 20, 2026
Merged

Fix CI failures caused by docs build memory exhaustion#2580
pankajkoti merged 4 commits into
mainfrom
separate-out-docs-build

Conversation

@pankajkoti
Copy link
Copy Markdown
Contributor

The Sphinx docs build had outgrown the memory available on
pre-commit.ci runners and was failing consistently on every pull
request, including ones that touched no documentation at all. The
result was a red pre-commit.ci check on unrelated PRs, blocked merges,
time lost to investigating failures that had nothing to do with the
change under review, and an erosion of trust in required checks.

Running the docs build on GitHub Actions restores that trust. PRs
that do not touch docs or library sources now pass cleanly. PRs that
do touch them get a reliable pass/fail signal with enough memory to
actually complete the build, so warnings surface where they belong
rather than being masked by OOM failures. pre-commit.ci is left to
focus on what it is good at, fast lint and format checks, and
reviewers and contributors get back the signal they rely on to ship
changes confidently.

related: #2578
related: #2579

pankajkoti and others added 2 commits April 20, 2026 13:28
The Sphinx docs build had outgrown the memory available on
pre-commit.ci runners and was failing consistently on every pull
request, including ones that touched no documentation at all. The
result was a red pre-commit.ci check on unrelated PRs, blocked merges,
time lost to investigating failures that had nothing to do with the
change under review, and an erosion of trust in required checks.

Running the docs build on GitHub Actions restores that trust. PRs
that do not touch docs or library sources now pass cleanly. PRs that
do touch them get a reliable pass/fail signal with enough memory to
actually complete the build, so warnings surface where they belong
rather than being masked by OOM failures. pre-commit.ci is left to
focus on what it is good at, fast lint and format checks, and
reviewers and contributors get back the signal they rely on to ship
changes confidently.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Revert before merging to main. This lets the workflow exercise on a
push to this branch so we can confirm the Sphinx build succeeds on
GitHub Actions before the fix reaches main.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Comment thread .github/workflows/docs-build.yml Outdated
@pankajkoti
Copy link
Copy Markdown
Contributor Author

@pankajkoti pankajkoti marked this pull request as ready for review April 20, 2026 08:06
Copilot AI review requested due to automatic review settings April 20, 2026 08:06
@pankajkoti pankajkoti requested review from a team and corsettigyg as code owners April 20, 2026 08:06
@pankajkoti pankajkoti self-assigned this Apr 20, 2026
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Moves the Sphinx docs build out of pre-commit.ci (where it was OOM’ing) into a dedicated GitHub Actions workflow so docs builds run with adequate resources and only when docs/library sources change.

Changes:

  • Remove the local check-docs-build hook from .pre-commit-config.yaml.
  • Add a new .github/workflows/docs-build.yml workflow to install docs deps and run sphinx-build.
  • Update .github/workflows/test.yml changed-files filter to treat docs-build.yml as non-code (so it won’t trigger full test runs by itself).

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.

File Description
.pre-commit-config.yaml Removes the Sphinx build hook that was exhausting memory on pre-commit.ci.
.github/workflows/test.yml Ensures edits to the new docs-build workflow don’t force the full test matrix to run.
.github/workflows/docs-build.yml Introduces a dedicated CI docs build job for PRs/changes impacting docs or cosmos/.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread .github/workflows/docs-build.yml Outdated
Comment thread .github/workflows/docs-build.yml
Comment thread .github/workflows/docs-build.yml
@codecov
Copy link
Copy Markdown

codecov Bot commented Apr 20, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 98.05%. Comparing base (37fd06c) to head (c3d9af7).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #2580   +/-   ##
=======================================
  Coverage   98.05%   98.05%           
=======================================
  Files         103      103           
  Lines        7601     7601           
=======================================
  Hits         7453     7453           
  Misses        148      148           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings April 20, 2026 08:53
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated no new comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copy link
Copy Markdown
Collaborator

@tatiana tatiana left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for fixing this, @pankajkoti !

@pankajkoti pankajkoti merged commit 49b23f9 into main Apr 20, 2026
130 checks passed
@pankajkoti pankajkoti deleted the separate-out-docs-build branch April 20, 2026 09:21
@pankajkoti pankajkoti mentioned this pull request Apr 23, 2026
@tatiana tatiana added this to the Cosmos 1.14.1 milestone Apr 23, 2026
tatiana pushed a commit that referenced this pull request Apr 23, 2026
Bug Fixes

* Fix ``ExecutionMode.WATCHER`` producer retry behaviour by @tatiana in
#2559
* Prevent watcher producer skip propagating to downstream tasks via
gateway task by @johnhoran and @tatiana in #2597
* Keep watcher sensor polling when producer is still running by
@pankajkoti in #2592
* Fix circular import error in Cosmos plugin discovery under Astro
Runtime by @tatiana in #2538
* Fix ``CosmosRichLogger`` crash on ``None`` log message by @tatiana in
#2540
* Enable inlets and outlets using dbt Fusion on Airflow 3 by
@ichirotakami in #2561
* Fix incorrectly skipped source downstream tasks in
``ExecutionMode.WATCHER`` by @pankajastro in #2563
* Fix duplicate logs in ``dbt build`` when source freshness is enabled
by @pankajastro in #2564
* Warn and normalize when ``source_rendering_behavior=None`` is passed
by @pankajastro in #2570
* Gracefully handle ``Variable.set()`` failures on Astro Remote
Execution by @hkc-8010 in #2573
* Skip malformed YAML selectors instead of failing entirely by
@YourRoyalLinus in #2577

Docs

* Update watcher test behavior docs for Cosmos 1.14.0 by @tatiana in
#2549
* Add redirect for moved partial-parsing docs page by @tatiana in #2550
* Document ``ExecutionMode.WATCHER`` and ``depends_on_past`` limitation
by @tatiana in #2602
* Restore memory-optimised imports docs for Cosmos < 1.14.0 by
@pankajkoti in #2604

Others

* Speed up Airflow 3.1+ integration tests by caching
InProcessExecutionAPI by @pankajkoti in #2547
* Improve stability of cache hash unit tests by @tatiana in #2539
* Fix mypy 1.20.0 type check failures by @pankajkoti in #2546
* Fix CI failures caused by docs build memory exhaustion by @pankajkoti
in #2580
* Fix dbt Fusion broken integration tests by @tatiana in #2581
* Fix flaky ``cosmos_manifest_selectors_example`` DAG in CI by
@pankajkoti in #2593
* Reduce pre-commit autoupdate frequency PRs by @tatiana in #2544
* Bump ``reviewdog/action-actionlint`` from 1.71.0 to 1.72.0 by
@dependabot in #2542
* Skip watcher gateway test on Airflow 3.0 by @tatiana in #2607

closes: astronomer/oss-integrations-private#381
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants