Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Catch documentation and examples errors #736

Merged
merged 14 commits into from
Nov 15, 2023
Merged

Conversation

rafmudaf
Copy link
Collaborator

@rafmudaf rafmudaf commented Nov 10, 2023

Improve CI configuration to abort when documentation build errors and test for repo dependency in Examples

This pull request adds a setting to the documentation configuration that will stop the build when a notebook within the docs has an error in any code cell. Currently, the documentation is configured to proceed with building the docs even if a notebook errors. This was made clear recently during a FLORIS workshop where errors in the documentation were a surprise because GitHub Actions had not failed.

Another change included here runs the examples in a directory outside of the top-level repository directory. This tests for whether any feature described in the examples is relying on the directory structure of the repository. For example, see #737.

Note accepting this pull request means the documentation CI pipeline will fail. This is expected and must be resolved through other fixes to the documentation ASAP. See #733 for more info.

Additionally, I fixed spelling errors in docstrings throughout the software, and addressed a few warnings from Jupyter Book and Sphinx. Other warnings from autodoc and auto summary Sphinx extensions are suppressed since they are a consequence of autogenerating the API documentation.

Related issue

#733
#737

Impacted areas of the software

Docs
Automated check for running the examples

@rafmudaf rafmudaf added bug Something isn't working enhancement An improvement of an existing feature documentation labels Nov 10, 2023
@rafmudaf rafmudaf added this to the v3.6 milestone Nov 10, 2023
@rafmudaf rafmudaf self-assigned this Nov 10, 2023
This ensures there’s no reliance on the repo struture to find source files
@rafmudaf rafmudaf changed the title Stop documentation build on notebook error Catch documentation and examples errors Nov 11, 2023
@paulf81
Copy link
Collaborator

paulf81 commented Nov 13, 2023

This seems like a good change to make to be made aware of any broken notebooks, support!

@rafmudaf rafmudaf requested review from RHammond2 and removed request for bayc and misi9170 November 14, 2023 18:16
docs/_config.yml Outdated
Comment on lines 54 to 55
nb_execution_show_tb: True # Shows the stack trace in stdout; its suppressed otherwise
nb_execution_raise_on_error: true # Stops the Sphinx build if there is an error in a notebook. See https://github.com/executablebooks/jupyter-book/issues/2011
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These configurations should be outside the config, but still under sphinx, and have lowercase true as the value. However, this doesn't break the build process.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Per our conversation a few moments ago, it seems that this is heavily dependent on the version of jupyter-book for where these settings should live. However, neither version solved the issue where it doesn't break the build process.

Jupyter Book v0.14 supports setting this:
nb_execution_show_tb: true
nb_execution_raise_on_error: true

And sphinx-book-theme 0.4.0rc1 is the required version for JupyterBook v0.14. Note that there is an incompatibility with FLORIS docs and JupyterBook v0.15, sphinx-book-theme v1.0.
Mostly just to make the docs build on GitHub to test that it exits on error
@rafmudaf
Copy link
Collaborator Author

@RHammond2 I've pinned the Jupyter Book version to v0.14.0 since that one supports the nb_execution_show_tb and nb_execution_raise_on_error settings while v0.13 does not. FWIW I couldn't actually verify that in the code that 0.13 does not support those, but I found this line changed in v0.14 and my own testing verifies this.

This change now catches any errors in notebook execution during the documentation build process. You can see the failing Actions job on my fork.

Copy link
Collaborator

@RHammond2 RHammond2 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good to me given that the failure is breaking the build now!

@rafmudaf rafmudaf merged commit 1a2e868 into NREL:develop Nov 15, 2023
5 of 8 checks passed
@rafmudaf rafmudaf mentioned this pull request Nov 15, 2023
4 tasks
@rafmudaf rafmudaf deleted the docs_bug_fixes branch November 15, 2023 21:28
@misi9170 misi9170 mentioned this pull request Apr 5, 2024
4 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working documentation enhancement An improvement of an existing feature
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants