Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Notebooks failing in readthedocs build does not trigger red status #799

Closed
AdamGleave opened this issue Oct 4, 2023 · 2 comments · Fixed by #803
Closed

Notebooks failing in readthedocs build does not trigger red status #799

AdamGleave opened this issue Oct 4, 2023 · 2 comments · Fixed by #803
Assignees
Labels
bug Something isn't working

Comments

@AdamGleave
Copy link
Member

Bug description

Running notebooks is currently failing in the readthedocs build but as a warning only so CI wrongly looks green.

Steps to reproduce

From the logs, there are a variety of failures that are warnings only:

/home/docs/checkouts/readthedocs.org/user_builds/imitation/checkouts/latest/docs/tutorials/3_train_gail.ipynb: WARNING: Executing notebook failed: CellTimeoutError [mystnb.exec]
/home/docs/checkouts/readthedocs.org/user_builds/imitation/checkouts/latest/docs/tutorials/3_train_gail.ipynb: WARNING: Notebook exception traceback saved in: /home/docs/checkouts/readthedocs.org/user_builds/imitation/checkouts/latest/_readthedocs/html/reports/tutorials/3_train_gail.err.log [mystnb.exec]
/home/docs/checkouts/readthedocs.org/user_builds/imitation/checkouts/latest/docs/tutorials/4_train_airl.ipynb: WARNING: Executing notebook failed: CellExecutionError [mystnb.exec]
/home/docs/checkouts/readthedocs.org/user_builds/imitation/checkouts/latest/docs/tutorials/4_train_airl.ipynb: WARNING: Notebook exception traceback saved in: /home/docs/checkouts/readthedocs.org/user_builds/imitation/checkouts/latest/_readthedocs/html/reports/tutorials/4_train_airl.err.log [mystnb.exec]
/home/docs/checkouts/readthedocs.org/user_builds/imitation/checkouts/latest/docs/tutorials/5a_train_preference_comparisons_with_cnn.ipynb: WARNING: Executing notebook failed: CellExecutionError [mystnb.exec]
/home/docs/checkouts/readthedocs.org/user_builds/imitation/checkouts/latest/docs/tutorials/5a_train_preference_comparisons_with_cnn.ipynb: WARNING: Notebook exception traceback saved in: /home/docs/checkouts/readthedocs.org/user_builds/imitation/checkouts/latest/_readthedocs/html/reports/tutorials/5a_train_preference_comparisons_with_cnn.err.log [mystnb.exec]
/home/docs/checkouts/readthedocs.org/user_builds/imitation/checkouts/latest/docs/tutorials/8a_train_sqil_sac.ipynb: WARNING: Executing notebook failed: CellExecutionError [mystnb.exec]

Suggest we either make these warnings fatal, or unskip testing the notebooks in tests/test_examples.py so we test them on Linux again.

@AdamGleave AdamGleave added the bug Something isn't working label Oct 4, 2023
@AdamGleave
Copy link
Member Author

@ernestum can you investigate this -- I think you have the most context on this as you made the original change to run notebooks as part of docs build in #565

This is a priority to fix as it is making the CI unreliable.

@ernestum
Copy link
Collaborator

ernestum commented Oct 4, 2023

I'm on it (#803). I hope a simple config change will do the trick.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants