Migrated user guide notebooks to MyST-NB and added sphinx extension by mmccarty · Pull Request #10685 · NVIDIA/cudf

mmccarty · 2022-04-19T19:31:18Z

This PR adds MyST-NB to the sphinx docs build. This will allow the notebooks to be executed when the docs are built to ensure they stay up-to-date with the latest version of the software.

Conda env yml updates
Sphinx conf.py updates to enable and configure myst-nb
Minor fixes to the 10min.ipynb notebook to find test data file path.

review-notebook-app · 2022-04-19T19:31:21Z

Check out this pull request on

See visual diffs & provide feedback on Jupyter Notebooks.

Powered by ReviewNB

codecov · 2022-04-19T20:40:05Z

Codecov Report

Merging #10685 (e1b2d13) into branch-22.06 (01d08af) will increase coverage by 0.04%.
The diff coverage is 98.21%.

❗ Current head e1b2d13 differs from pull request most recent head 14e7574. Consider uploading reports for the commit 14e7574 to get more accurate results

@@               Coverage Diff                @@
##           branch-22.06   #10685      +/-   ##
================================================
+ Coverage         86.35%   86.39%   +0.04%     
================================================
  Files               142      142              
  Lines             22335    22302      -33     
================================================
- Hits              19287    19268      -19     
+ Misses             3048     3034      -14

Impacted Files	Coverage Δ
python/cudf/cudf/core/frame.py	`93.41% <ø> (ø)`
python/cudf/cudf/utils/utils.py	`90.35% <ø> (+0.06%)`	⬆️
python/cudf/cudf/core/column/numerical.py	`96.17% <50.00%> (+0.29%)`	⬆️
python/cudf/cudf/core/column/categorical.py	`89.97% <100.00%> (+0.20%)`	⬆️
python/cudf/cudf/core/column/column.py	`89.43% <100.00%> (-0.02%)`	⬇️
python/cudf/cudf/core/column/lists.py	`92.91% <100.00%> (+1.39%)`	⬆️
python/cudf/cudf/core/column/string.py	`89.21% <100.00%> (+0.10%)`	⬆️
python/cudf/cudf/core/dataframe.py	`93.75% <100.00%> (+<0.01%)`	⬆️
python/cudf/cudf/core/groupby/groupby.py	`91.68% <100.00%> (+0.27%)`	⬆️
python/cudf/cudf/core/multiindex.py	`92.28% <100.00%> (+0.13%)`	⬆️
... and 9 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d5e6941...14e7574. Read the comment docs.

shwina · 2022-04-20T17:30:52Z

@mmccarty there's a conflict here due to #10690 modifying guide-to-udfs.ipynb. However this PR deleted that file and replaced it with guide-to-udfs.md.

I incorporated those changes into guide-to-udfs.md as part of my downstream PR #10691 -- you can just steal it from here.

shwina · 2022-04-20T20:03:04Z

Checked this branch out locally and it looks great! The rendered "notebooks" are also neater than the ones rendered by nbsphinx. I'm +1 to this change.

That being said, there's a downside to moving to Markdown notebooks: the outputs from cell execution aren't included in the source. This means that reviewing significant changes to markdown notebooks is pretty much impossible without checking out the branch locally and running the notebooks manually.

Longer-term, I can imagine CI building docs on every doc-touching PR and providing a way to preview the rendered docs to make review easier.

That would still not be quite as nice as ReviewNB :(

mmccarty · 2022-04-21T14:20:57Z

@mmccarty there's a conflict here due to #10690 modifying guide-to-udfs.ipynb. However this PR deleted that file and replaced it with guide-to-udfs.md.

Reverted .ipynb to .md conversion and rebased with branch-22.04 to resolve this conflict.

shwina

This LGTM

ajschmidt8

@mmccarty, can you add sphinx-autobuild to https://github.com/rapidsai/integration/blob/branch-22.06/conda/recipes/rapids-doc-env/meta.yaml and myst-nb to https://github.com/rapidsai/integration/blob/branch-22.06/conda/recipes/rapids-notebook-env/meta.yaml?

That will need to be done before ops can approve this PR.

mmccarty · 2022-04-21T18:22:04Z

Will do! Thanks @ajschmidt8

mmccarty · 2022-04-21T18:31:50Z

@ajschmidt8 I'm pretty sure those dependencies are only needed in the docs environment. Where we ended up should not impact notebook execution in Jupyter.

galipremsagar · 2022-04-22T15:44:45Z

It looks like ORC section in 10 minutes to CUDF and Dask-CUDF notebook when rendered doesn't have a horizontal scroll and thus overlaps onto the sidebar text like:

Here is a comparison of nbsphinx:
https://docs.rapids.ai/api/cudf/nightly/user_guide/10min.html#ORC

Same in Converting a CuPy Array to a cuDF DataFrame section and sections below it:

nbsphinx: https://docs.rapids.ai/api/cudf/nightly/user_guide/10min-cudf-cupy.html#Converting-a-CuPy-Array-to-a-cuDF-DataFrame

Can we get the horizontal scroll with myst-nb too?

shwina · 2022-04-22T16:06:52Z

@galipremsagar looks related to jupyter-book/jupyter-book#1611 perhaps?

galipremsagar · 2022-04-22T16:26:16Z

@galipremsagar looks related to executablebooks/jupyter-book#1611 perhaps?

Yes, to fix it we will have to add the following rule in https://github.com/rapidsai/cudf/blob/branch-22.06/docs/cudf/source/_static/params.css

.output.text_html {
    overflow: auto;
}

mmccarty · 2022-04-22T19:52:48Z

rerun tests

shwina · 2022-04-22T21:13:30Z

Looking into the CI failures here.

shwina · 2022-04-25T20:15:27Z

rerun tests

shwina · 2022-04-25T21:41:48Z

rerun tests

vyasr

One minor request to add a comment, otherwise LGTM.

vyasr · 2022-04-25T22:14:13Z

I committed my proposed change since it looks like this PR is otherwise ready to merge pending tests passing.

shwina · 2022-04-26T00:45:50Z

@gpucibot merge

Follow-up from #10685 to fix deprecation warnings in the 10 minute notebook. Fixes: #10613 Changes: - Fixed deprecation warning for `Series.applymap` ➡️ `Series.apply` - Removed two cells demonstrating `Series.append`. This has also been removed from the Pandas 10 minute notebook because the feature is deprecated. - Refactored ORC file path logic to be a bit simpler Authors: - Bradley Dice (https://github.com/bdice) Approvers: - Mike McCarty (https://github.com/mmccarty) - GALI PREM SAGAR (https://github.com/galipremsagar) URL: #10738

github-actions Bot added the conda label Apr 19, 2022

This was referenced Apr 20, 2022

Fix sphinx/jupyter heading issue in UDF notebook #10690

Merged

Reorganize cuDF Python docs #10691

Merged

shwina added non-breaking Non-breaking change doc Documentation labels Apr 20, 2022

mmccarty marked this pull request as ready for review April 20, 2022 18:20

mmccarty requested review from a team as code owners April 20, 2022 18:20

mmccarty requested review from PointKernel, galipremsagar, isVoid and jrhemstad April 20, 2022 18:30

Using MyST-NB sphinx extension for Notebook execution and rendering

43ddc2d

shwina approved these changes Apr 21, 2022

View reviewed changes

ajschmidt8 reviewed Apr 21, 2022

View reviewed changes

mmccarty mentioned this pull request Apr 21, 2022

Added myst-nb and sphinx-autobuild to rapids-docs-env rapidsai/integration#468

Merged

ajschmidt8 approved these changes Apr 22, 2022

View reviewed changes

shwina reviewed Apr 22, 2022

View reviewed changes

Comment thread docs/cudf/source/conf.py

set overflow for notebook output

21bdce9

galipremsagar reviewed Apr 22, 2022

View reviewed changes

Comment thread docs/cudf/source/_static/params.css Outdated

Update docs/cudf/source/_static/params.css

9e22eda

galipremsagar approved these changes Apr 22, 2022

View reviewed changes

bdice reviewed Apr 22, 2022

View reviewed changes

Comment thread conda/environments/cudf_dev_cuda11.5.yml

Comment thread docs/cudf/source/user_guide/10min.ipynb

vyasr approved these changes Apr 25, 2022

View reviewed changes

Comment thread docs/cudf/source/_static/params.css

Update docs/cudf/source/_static/params.css

14e7574

bdice mentioned this pull request Apr 25, 2022

[BUG] Series.apply does not retain the Series name #10715

Closed

rapids-bot Bot merged commit 93070fa into NVIDIA:branch-22.06 Apr 26, 2022

bdice mentioned this pull request Apr 26, 2022

Revise 10 minutes notebook. #10738

Merged

Uh oh!

Conversation

mmccarty commented Apr 19, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

review-notebook-app Bot commented Apr 19, 2022

Uh oh!

codecov Bot commented Apr 19, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

shwina commented Apr 20, 2022

Uh oh!

shwina commented Apr 20, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mmccarty commented Apr 21, 2022

Uh oh!

shwina left a comment

Choose a reason for hiding this comment

Uh oh!

ajschmidt8 left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mmccarty commented Apr 21, 2022

Uh oh!

mmccarty commented Apr 21, 2022

Uh oh!

galipremsagar commented Apr 22, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

shwina commented Apr 22, 2022

Uh oh!

galipremsagar commented Apr 22, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

mmccarty commented Apr 22, 2022

Uh oh!

shwina commented Apr 22, 2022

Uh oh!

shwina commented Apr 25, 2022

Uh oh!

shwina commented Apr 25, 2022

Uh oh!

vyasr left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

vyasr commented Apr 25, 2022

Uh oh!

shwina commented Apr 26, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

mmccarty commented Apr 19, 2022 •

edited

Loading

codecov Bot commented Apr 19, 2022 •

edited

Loading

shwina commented Apr 20, 2022 •

edited

Loading

ajschmidt8 left a comment •

edited

Loading

galipremsagar commented Apr 22, 2022 •

edited

Loading

galipremsagar commented Apr 22, 2022 •

edited

Loading