Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tutorials infrastructure: switch to text-based notebook workflow #59

Closed
wants to merge 38 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
38 commits
Select commit Hold shift + click to select a range
7ee54d5
Adding jupytext options to Makefile
Jun 23, 2021
0803983
Migrating from the "old" format: toc is not a mapping
Jun 23, 2021
55ff765
Moving builds to their folder
Jun 23, 2021
f3bf746
Building site instead of book
Jun 23, 2021
de2d146
Adding Makefile on website
Jun 23, 2021
e345a61
Removing website Makefile refs
Jun 23, 2021
1d4e64c
Adding website/conf.py
Jun 23, 2021
37c53d5
Starting index.md
Jun 23, 2021
3086296
Adding Indices and tables to index.md
Jun 23, 2021
0c22b39
Ignoring website local builds
Jun 23, 2021
568fe59
An example — testing jupytext conversion
Jun 23, 2021
2437a6a
Adding example to toctree on index.md
Jun 23, 2021
831741f
Creating a link from website/images to ../images
Jun 23, 2021
e818bc4
Testing toc structure
Jun 23, 2021
747d842
Organizing folders
Jun 23, 2021
f217e16
Changing intro.md
Jun 24, 2021
9244886
Testing python instead of ipython3 on {code-cell}s
Sep 13, 2021
872d226
jb-article -> jb-book
Sep 14, 2021
9803c81
Adding more converted notebooks
Sep 14, 2021
7af90a1
Changing - to _ on adv3 filename
Sep 15, 2021
fa7fed3
Adding some of the newly converted notebooks to the toc
Sep 15, 2021
71f8a0b
Fix tabs in Makefile
stefanv Dec 2, 2021
ae4c3fe
Require book directory for building html
stefanv Dec 2, 2021
caea7c1
Fix incorrect build path
stefanv Dec 2, 2021
f18a5cd
website -> site
Dec 2, 2021
78a689b
website -> site
Dec 2, 2021
83728db
website -> site
Dec 2, 2021
d4325e7
Refactor the build system
stefanv Dec 3, 2021
9edcefd
Fix skdemo
stefanv Dec 3, 2021
ef683cc
Fix up color_and_exposure module
stefanv Dec 3, 2021
f95be21
Fix image links
stefanv Dec 3, 2021
28d0e4c
Fix notebooks; fix skdemo; fix notebook build path
stefanv Dec 3, 2021
4103ab0
Require imagecodecs for reading tiffs
stefanv Dec 3, 2021
db180a5
Fix filters imports
stefanv Dec 3, 2021
64cea04
threshold_adaptive is not threshold_local
stefanv Dec 3, 2021
bfcc8c5
Final fixes to adv0_chromosomes
stefanv Dec 3, 2021
ad248c4
Fix adv1_lesion_quantification
stefanv Dec 3, 2021
6edeb56
Fix adv2_microarray
stefanv Dec 10, 2021
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 7 additions & 6 deletions .github/workflows/deploy.yml
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@ jobs:
python-version: [3.8]
# TODO Uncomment when skimage theme is ready
# node-version: [15]

steps:
- uses: actions/[email protected]

Expand Down Expand Up @@ -60,17 +61,17 @@ jobs:
# - name: Build the theme
# run: yarn build:theme-prod

# Build the book
- name: Build the book
# Build site
- name: Build site
env:
DISPLAY: ":99.0"
run: jupyter-book build .
run: make -C site/ html

# Deploy the book's HTML to github pages
# Deploy site to github pages
- name: GitHub Pages action
if: github.repository == 'scikit-image/skimage-tutorials' && github.ref == 'refs/heads/main' && github.event_name == 'push'
uses: peaceiris/[email protected]
with:
github_token: ${{ secrets.GITHUB_TOKEN }}
publish_dir: ./_build/html
cname: scikit-image.org
publish_dir: ./site/_build/html
cname: scikit-image.org
9 changes: 6 additions & 3 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,11 +1,14 @@
\#*
.#*
_requirements.installed
build
*.DS_Store
*.html
*.ipynb_checkpoints
*.pyc
*~
*pano-advanced-output.png
book/lessons

# Ignore built website
book/_build

# Ignore auto-generated content
book/_modules
45 changes: 17 additions & 28 deletions Makefile
Original file line number Diff line number Diff line change
@@ -1,41 +1,30 @@
OBSHELL=/bin/bash
.PHONY: clean book

.DEFAULT_GOAL = html
.DEFAULT_GOAL = book

LESSONS_DIR = lessons
GENERATED_LESSONS_DIR = book/lessons
NB_DIR = modules
EXEC_NB_DIR = book/_modules

MARKDOWNS = $(wildcard $(NB_DIR)/*.md)
NOTEBOOKS = $(patsubst $(NB_DIR)/%.md, $(EXEC_NB_DIR)/%.ipynb, $(MARKDOWNS))

_requirements.installed:
pip install -q -r requirements.txt
touch _requirements.installed

MARKDOWNS = $(wildcard $(LESSONS_DIR)/*.md)
MD_OUTPUTS = $(patsubst $(LESSONS_DIR)/%.md, $(GENERATED_LESSONS_DIR)/%.md, $(MARKDOWNS))
NOTEBOOKS = $(patsubst %.md, %.ipynb, $(MD_OUTPUTS))

.SECONDARY: $(MD_OUTPUTS) $(NOTEBOOKS)

$(GENERATED_LESSONS_DIR)/%.ipynb:$(LESSONS_DIR)/%.md book/lessons book/lessons/images
# This does not work, due to bug in notedown; see https://github.com/aaren/notedown/issues/53
#notedown --match=python --precode='%matplotlib inline' $< > $@
notedown --match=python $< > $@
jupyter nbconvert --execute --inplace $@ --ExecutePreprocessor.timeout=-1

%.md:%.ipynb
jupyter nbconvert --to=mdoutput --output="$(notdir $@)" --output-dir=$(GENERATED_LESSONS_DIR) $<
# $(eval NBSTRING := [📂 Download lesson notebook](.\/$(basename $(notdir $@)).ipynb)\n\n)
# sed -i'.bak' '1s/^/$(NBSTRING)/' $@

$(EXEC_NB_DIR):
mkdir book/_modules

book/lessons:
mkdir -p book/lessons
$(EXEC_NB_DIR)/skdemo:
ln -s $(PWD)/lectures/skdemo $(EXEC_NB_DIR)/skdemo

book/lessons/images:
ln -s ${PWD}/lessons/images ${PWD}/book/lessons/images
$(EXEC_NB_DIR)/%.ipynb:$(NB_DIR)/%.md $(EXEC_NB_DIR) $(EXEC_NB_DIR)/skdemo
@# Jupytext will also catch and print execution errors
@# unless a cell is marked with the `raises-exception` tag
jupytext --execute --to ipynb --run-path $(NB_DIR) --output $@ $<

html: | _requirements.installed $(NOTEBOOKS) $(MD_OUTPUTS)
book: _requirements.installed $(NOTEBOOKS)
@export SPHINXOPTS=-W; make -C book html
cp $(GENERATED_LESSONS_DIR)/*.ipynb book/build/html/lessons/

clean:
rm -rf $(GENERATED_LESSONS_DIR)/*
make -C book clean
29 changes: 23 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,4 @@
scikit-image tutorials
======================
# scikit-image tutorials

A collection of tutorials for the [scikit-image](http://skimage.org) package.

Expand All @@ -16,8 +15,7 @@ Refer to [the gallery](http://scikit-image.org/docs/dev/auto_examples/) as
well as [scikit-image demos](https://github.com/scikit-image/skimage-demos)
for more examples.

Usage
-----
## Usage

These usage guidelines are based on goodwill. They are not a legal contract.

Expand All @@ -42,9 +40,28 @@ For more information on these guidelines, which are sometimes known as
CC0 (+BY), see [this blog post](http://www.dancohen.org/2013/11/26/cc0-by/) by
Dan Cohen.

Contributing
------------
## Contributing

If you make any modifications to these tutorials that you think would benefit
the community at large, please
[create a pull request](http://scikit-image.org/docs/dev/contribute.html)!

The tutorials live at
https://github.com/scikit-image/skimage-tutorials


## Contributor notes

- Notebooks are stored in `modules`; see [modules/00_images_are_arrays.md](modules/00_images_are_arrays.md)
for an example.
- They use the [myst](https://myst-nb.readthedocs.io/en/latest/)
notebook format
- Cells can be tagged with:
`remove-input` : Get rid of the input but display the output
`remove-output` : Show the input but not the output
`raises-exception` : This cell is expected to fail execution, so
don't halt the book build because of it.

To build the book, run `make`. Results appear in `book/_build`.
Notebooks can be edited in your favorite text editor or in Jupyter (as
long as Jupytext is installed).
10 changes: 0 additions & 10 deletions _config.yml

This file was deleted.

5 changes: 3 additions & 2 deletions _toc.yml
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
- file: lectures/00_images_are_arrays
format: jb-book
root: lectures/00_images_are_arrays
sections:
- file: lectures/1_image_filters
- file: lectures/3_morphological_operations
- file: lectures/4_segmentation
- file: lectures/three_dimensional_image_processing

20 changes: 20 additions & 0 deletions book/Makefile
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
.PHONY: help Makefile clean

# You can set these variables from the command line, and also
# from the environment for the first two.
SPHINXOPTS ?=
SPHINXBUILD ?= sphinx-build
SOURCEDIR = .
BUILDDIR = _build

# Put it first so that "make" without argument is like "make help".
help:
@$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)

clean:
rm -rf _build

# Catch-all target: route all unknown targets to Sphinx using the new
# "make mode" option. $(O) is meant as a shortcut for $(SPHINXOPTS).
%: Makefile
@$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
Binary file added book/_static/favicon.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added book/_static/skimage-logo.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
15 changes: 15 additions & 0 deletions book/applications.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
# scikit-image applications

A collection of tutorials highlighting the use of scikit-image for applications in science.

```{toctree}
---
maxdepth: 1
---
_modules/adv0_chromosomes.md
_modules/adv1_lesion-quantification.md
_modules/adv2_microarray.md
_modules/adv3_panorama_stitching.md
_modules/adv4_warping.md
_modules/adv5_pores.md
```
71 changes: 71 additions & 0 deletions book/conf.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,71 @@
# Configuration file for the Sphinx documentation builder.
#
# This file only contains a selection of the most common options. For a full
# list see the documentation:
# https://www.sphinx-doc.org/en/master/usage/configuration.html

# -- Path setup --------------------------------------------------------------

# If extensions (or modules to document with autodoc) are in another directory,
# add these directories to sys.path here. If the directory is relative to the
# documentation root, use os.path.abspath to make it absolute, like shown here.
#
# import os
# import sys
# sys.path.insert(0, os.path.abspath('.'))


# -- Project information -----------------------------------------------------

project = 'scikit-image tutorials'
copyright = '2021, the scikit-image community'
author = 'the scikit-image community'


# -- General configuration ---------------------------------------------------

# Add any Sphinx extension module names here, as strings. They can be
# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
# ones.
extensions = [
'myst_nb',
'sphinx_copybutton',
]
jupyter_execute_notebooks = "off"

# Add any paths that contain templates here, relative to this directory.
templates_path = ['_templates']

# List of patterns, relative to source directory, that match files and
# directories to ignore when looking for source files.
# This pattern also affects html_static_path and html_extra_path.
exclude_patterns = ['_build', 'Thumbs.db', '.DS_Store', 'notebooks']


# -- Options for HTML output -------------------------------------------------

# The theme to use for HTML and HTML Help pages. See the documentation for
# a list of builtin themes.
#
html_theme = 'sphinx_book_theme'
html_title = 'scikit-image Tutorials'
html_logo = '_static/skimage-logo.png'
html_favicon = '_static/favicon.png'
html_theme_options = {
"github_url": "https://github.com/scikit-image/skimage-tutorials/",
"repository_url": "https://github.com/scikit-image/skimage-tutorials/",
"repository_branch": "main",
"use_repository_button": True,
"use_issues_button": True,
"use_edit_page_button": True,
"path_to_docs": "site/",
"launch_buttons": {
"binderhub_url": "https://mybinder.org",
},
}


# Add any paths that contain custom static files (such as style sheets) here,
# relative to this directory. They are copied after the builtin static files,
# so a file named "default.css" will overwrite the builtin "default.css".
html_static_path = ['_static']
1 change: 1 addition & 0 deletions book/images
21 changes: 21 additions & 0 deletions book/index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
# scikit-image tutorials

[![Binder](http://mybinder.org/badge_logo.svg)][launch_binder]

[launch_binder]: http://mybinder.org/v2/gh/scikit-image/skimage-tutorials/main?urlpath=lab/tree/content

## Content

```{toctree}
---
maxdepth: 2
---
introduction
applications
```

## Indices and tables

* {ref}`genindex`
* {ref}`modindex`
* {ref}`search`
11 changes: 11 additions & 0 deletions book/introduction.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
# An introduction to scikit-image

Basic tutorials to prepare you for your journey on scikit-image.

```{toctree}
---
maxdepth: 1
---
_modules/tour_of_skimage
_modules/00_images_are_arrays
```
Binary file added images/censure_example.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added images/circle_detection.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added images/fog_tunnel.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added images/particle_detection.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added images/power_law_growth_regimes.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
2 changes: 1 addition & 1 deletion lectures/skdemo/_skdemo.py
Original file line number Diff line number Diff line change
Expand Up @@ -74,7 +74,7 @@ def imshow_all(*images, **kwargs):
kwargs.setdefault('vmin', vmin)
kwargs.setdefault('vmax', vmax)

nrows, ncols = kwargs.get('shape', (1, len(images)))
nrows, ncols = kwargs.pop('shape', (1, len(images)))

size = nrows * kwargs.pop('size', 5)
width = size * len(images)
Expand Down
Loading