diff --git a/.readthedocs.yaml b/.readthedocs.yaml
index dde32e62f..7bb04d170 100644
--- a/.readthedocs.yaml
+++ b/.readthedocs.yaml
@@ -6,6 +6,7 @@ version: 2
sphinx:
configuration: docs/source/conf.py
+ builder: dirhtml
build:
os: ubuntu-22.04
diff --git a/docs/requirements.txt b/docs/requirements.txt
index 5c38c561a..2e0c7627c 100644
--- a/docs/requirements.txt
+++ b/docs/requirements.txt
@@ -1,7 +1,8 @@
-myst-parser>=0.18
-pydata-sphinx-theme>=0.11
+myst-parser<4
+pydata-sphinx-theme
sphinx-autobuild
sphinx-copybutton
+sphinx-design
sphinxcontrib-autoprogram>=0.1.7
sphinxext-opengraph
sphinxext-rediraffe
diff --git a/docs/source/_static/images/logo-dark.png b/docs/source/_static/images/logo-dark.png
new file mode 100644
index 000000000..36c986804
Binary files /dev/null and b/docs/source/_static/images/logo-dark.png differ
diff --git a/docs/source/_static/images/logo.png b/docs/source/_static/images/logo.png
index 9e03007d0..c3a497984 100644
Binary files a/docs/source/_static/images/logo.png and b/docs/source/_static/images/logo.png differ
diff --git a/docs/source/_static/images/whentouse.svg b/docs/source/_static/images/whentouse.svg
new file mode 100644
index 000000000..089e25011
--- /dev/null
+++ b/docs/source/_static/images/whentouse.svg
@@ -0,0 +1,4 @@
+
+
+
\ No newline at end of file
diff --git a/docs/source/architecture.md b/docs/source/architecture.md
index 2d14daf36..40e91a4c3 100644
--- a/docs/source/architecture.md
+++ b/docs/source/architecture.md
@@ -20,11 +20,11 @@ with explicit **versions** is all that is needed.
In repo2docker, a Buildpack does the following things:
1. **Detect** if it can handle a given repository
-2. **Build** a base language environment in the docker image
-3. **Copy** the contents of the repository into the docker image
-4. **Assemble** a specific environment in the docker image based on repository contents
-5. **Push** the built docker image to a specific docker registry (optional)
-6. **Run** the build docker image as a docker container (optional)
+2. **Build** a base language environment in the Docker image
+3. **Copy** the contents of the repository into the Docker image
+4. **Assemble** a specific environment in the Docker image based on repository contents
+5. **Push** the built Docker image to a specific Docker registry (optional)
+6. **Run** the build Docker image as a Docker container (optional)
### Detect
@@ -56,15 +56,15 @@ repositories built with the same buildpack.
For example, in `CondaBuildPack`, the base environment consists of installing [miniconda](https://conda.io/miniconda.html)
and basic notebook packages (from `repo2docker/buildpacks/conda/environment.yml`). This is going
to be the same for most repositories built with `CondaBuildPack`, so we want to use
-[docker layer caching](https://thenewstack.io/understanding-the-docker-cache-for-faster-builds/) as
+[Docker layer caching](https://thenewstack.io/understanding-the-docker-cache-for-faster-builds/) as
much as possible for performance reasons. Next time a repository is built with `CondaBuildPack`,
-we can skip straight to the **copy** step (since the base environment docker image _layers_ have
+we can skip straight to the **copy** step (since the base environment Docker image _layers_ have
already been built and cached).
The `get_build_scripts` and `get_build_script_files` methods are primarily used for this.
`get_build_scripts` can return arbitrary bash script lines that can be run as different users,
and `get_build_script_files` is used to copy specific scripts (such as a conda installer) into
-the image to be run as pat of `get_build_scripts`. Code in either has following constraints:
+the image to be run as part of `get_build_scripts`. Code in either has following constraints:
1. You can _not_ use the contents of repository in them, since this happens before the repository
is copied into the image. For example, `pip install -r requirements.txt` will not work,
@@ -91,22 +91,20 @@ This usually means installing required libraries specified in a format native to
Most of this work is done in `get_assemble_scripts` method. It can return arbitrary bash script
lines that can be run as different users, and has access to the repository contents (unlike
-`get_build_scripts`). The docker image layers produced by this usually can not be cached,
+`get_build_scripts`). The Docker image layers produced by this usually can not be cached,
so less restrictions apply to this than to `get_build_scripts`.
-At the end of the assemble step, the docker image is ready to be used in various ways!
+At the end of the assemble step, the Docker image is ready to be used in various ways!
### Push
-Optionally, repo2docker can **push** a built image to a [docker registry](https://docs.docker.com/registry/),
+Optionally, repo2docker can **push** a built image to a [Docker registry](https://docs.docker.com/registry/),
if you specify the `--push` flag.
### Run
-Optionally, repo2docker can **run** the built image and allow the user to access the Jupyter Notebook
-running inside by default. This is also done as a convenience only (since you can do the same with `docker run`
-after using repo2docker only to build), and implemented in `Repo2Docker.run`. It is activated by default
-unless the `--no-run` commandline flag is passed.
+Optionally, repo2docker can **run** the built image and allow the user to access the Jupyter Notebook running inside by default.
+This is also done as a convenience only (since you can do the same with `docker run` after using repo2docker only to build), and implemented in `Repo2Docker.run`. It is activated by default unless the `--no-run` commandline flag is passed.
## ContentProviders
diff --git a/docs/source/changelog.md b/docs/source/changelog.md
index 58cef1cff..dc415acab 100644
--- a/docs/source/changelog.md
+++ b/docs/source/changelog.md
@@ -134,7 +134,7 @@
- pre-commit update to fix ci [#1238](https://github.com/jupyterhub/repo2docker/pull/1238) ([@minrk](https://github.com/minrk))
- Upgrade to mamba 1.1 and enable rich SAT error messages [#1232](https://github.com/jupyterhub/repo2docker/pull/1232) ([@SylvainCorlay](https://github.com/SylvainCorlay))
- Upgrade to mamba 1.0 [#1213](https://github.com/jupyterhub/repo2docker/pull/1213) ([@SylvainCorlay](https://github.com/SylvainCorlay))
-- docker image: update alpine to 3.16 [#1212](https://github.com/jupyterhub/repo2docker/pull/1212) ([@consideRatio](https://github.com/consideRatio))
+- Docker image: update alpine to 3.16 [#1212](https://github.com/jupyterhub/repo2docker/pull/1212) ([@consideRatio](https://github.com/consideRatio))
- reconcile CLI/config priority [#1211](https://github.com/jupyterhub/repo2docker/pull/1211) ([@minrk](https://github.com/minrk))
- pipfile: pass --clear flag, and do it separetely to not be ignored [#1208](https://github.com/jupyterhub/repo2docker/pull/1208) ([@consideRatio](https://github.com/consideRatio))
- run submodule test over https [#1204](https://github.com/jupyterhub/repo2docker/pull/1204) ([@minrk](https://github.com/minrk))
@@ -299,12 +299,12 @@ The repo2docker container image has moved to [quay.io/jupyterhub/repo2docker](ht
- Workaround docker-py dependency's failure to import six {pr}`1066:` by {user}`consideratio`
- fix: add chardet, a not explicitly declared dependency {pr}`1064` by {user}`johnhoman`
-- Add build-base to build stage of docker image {pr}`1051` by {user}`yuvipanda`
+- Add build-base to build stage of Docker image {pr}`1051` by {user}`yuvipanda`
- Fix regression in hydroshare introduced after moving to requests {pr}`1034` by {user}`MridulS`
### Other merged PRs
-- Update README quay.io URL, Add docker latest tag {pr}`1075` by {user}`manics`
+- Update README quay.io URL, add Docker latest tag {pr}`1075` by {user}`manics`
- GitHub workflow build and push to Docker hub {pr}`1071` by {user}`manics`
- Rename master branch to main {pr}`1068` by {user}`manics`
- Remove Pipfile & Pipfile.lock {pr}`1054` by {user}`yuvipanda`
@@ -343,7 +343,7 @@ The repo2docker container image has moved to [quay.io/jupyterhub/repo2docker](ht
### Other merged PRs
- Cleanup install_requires including duplicates {pr}`1020` by {user}`manics`
-- bump docker action version {pr}`1018` by {user}`minrk`
+- bump Docker action version {pr}`1018` by {user}`minrk`
- bump python in circleci test {pr}`1013` by {user}`minrk`
- Investigating the missing logs {pr}`1008` by {user}`betatim`
- Experiment with different install mechanism to get code coverage stats again {pr}`982` by {user}`betatim`
@@ -376,7 +376,7 @@ The repo2docker container image has moved to [quay.io/jupyterhub/repo2docker](ht
- chmod start script from repo2docker-entrypoint {pr}`886` by {user}`danlester`
- pypi jupyter-offlinenotebook==0.1.0 {pr}`880` by {user}`manics`
- Add support for Julia 1.4.1 {pr}`878` by {user}`davidanthoff`
-- Change --env option to work like docker's {pr}`874` by {user}`hwine`
+- Change --env option to work like Docker's {pr}`874` by {user}`hwine`
- Add support for Julia 1.4.0 {pr}`870` by {user}`davidanthoff`
- Update server proxy and rsession proxy {pr}`869` by {user}`betatim`
- Use miniforge instead of miniconda to get conda {pr}`859` by {user}`yuvipanda`
@@ -549,7 +549,7 @@ Release date: 2019-02-21
### New features
-- Add additional metadata to docker images about how they were built {pr}`500` by
+- Add additional metadata to Docker images about how they were built {pr}`500` by
{user}`jrbourbeau`.
- Allow users to install global NPM packages: {pr}`573` by {user}`GladysNalvarte`.
- Add documentation on switching the user interface presented by a
@@ -579,7 +579,7 @@ Release date: 2019-02-21
- Fix quoting issue in `GIT_CREDENTIAL_ENV` environment variable by
{user}`minrk` in {pr}`572`.
- Change to using the first 8 characters of each Git commit, not the last 8,
- to tag each built docker image of repo2docker itself. {user}`minrk` in {pr}`562`.
+ to tag each built Docker image of repo2docker itself. {user}`minrk` in {pr}`562`.
- Allow users to select the Julia when using a `requirements.txt` by
{user}`yuvipanda` in {pr}`557`.
- Set `JULIA_DEPOT_PATH` to install packages outside the home directory by
diff --git a/docs/source/cli.md b/docs/source/cli.md
new file mode 100644
index 000000000..ac60bda8e
--- /dev/null
+++ b/docs/source/cli.md
@@ -0,0 +1,87 @@
+# Command-line usage and API
+
+`repo2docker` is called with this command:
+
+```
+jupyter-repo2docker
+```
+
+where `` is a repository in one of [the supported repository providers](#repository-providers).
+
+For example, the following command will build an image of Peter Norvig's
+[Pytudes] repository:
+
+```
+jupyter-repo2docker https://github.com/norvig/pytudes
+```
+
+Building the image may take a few minutes.
+
+[Pytudes] uses a [`requirements.txt` file](https://github.com/norvig/pytudes/blob/HEAD/requirements.txt) to specify its Python environment. Because of this, `repo2docker` will use `pip` to install dependencies listed in this `requirements.txt` file, and these will be present in the generated Docker image. To learn more about configuration files in `repo2docker` visit [](#config-files).
+
+When the image is built, a message will be output to your terminal:
+
+```
+Copy/paste this URL into your browser when you connect for the first time,
+to login with a token:
+ http://0.0.0.0:36511/?token=f94f8fabb92e22f5bfab116c382b4707fc2cade56ad1ace0
+```
+
+Pasting the URL into your browser will open Jupyter Notebook with the
+dependencies and contents of the source repository in the built image.
+
+## Debug repo2docker with `--debug` and `--no-build`
+
+To debug the container image being built, pass the `--debug` parameter:
+
+> ```bash
+> jupyter-repo2docker --debug https://github.com/norvig/pytudes
+> ```
+
+This will print the generated `Dockerfile`, build it, and run it.
+
+To see the generated `Dockerfile` without actually building it, pass `--no-build` to the commandline.
+This `Dockerfile` output is for **debugging purposes** of `repo2docker` only - it can not be used by Docker directly.
+
+> ```bash
+> jupyter-repo2docker --no-build --debug https://github.com/norvig/pytudes
+> ```
+
+## Build from a branch, commit or tag
+
+To build a particular branch and commit, use the argument `--ref` and
+specify the `branch-name` or `commit-hash`. For example:
+
+```
+jupyter-repo2docker --ref 9ced85dd9a84859d0767369e58f33912a214a3cf https://github.com/norvig/pytudes
+```
+
+:::{tip}
+For reproducible builds, we recommend specifying a commit-hash to
+deterministically build a fixed version of a repository. Not specifying a
+commit-hash will result in the latest commit of the repository being built.
+:::
+
+## Set environment variables during builds
+
+When running repo2docker locally you can use the `-e` or `--env` command-line
+flag for each variable that you want to define.
+
+For example:
+
+```bash
+jupyter-repo2docker -e VAR1=val1 -e VAR2=val2 ...
+```
+
+You can also configure environment variables for all users of a repository using the
+[](#config-start) configuration file.
+
+(command-line-api)=
+
+## Command-line API
+
+```{autoprogram} repo2docker.__main__:argparser
+:prog: jupyter-repo2docker
+```
+
+[pytudes]: https://github.com/norvig/pytudes
diff --git a/docs/source/conf.py b/docs/source/conf.py
index 2dcb08615..590400c75 100644
--- a/docs/source/conf.py
+++ b/docs/source/conf.py
@@ -22,6 +22,7 @@
"sphinxcontrib.autoprogram",
"sphinxext.opengraph",
"sphinxext.rediraffe",
+ "sphinx_design",
]
root_doc = "index"
source_suffix = [".md", ".rst"]
@@ -40,6 +41,7 @@
#
myst_enable_extensions = [
"colon_fence",
+ "substitution",
]
@@ -58,8 +60,11 @@
from repo2docker.buildpacks.conda import CondaBuildPack
-default_python = CondaBuildPack.major_pythons["3"]
-
+default_python = f"`Python {CondaBuildPack.major_pythons['3']}`"
+myst_substitutions = {
+ "default_python": default_python,
+ "default_python_version": default_python,
+}
rst_prolog = f"""
.. |default_python| replace:: **Python {default_python}**
.. |default_python_version| replace:: {default_python}
@@ -73,12 +78,20 @@
html_favicon = "_static/images/favicon.ico"
html_static_path = ["_static"]
html_css_files = ["custom.css"]
+html_sidebars = {
+ "changelog": [],
+ "start": [],
+}
# pydata_sphinx_theme reference: https://pydata-sphinx-theme.readthedocs.io/en/latest/
html_theme = "pydata_sphinx_theme"
html_theme_options = {
"use_edit_page_button": True,
"github_url": "https://github.com/jupyterhub/repo2docker",
+ "logo": {
+ "image_dark": "_static/images/logo-dark.png",
+ "image_light": "_static/images/logo.png",
+ },
}
html_context = {
"github_user": "jupyterhub",
diff --git a/docs/source/config_files.rst b/docs/source/config_files.rst
deleted file mode 100644
index 975f82c85..000000000
--- a/docs/source/config_files.rst
+++ /dev/null
@@ -1,245 +0,0 @@
-.. _config-files:
-
-===================
-Configuration Files
-===================
-
-``repo2docker`` looks for configuration files in the repository being built
-to determine how to build it. In general, ``repo2docker`` uses the same
-configuration files as other software installation tools,
-rather than creating new custom configuration files.
-
-A number of ``repo2docker`` configuration files can be combined to compose more
-complex setups.
-
-The `binder examples `_ organization on
-GitHub contains a list of sample repositories for common configurations
-that ``repo2docker`` can build with various configuration files such as
-Python and R installation in a repository.
-
-A list of supported configuration files (roughly in the order of build priority)
-can be found on this page (and to the right).
-
-.. _environment.yml:
-
-``environment.yml`` - Install a conda environment
-=================================================
-
-``environment.yml`` is the standard configuration file used by `conda `_
-that lets you install any kind of package,
-including Python, R, and C/C++ packages.
-``repo2docker`` does not use your ``environment.yml`` to create and activate a new conda environment.
-Rather, it updates a base conda environment `defined here `_ with the packages listed in your ``environment.yml``.
-This means that the environment will always have the same default name, not the name
-specified in your ``environment.yml``.
-
-.. note::
-
- You can install files from pip in your ``environment.yml`` as well.
- For example, see the `binder-examples environment.yml
- `_
- file.
-
-You can also specify which Python version to install in your built environment
-with ``environment.yml``. By default, ``repo2docker`` installs
-|default_python| with your ``environment.yml`` unless you include the version of
-Python in this file. ``conda`` Should support all versions of Python,
-though ``repo2docker`` support is best with Python 3.7-3.11.
-
-.. warning::
- If you include a Python version in a ``runtime.txt`` file in addition to your
- ``environment.yml``, your ``runtime.txt`` will be ignored.
-
-.. _Pipfile:
-
-``Pipfile`` and/or ``Pipfile.lock`` - Install a Python environment
-==================================================================
-
-`pipenv `_ allows you to manage a virtual
-environment Python dependencies. When using ``pipenv``, you end up with
-``Pipfile`` and ``Pipfile.lock`` files. The lock file contains explicit details
-about the packages that has been installed that met the criteria within the
-``Pipfile``.
-
-If both ``Pipfile`` and ``Pipfile.lock`` are found by repo2docker, the former
-will be ignored in favor of the lock file. Also note that these files
-distinguish packages and development packages and that repo2docker will install
-both kinds.
-
-.. _requirements.txt:
-
-``requirements.txt`` - Install a Python environment
-===================================================
-
-This specifies a list of Python packages that should be installed in your
-environment. Our
-`requirements.txt example `_
-on GitHub shows a typical requirements file.
-
-
-.. _setup.py:
-
-``setup.py`` - Install Python packages
-======================================
-
-To install your repository like a Python package, you may include a
-``setup.py`` file. repo2docker installs ``setup.py`` files by running
-``pip install -e .``.
-
-.. _Project.toml:
-
-``Project.toml`` - Install a Julia environment
-==============================================
-
-A ``Project.toml`` (or ``JuliaProject.toml``) file can specify both the
-version of Julia to be used and a list of Julia packages to be installed.
-If a ``Manifest.toml`` is present, it will determine the exact versions
-of the Julia packages that are installed.
-
-
-.. _REQUIRE:
-
-``REQUIRE`` - Install a Julia environment (legacy)
-==================================================
-
-``REQUIRE`` files no longer work, and are no longer supported.
-The recommended way of installing a Julia environment is to use a ``Project.toml`` file.
-
-
-.. _install.R:
-
-``install.R`` - Install an R/RStudio environment
-================================================
-
-This is used to install R libraries pinned to a specific snapshot on
-`Posit Package Manager `_.
-To set the date of the snapshot add a runtime.txt_.
-For an example ``install.R`` file, visit our `example install.R file `_.
-
-
-.. _apt.txt:
-
-``apt.txt`` - Install packages with apt-get
-===========================================
-
-A list of Debian packages that should be installed. The base image used is usually the latest released
-version of Ubuntu.
-
-We use ``apt.txt``, for example, to install LaTeX in our
-`example apt.txt for LaTeX `_.
-
-
-.. _DESCRIPTION:
-
-``DESCRIPTION`` - Install an R package
-======================================
-
-To install your repository like an R package, you may include a
-``DESCRIPTION`` file. repo2docker installs the package and dependencies
-from the ``DESCRIPTION`` by running ``devtools::install_local(getwd())``.
-
-You can also have have a ``runtime.txt`` file that is formatted as
-``r---
``, where YYYY-MM-DD is a snapshot of CRAN that will be used
-for your R installation. If ``runtime.txt`` isn't provided in this case, a
-recent date will be used.
-
-
-.. _postBuild:
-
-``postBuild`` - Run code after installing the environment
-=========================================================
-
-A script that can contain arbitrary commands to be run after the whole repository has been built. If you
-want this to be a shell script, make sure the first line is ``#!/bin/bash``.
-
-Note that by default the build will not be stopped if an error occurs inside a shell script.
-You should include ``set -e`` or the equivalent at the start of the script to avoid errors being silently ignored.
-
-An example use-case of ``postBuild`` file is JupyterLab's demo on mybinder.org.
-It uses a ``postBuild`` file in a folder called ``.binder`` to `prepare
-their demo for binder `_.
-
-
-.. _start:
-
-``start`` - Run code before the user sessions starts
-====================================================
-
-A script that can contain simple commands to be run at runtime (as an
-`ENTRYPOINT `_
-to the docker container). If you want this to be a shell script, make sure the
-first line is ``#!/bin/bash``. The last line must be ``exec "$@"``
-or equivalent.
-
-Use this to set environment variables that software installed in your container
-expects to be set. This script is executed each time your binder is started and
-should at most take a few seconds to run.
-
-If you only need to run things once during the build phase use :ref:`postBuild`.
-
-
-.. TODO: Discuss runtime limits, best practices, etc.
-
-.. _runtime.txt:
-
-``runtime.txt`` - Specifying runtimes
-=====================================
-
-Sometimes you want to specify the version of the runtime
-(e.g. the version of Python or R),
-but the environment specification format will not let you specify this information
-(e.g. requirements.txt or install.R).
-For these cases, we have a special file, ``runtime.txt``.
-
-.. note::
-
- ``runtime.txt`` is only supported when used with environment specifications
- that do not already support specifying the runtime
- (when using ``environment.yml`` for conda or ``Project.toml`` for Julia,
- ``runtime.txt`` will be ignored).
-
-Have ``python-x.y`` in ``runtime.txt`` to run the repository with Python version x.y.
-See our `Python2 example repository `_.
-
-Have ``r----
`` in ``runtime.txt`` to run the repository with R version RVERSION and libraries from a YYYY-MM-DD snapshot of `Posit Package Manager `__.
-RVERSION can be set to 3.4, 3.5, 3.6, or to patch releases for the 3.5 and 3.6 series.
-If you do not specify a version, the latest release will be used (currently R 3.6).
-See our `R example repository `_.
-
-.. _default.nix:
-
-``default.nix`` - the nix package manager
-=========================================
-
-Specify packages to be installed by the `nix package manager `_.
-When you use this config file all other configuration files (like ``requirements.txt``)
-that specify packages are ignored. When using ``nix`` you have to specify all
-packages and dependencies explicitly, including the Jupyter notebook package that
-repo2docker expects to be installed. If you do not install Jupyter explicitly
-repo2docker will no be able to start your container.
-
-`nix-shell `_ is used to evaluate
-a ``nix`` expression written in a ``default.nix`` file. Make sure to
-`pin your nixpkgs `_
-to produce a reproducible environment.
-
-To see an example repository visit
-`nix binder example `_.
-
-
-.. _dockerfile:
-
-``Dockerfile`` - Advanced environments
-======================================
-
-In the majority of cases, providing your own Dockerfile is not necessary as the base
-images provide core functionality, compact image sizes, and efficient builds. We recommend
-trying the other configuration files before deciding to use your own Dockerfile.
-
-With Dockerfiles, a regular Docker build will be performed.
-
-.. note::
- If a Dockerfile is present, all other configuration files will be ignored.
-
-See the `Advanced Binder Documentation `_ for
-best-practices with Dockerfiles.
diff --git a/docs/source/configuration/actions.md b/docs/source/configuration/actions.md
new file mode 100644
index 000000000..a441893da
--- /dev/null
+++ b/docs/source/configuration/actions.md
@@ -0,0 +1,39 @@
+# Configuration files for post-build actions
+
+These files control behavior that happens _after_ the image is initially built (AKA, after the packages and languages have been installed). It's useful if you need to ensure files are in a particular place, or certain commands are run when a new session launches.
+
+:::{note}
+After building the image, all actions are run as a user named `jovyan`.
+You do not have root permissions on the machine!
+
+See [the Jupyter Docker Stacks definition of `jovyan`](https://jupyter-docker-stacks.readthedocs.io/en/latest/using/faq.html#who-is-jovyan) for more details.
+:::
+
+(postbuild)=
+
+## `postBuild` - Run code after installing the environment
+
+A script that can contain arbitrary commands to be run after the whole repository has been built. If you want this to be a shell script, make sure the first line is `#!/bin/bash`.
+
+Note that by default the build will not be stopped if an error occurs inside a shell script.
+You should include `set -e` or the equivalent at the start of the script to avoid errors being silently ignored.
+
+An example use-case of `postBuild` file is JupyterLab's demo on mybinder.org.
+It uses a `postBuild` file in a folder called `.binder` to [prepare
+their demo for binder](https://github.com/jupyterlab/jupyterlab-demo/blob/HEAD/.binder/postBuild).
+
+(config-start)=
+
+## `start` - Run code before the user sessions starts
+
+A script that can contain simple commands to be run at runtime (as an
+[ENTRYPOINT](https://docs.docker.com/engine/reference/builder/#entrypoint)
+to the Docker container). If you want this to be a shell script, make sure the first line is `#!/bin/bash`. The last line must be `exec "$@"` or equivalent.
+
+Use this to set environment variables that software installed in your container
+expects to be set. This script is executed each time your binder is started and
+should at most take a few seconds to run.
+
+If you only need to run things once during the build phase use [](#postBuild).
+
+% TODO: Discuss runtime limits, best practices, etc.
diff --git a/docs/source/configuration/development.md b/docs/source/configuration/development.md
new file mode 100644
index 000000000..1084303c1
--- /dev/null
+++ b/docs/source/configuration/development.md
@@ -0,0 +1,33 @@
+# Configuration files for software development workflows
+
+(pipfile)=
+
+## `Pipfile` and/or `Pipfile.lock` - Install a Python environment
+
+[pipenv](https://github.com/pypa/pipenv/) allows you to manage a virtual
+environment Python dependencies. When using `pipenv`, you end up with
+`Pipfile` and `Pipfile.lock` files. The lock file contains explicit details
+about the packages that has been installed that met the criteria within the
+`Pipfile`.
+
+If both `Pipfile` and `Pipfile.lock` are found by `repo2docker`, the former
+will be ignored in favor of the lock file. Also note that these files
+distinguish packages and development packages and that `repo2docker` will install
+both kinds.
+
+(requirements-txt)=
+
+## `requirements.txt` - Install a Python environment
+
+This specifies a list of Python packages that should be installed in your
+environment. Our
+[requirements.txt example](https://github.com/binder-examples/requirements/blob/HEAD/requirements.txt)
+on GitHub shows a typical requirements file.
+
+(setup-py)=
+
+## `setup.py` - Install Python packages
+
+To install your repository like a Python package, you may include a
+`setup.py` file. `repo2docker` installs `setup.py` files by running
+`pip install -e .`.
diff --git a/docs/source/configuration/index.md b/docs/source/configuration/index.md
new file mode 100644
index 000000000..c64189d8c
--- /dev/null
+++ b/docs/source/configuration/index.md
@@ -0,0 +1,26 @@
+(config-files)=
+
+# Configuration files supported by repo2docker
+
+`repo2docker` looks for configuration files in the repository being built
+to determine how to build it. In general, `repo2docker` uses the same
+configuration files as other software installation tools,
+rather than creating new custom configuration files.
+
+:::{seealso}
+The [binder examples](https://github.com/binder-examples) organization on
+GitHub contains a list of sample repositories for common configurations
+that `repo2docker` can build with various configuration files such as
+Python and R installation in a repository.
+:::
+
+A list of supported configuration files (roughly in the order of build priority)
+can be found on this page (and to the right).
+
+```{toctree}
+:maxdepth: 2
+./research
+./development
+./system
+./actions
+```
diff --git a/docs/source/configuration/index.rst b/docs/source/configuration/index.rst
deleted file mode 100644
index de8e579e3..000000000
--- a/docs/source/configuration/index.rst
+++ /dev/null
@@ -1,15 +0,0 @@
-===========================
-Configuring your repository
-===========================
-
-Information about configuring your repository to work with repo2docker,
-and controlling elements of the built environment using configuration files.
-
-For information on where to put your configuration files see :ref:`usage-config-file-location`.
-
-.. toctree::
- :maxdepth: 2
- :caption: Complete list of configuration files
-
- ../config_files
- ../specification
diff --git a/docs/source/configuration/research.md b/docs/source/configuration/research.md
new file mode 100644
index 000000000..0df6d4a85
--- /dev/null
+++ b/docs/source/configuration/research.md
@@ -0,0 +1,66 @@
+# Configuration for research and data science workflows
+
+(environment-yml)=
+
+## `environment.yml` - Install a conda environment
+
+`environment.yml` is the standard configuration file used by [conda](https://conda.io)
+that lets you install any kind of package,
+including Python, R, and C/C++ packages.
+`repo2docker` does not use your `environment.yml` to create and activate a new conda environment.
+Rather, it updates a base conda environment [defined here](https://github.com/jupyterhub/repo2docker/blob/HEAD/repo2docker/buildpacks/conda/environment.yml) with the packages listed in your `environment.yml`.
+This means that the environment will always have the same default name, not the name
+specified in your `environment.yml`.
+
+:::{note}
+You can install files from pip in your `environment.yml` as well.
+For example, see the [binder-examples environment.yml](https://github.com/binder-examples/python-conda_pip/blob/HEAD/environment.yml) file. See [the `conda` environment management instructions](https://docs.conda.io/projects/conda/en/latest/user-guide/tasks/manage-environments.html#creating-an-environment-file-manually) for more information.
+:::
+
+You can also specify which Python version to install in your built environment with `environment.yml`.
+By default, `repo2docker` installs {{ default_python }} with your `environment.yml` unless you include the version of Python in the `environment.yml` of your Git repository.
+`conda` should support all versions of Python, though `repo2docker` support is best with `Python 3.7-3.11`.
+
+:::{warning}
+If you include a Python version in a `runtime.txt` file in addition to your
+`environment.yml`, your `runtime.txt` will be ignored.
+:::
+
+(install-r)=
+
+## `install.R` - Install packages with R/RStudio
+
+This is used to install R libraries pinned to a specific snapshot on
+[Posit Package Manager](https://packagemanager.posit.co/).
+For an example `install.R` file, visit our [example `install.R` file](https://github.com/binder-examples/r/blob/HEAD/install.R).
+
+To set the date of the snapshot, or to specify a specific version of R, add a [runtime.txt](#runtime-txt).
+
+(description)=
+
+## `DESCRIPTION` - Install an R package
+
+To install your repository like an R package, you may include a `DESCRIPTION` file.
+`repo2docker` installs the package and dependencies from the `DESCRIPTION` by running `devtools::install_local(getwd())`.
+
+To define the date of the package manager snapshot, add a line to [`runtime.txt`](#runtime-txt) like so:
+
+```
+r---
+```
+
+Where `YYYY-MM-DD` is a snapshot of [CRAN](https://cran.r-project.org/) that will be used for your R installation.
+If `runtime.txt` isn't provided in this case, the most recent date on CRAN will be used.
+
+(project-toml)=
+
+## `Project.toml` - Install a Julia environment
+
+A `Project.toml` (or `JuliaProject.toml`) file can specify both the
+version of Julia to be used and a list of Julia packages to be installed.
+If a `Manifest.toml` is present, it will determine the exact versions
+of the Julia packages that are installed.
+
+:::{admonition} `REQUIRE` files are no longer supported
+The recommended way of installing a Julia environment is to use a `Project.toml` file.
+:::
diff --git a/docs/source/configuration/system.md b/docs/source/configuration/system.md
new file mode 100644
index 000000000..b3f5f9626
--- /dev/null
+++ b/docs/source/configuration/system.md
@@ -0,0 +1,72 @@
+# System-wide configuration
+
+(apt-txt)=
+
+## `apt.txt` - Install packages with apt-get
+
+A list of Debian packages that should be installed. The base image used is usually the latest released
+version of Ubuntu.
+
+We use `apt.txt`, for example, to install LaTeX in our
+[example apt.txt for LaTeX](https://github.com/binder-examples/latex/blob/HEAD/apt.txt).
+
+(runtime-txt)=
+
+## `runtime.txt` - Specifying runtimes
+
+Sometimes you want to specify the version of the runtime (e.g. the version of Python or R), but the environment specification format will not let you specify this information (e.g. `requirements.txt` or `install.R`).
+For these cases, we have a special file, `runtime.txt`.
+
+:::{warning}
+`runtime.txt` is only supported when used with environment specifications
+that do not already support specifying the runtime
+(when using [`environment.yml`](#environment-yml) for conda or [`Project.toml`](#project-toml) for Julia, `runtime.txt` will be ignored).
+:::
+
+### Set the Python version
+
+Add the line `python-x.y` in `runtime.txt` to run the repository with Python version x.y.
+See our [Python2 example repository](https://github.com/binder-examples/python2_runtime/blob/HEAD/runtime.txt).
+
+### Set the R version
+
+Add the line `r----
` in `runtime.txt` to run the repository with R version `RVERSION` and libraries from a `YYYY-MM-DD` snapshot of the [Posit Package Manager](https://packagemanager.posit.co/client/#/repos/2/overview).
+
+`RVERSION` can be set to 3.4, 3.5, 3.6, or to patch releases for the 3.5 and 3.6 series.
+If you do not specify a version, the latest release will be used.
+
+See our [R example repository](https://github.com/binder-examples/r/blob/HEAD/runtime.txt).
+
+(default-nix)=
+
+## `default.nix` - the nix package manager
+
+Specify packages to be installed by the [nix package manager](https://github.com/NixOS/nixpkgs).
+When you use this config file all other configuration files (like `requirements.txt`)
+that specify packages are ignored. When using `nix` you have to specify all
+packages and dependencies explicitly, including the Jupyter notebook package that
+`repo2docker` expects to be installed. If you do not install Jupyter explicitly
+`repo2docker` will not be able to start your container.
+
+[nix-shell](https://nixos.org/nix/manual/#sec-nix-shell) is used to evaluate
+a `nix` expression written in a `default.nix` file. Make sure to
+[pin your nixpkgs](https://discourse.nixos.org/t/nixops-pinning-nixpkgs/734)
+to produce a reproducible environment.
+
+To see an example repository visit
+[nix binder example](https://github.com/binder-examples/nix).
+
+(dockerfile)=
+
+## `Dockerfile` - Advanced environments
+
+In the majority of cases, providing your own `Dockerfile` is not necessary as the base images provide core functionality, compact image sizes, and efficient builds. We recommend trying the other configuration files before deciding to use your own `Dockerfile`.
+
+With `Dockerfile`s, a regular Docker build will be performed.
+
+:::{warning}
+If a Dockerfile is present, all other configuration files will be ignored.
+:::
+
+See the [Advanced Binder Documentation](https://mybinder.readthedocs.io/en/latest/tutorials/dockerfile.html) for
+best-practices with Dockerfiles.
diff --git a/docs/source/contributing/buildpack.md b/docs/source/contribute/buildpack.md
similarity index 100%
rename from docs/source/contributing/buildpack.md
rename to docs/source/contribute/buildpack.md
diff --git a/docs/source/contributing/contentprovider.rst b/docs/source/contribute/contentprovider.rst
similarity index 100%
rename from docs/source/contributing/contentprovider.rst
rename to docs/source/contribute/contentprovider.rst
diff --git a/docs/source/contributing/contributing.md b/docs/source/contribute/contributing.md
similarity index 99%
rename from docs/source/contributing/contributing.md
rename to docs/source/contribute/contributing.md
index 8e26bcda2..d68ea5d4b 100644
--- a/docs/source/contributing/contributing.md
+++ b/docs/source/contribute/contributing.md
@@ -150,7 +150,7 @@ according to black's style guide. You can activate it with `pre-commit install`.
As part of our continuous integration tests we will check that code is
formatted properly and the tests will fail if this is not the case.
-### Verify that docker is installed and running
+### Verify that Docker is installed and running
If you do not already have [Docker](https://www.docker.com/), you should be able
to download and install it for your operating system using the links from the
diff --git a/docs/source/contributing/index.rst b/docs/source/contribute/index.rst
similarity index 71%
rename from docs/source/contributing/index.rst
rename to docs/source/contribute/index.rst
index 09939de58..75f2342da 100644
--- a/docs/source/contributing/index.rst
+++ b/docs/source/contribute/index.rst
@@ -1,6 +1,6 @@
-============
-Contributing
-============
+=================
+Contributor guide
+=================
The repo2docker community is welcoming of all kinds of help and
participation from others. Below are a few ways that you can get involved,
@@ -8,11 +8,16 @@ as well as resources for understanding the structure and design of the
repo2docker package.
.. toctree::
+ :caption: Contributing to repo2docker
contributing
roadmap
+ tasks
+
+.. toctree::
+ :caption: Developer guide
+
../architecture
../design
- tasks
buildpack
contentprovider
diff --git a/docs/source/contributing/roadmap.md b/docs/source/contribute/roadmap.md
similarity index 100%
rename from docs/source/contributing/roadmap.md
rename to docs/source/contribute/roadmap.md
diff --git a/docs/source/contributing/tasks.md b/docs/source/contribute/tasks.md
similarity index 95%
rename from docs/source/contributing/tasks.md
rename to docs/source/contribute/tasks.md
index 30be8c662..6e5835558 100644
--- a/docs/source/contributing/tasks.md
+++ b/docs/source/contribute/tasks.md
@@ -39,10 +39,9 @@ Mercurial and hg-evolve), one can use the environment variable
Some of the tests have non-python requirements for your development machine. They are:
- `git-lfs` must be installed ([instructions](https://github.com/git-lfs/git-lfs)). It need not be activated -- there is no need to run the `git lfs install` command. It just needs to be available to the test suite.
-
- If your test failure messages include "`git-lfs filter-process: git-lfs: command not found`", this step should address the problem.
-- Minimum Docker Image size of 128GB is required. If you are not running docker on a linux OS, you may need to expand the runtime image size for your installation. See Docker's instructions for [macOS](https://docs.docker.com/docker-for-mac/space/) or [Windows 10](https://docs.docker.com/docker-for-windows/#resources) for more information.
+- Minimum Docker Image size of `128GB` is required. If you are not running Docker on a Linux OS, you may need to expand the runtime image size for your installation. See Docker's [instructions for macOS](https://docs.docker.com/docker-for-mac/space/) or [instructions for Windows 10](https://docs.docker.com/docker-for-windows/#resources) for more information.
- If your test failure messages include "`No space left on device: '/home/...`", this step should address the problem.
## Update and Freeze BuildPack Dependencies
@@ -69,7 +68,6 @@ See the subsections below for more detailed instructions.
### Conda dependencies
1. There are two files related to conda dependencies. Edit as needed.
-
- `repo2docker/buildpacks/conda/environment.yml`
Contains list of packages to install in Python3 conda environments,
diff --git a/docs/source/design.md b/docs/source/design.md
index 6077621d7..6d7cdcad7 100644
--- a/docs/source/design.md
+++ b/docs/source/design.md
@@ -43,7 +43,7 @@ This provides a few advantages:
Many ingredients go into making an image from a repository:
-1. version of the base docker image
+1. version of the base Docker image
1. version of `repo2docker` itself
1. versions of the libraries installed by the repository
@@ -62,7 +62,7 @@ explicitly in your dependencies.
`repo2docker` should do one thing, and do it well. This one thing is:
-> Given a repository, deterministically build a docker image from
+> Given a repository, deterministically build a Docker image from
> it.
There's also some convenience code (to run the built image) for users, but
diff --git a/docs/source/faq.md b/docs/source/faq.md
new file mode 100644
index 000000000..43187b89b
--- /dev/null
+++ b/docs/source/faq.md
@@ -0,0 +1,106 @@
+(faq)=
+
+# Frequently Asked Questions (FAQ)
+
+A collection of frequently asked questions with answers. If you have a question
+and have found an answer, send a PR to add it here!
+
+## Why is my repository failing to build with `ResolvePackageNotFound` ?
+
+If you used `conda env export` to generate your `environment.yml` it will generate a list of packages and versions of packages that is pinned to platform specific versions.
+These very specific versions are not available in the Linux Docker image used by `repo2docker`. A typical error message will look like the following:
+
+```
+Step 39/44 : RUN conda env update -n root -f "environment.yml" && conda clean -tipsy && conda list -n root
+---> Running in ebe9a67762e4
+Solving environment: ...working... failed
+
+ResolvePackageNotFound:
+- jsonschema==2.6.0=py36hb385e00_0
+- libedit==3.1.20181209=hb402a30_0
+- tornado==5.1.1=py36h1de35cc_0
+...
+```
+
+We recommend to use `conda env export --no-builds -f environment.yml` to export
+your environment and then edit the file by hand to remove platform specific
+packages like `appnope`.
+
+See {ref}`export-environment` for a recipe on how to create strict exports of
+your environment that will work with `repo2docker`.
+
+## Can I add executable files to the user's PATH?
+
+Yes! With a [](#postBuild) file, you can place any files that should be called from the command line in the folder `~/.local/bin`.
+This folder will be available in a user's PATH, and can be run from the command line (or as a subsequent build step.)
+
+## Can I use repo2docker to bootstrap my own `Dockerfile` to edit by hand?
+
+No, you can't.
+
+If you pass the `--debug` flag to `repo2docker`, it outputs the
+intermediate `Dockerfile` that is used to build the Docker image. While
+it is tempting to copy this as a base for your own `Dockerfile`, that is
+not supported & in most cases will not work. The `--debug` output is
+just our intermediate generated `Dockerfile`, and is meant to be built
+in a very specific way. Hence the output of `--debug` can not be
+built with a normal `docker build -t .` or similar traditional
+Docker command.
+
+Check out the [binder-examples](http://github.com/binder-examples/) GitHub
+organization for example repositories you can copy & modify for your own use!
+
+## Can I use repo2docker to edit a local host repository within a Docker environment?
+
+Yes: use the `--editable` or `-E` flag (don't confuse this with
+the `-e` flag for environment variables), and run repo2docker on a
+local repository:
+
+```
+repo2docker -E my-repository/
+```
+
+This builds a Docker container from the files in that repository
+(using, for example, a `requirements.txt` or `install.R` file),
+then runs that container, while connecting the working directory
+inside the container to the local repository outside the
+container. For example, in case there is a notebook file (`.ipynb`),
+this will open in a local web browser, and one can edit it and save
+it. The resulting notebook is updated in both the Docker container and
+the local repository. Once the container is exited, the changed file
+will still be in the local repository.
+
+This allows for easy testing of the container while debugging some
+items, as well as using a fully customizable container to edit
+notebooks (among others).
+
+:::{note}
+Editable mode is a convenience option that will bind the
+repository to the container working directory (usually
+`$HOME`). If you need to mount to a different location in
+the container, use the `--volumes` option instead. Similarly,
+for a fully customized user Dockerfile, this option is not
+guaranteed to work.
+:::
+
+## Why is my R shiny app not launching?
+
+If you are trying to run an R shiny app using the `/shiny/folder_containing_shiny`
+url option, but the launch returns "The application exited during initialization.",
+there might be something wrong with the specification of the app. One way of debugging
+the app in the container is by running the `rstudio` url, open either the ui or
+server file for the app, and run the app in the container rstudio. This way you can
+see the rstudio logs as it tries to initialize the shiny app. If you are missing a
+package or other dependency for the container, this will be obvious at this stage.
+
+## Why does repo2docker need to exist? Why not use tool like source2image?
+
+The Jupyter community believes strongly in building on top of pre-existing tools whenever
+possible (this is why repo2docker buildpacks largely build off of patterns that already
+exist in the data analytics community). We try to perform due-diligence and search for
+other communities to leverage and help, but sometimes it makes the most sense to build
+our own new tool. In the case of repo2docker, we spent time integrating with a pre-existing
+tool called [source2image](https://github.com/openshift/source-to-image/).
+This is an excellent open tool for containerization, but we
+ultimately decided that it did not fit the use-case we wanted to address. For more information,
+read our [blog post about why we built repo2docker](https://github.com/yuvipanda/words/blob/fd096dd49d87e624acd8bdf6d13c0cecb930bb3f/content/post/why-not-s2i.md).
diff --git a/docs/source/faq.rst b/docs/source/faq.rst
deleted file mode 100644
index d643264b8..000000000
--- a/docs/source/faq.rst
+++ /dev/null
@@ -1,192 +0,0 @@
-.. _faq:
-
-Frequently Asked Questions (FAQ)
-================================
-
-A collection of frequently asked questions with answers. If you have a question
-and have found an answer, send a PR to add it here!
-
-How should I specify another version of Python?
------------------------------------------------
-
-One can specify a Python version in the ``environment.yml`` file of a repository
-or ``runtime.txt`` file if using ``requirements.txt`` instead of ``environment.yml``.
-
-What versions of Python (or R or Julia...) are supported?
----------------------------------------------------------
-
-Python
-~~~~~~
-
-Repo2docker officially supports the following versions of Python
-(specified in your :ref:`environment.yml ` or
-:ref:`runtime.txt ` file):
-
-- 3.11 (added in 2023)
-- 3.10 (added in 2022, default in 2023)
-- 3.9 (added in 2021)
-- 3.8 (added in 0.11)
-- 3.7 (added in 0.7, default in 0.8)
-- 3.6 (default in 0.7 and earlier)
-- 3.5
-- 2.7
-
-Additional versions may work, as long as the
-`base environment `_
-can be installed for your version of Python.
-The most likely source of incompatibility is if one of the packages
-in the base environment is not packaged for your Python,
-either because the version of the package is too new and your chosen Python is too old,
-or vice versa.
-
-If an old version of Python is specified (3.6 or earlier in 2023), a separate environment for the kernel will be installed with your requested Python version.
-The notebook server will run in the default |default_python| environment.
-That is, your _notebooks_ will run with Python 3.6, while your notebook _server_ will run with |default_python|.
-
-These two environments can be distinguished with ``$NB_PYTHON_PREFIX/bin/python`` for the server and ``$KERNEL_PYTHON_PREFIX/bin/python`` for the kernel.
-Both of these environment variables area always defined, even when they are the same.
-
-Starting in 2023, the default version of Python used when Python version is unspecified will be updated more often.
-Python itself releases a new version every year now, and repo2docker will follow, with the default Python version generally trailing the latest stable version of Python itself by 1-2 versions.
-
-If you choose not to specify a Python version, your repository is _guaranteed_ to stop working, eventually.
-We **strongly** recommend specifying a Python version (in environment.yml, runtime.txt, Pipfile, etc.)
-
-Julia
-~~~~~
-
-All Julia versions since Julia 1.3 are supported via a :ref:`Project.toml `
-file, and this is the recommended way to install Julia environments.
-
-Julia < 1.3 and the older Julia REQUIRE file is no longer supported because required infrastructure has been removed.
-
-R
-~
-
-The default version of R is currently R 4.2. You can select the version of
-R you want to use by specifying it in the :ref:`runtime.txt `
-file.
-
-We support R versions 3.4, 3.5, 3.6, 4.0, 4.1 and 4.2.
-
-
-Why is my repository failing to build with ``ResolvePackageNotFound`` ?
---------------------------------------------------------------------------
-
-If you used ``conda env export`` to generate your ``environment.yml`` it will
-generate a list of packages and versions of packages that is pinned to platform
-specific versions. These very specific versions are not available in the linux
-docker image used by ``repo2docker``. A typical error message will look like
-the following::
-
- Step 39/44 : RUN conda env update -n root -f "environment.yml" && conda clean -tipsy && conda list -n root
- ---> Running in ebe9a67762e4
- Solving environment: ...working... failed
-
- ResolvePackageNotFound:
- - jsonschema==2.6.0=py36hb385e00_0
- - libedit==3.1.20181209=hb402a30_0
- - tornado==5.1.1=py36h1de35cc_0
- ...
-
-We recommend to use ``conda env export --no-builds -f environment.yml`` to export
-your environment and then edit the file by hand to remove platform specific
-packages like ``appnope``.
-
-See :ref:`export-environment` for a recipe on how to create strict exports of
-your environment that will work with ``repo2docker``.
-
-
-Can I add executable files to the user's PATH?
-----------------------------------------------
-
-Yes! With a :ref:`postBuild` file, you can place any files that should be called
-from the command line in the folder ``~/.local/``. This folder will be
-available in a user's PATH, and can be run from the command line (or as
-a subsequent build step.)
-
-How do I set environment variables?
------------------------------------
-
-To configure environment variables for all users of a repository use the
-:ref:`start ` configuration file.
-
-When running repo2docker locally you can use the ``-e`` or ``--env`` command-line
-flag for each variable that you want to define.
-
-For example ``jupyter-repo2docker -e VAR1=val1 -e VAR2=val2 ...``
-
-Can I use repo2docker to bootstrap my own Dockerfile?
------------------------------------------------------
-
-No, you can't.
-
-If you pass the ``--debug`` flag to ``repo2docker``, it outputs the
-intermediate Dockerfile that is used to build the docker image. While
-it is tempting to copy this as a base for your own Dockerfile, that is
-not supported & in most cases will not work. The ``--debug`` output is
-just our intermediate generated Dockerfile, and is meant to be built
-in a very specific way. Hence the output of ``--debug`` can not be
-built with a normal ``docker build -t .`` or similar traditional
-docker command.
-
-Check out the `binder-examples `_ GitHub
-organization for example repositories you can copy & modify for your own use!
-
-Can I use repo2docker to edit a local host repository within a Docker environment?
-----------------------------------------------------------------------------------
-
-Yes: use the ``--editable`` or ``-E`` flag (don't confuse this with
-the ``-e`` flag for environment variables), and run repo2docker on a
-local repository::
-
- repo2docker -E my-repository/
-
-This builds a Docker container from the files in that repository
-(using, for example, a ``requirements.txt`` or ``install.R`` file),
-then runs that container, while connecting the working directory
-inside the container to the local repository outside the
-container. For example, in case there is a notebook file (``.ipynb``),
-this will open in a local web browser, and one can edit it and save
-it. The resulting notebook is updated in both the Docker container and
-the local repository. Once the container is exited, the changed file
-will still be in the local repository.
-
-This allows for easy testing of the container while debugging some
-items, as well as using a fully customizable container to edit
-notebooks (among others).
-
-.. note::
-
- Editable mode is a convenience option that will bind the
- repository to the container working directory (usually
- ``$HOME``). If you need to mount to a different location in
- the container, use the ``--volumes`` option instead. Similarly,
- for a fully customized user Dockerfile, this option is not
- guaranteed to work.
-
-
-Why is my R shiny app not launching?
-----------------------------------------------------------------------------------
-
-If you are trying to run an R shiny app using the ``/shiny/folder_containing_shiny``
-url option, but the launch returns "The application exited during initialization.",
-there might be something wrong with the specification of the app. One way of debugging
-the app in the container is by running the ``rstudio`` url, open either the ui or
-server file for the app, and run the app in the container rstudio. This way you can
-see the rstudio logs as it tries to initialise the shiny app. If you a missing a
-package or other dependency for the container, this will be obvious at this stage.
-
-
-Why does repo2docker need to exist? Why not use tool like source2image?
------------------------------------------------------------------------
-
-The Jupyter community believes strongly in building on top of pre-existing tools whenever
-possible (this is why repo2docker buildpacks largely build off of patterns that already
-exist in the data analytics community). We try to perform due-diligence and search for
-other communities to leverage and help, but sometimes it makes the most sense to build
-our own new tool. In the case of repo2docker, we spent time integrating with a pre-existing
-tool called `source2image `_.
-This is an excellent open tool for containerization, but we
-ultimately decided that it did not fit the use-case we wanted to address. For more information,
-`here `_ is a short blog post about the decision and the reasoning behind it.
diff --git a/docs/source/getting-started/index.rst b/docs/source/getting-started/index.rst
deleted file mode 100644
index 96e5f2958..000000000
--- a/docs/source/getting-started/index.rst
+++ /dev/null
@@ -1,13 +0,0 @@
-===============
-Getting Started
-===============
-
-Instructions and information on how to get started with repo2docker
-on your own machine. Select from the pages listed below to begin.
-
-.. toctree::
- :maxdepth: 2
-
- ../install
- ../usage
- ../faq
\ No newline at end of file
diff --git a/docs/source/howto/breaking_changes.md b/docs/source/howto/breaking_changes.md
index 80ab1ed09..c51e4d686 100644
--- a/docs/source/howto/breaking_changes.md
+++ b/docs/source/howto/breaking_changes.md
@@ -6,13 +6,13 @@ Repo2docker occasionally has to make breaking changes in how repositories are bu
The base image used by repo2docker was [upgraded from Ubuntu 18.04 to Ubuntu 22.04](https://github.com/jupyterhub/repo2docker/pull/1287) in version 2023.10.0 due to Ubuntu 18.04 going out of support.
-This is unlikely to affect you unless you are using {ref}`apt.txt `.
+This is unlikely to affect you unless you are using {ref}`apt.txt `.
-{ref}`apt.txt ` installs packages from the official Ubuntu package repositories, and is intrinsically tied to the Ubuntu version.
+{ref}`apt.txt ` installs packages from the official Ubuntu package repositories, and is intrinsically tied to the Ubuntu version.
Many packages will be available in both Ubuntu 18.04 and Ubuntu 22.04, however some may be renamed (for example if multiple incompatible versions are available).
Some packages may be removed, or may not be compatible with the previous version.
-In this case you should see if your packages can be installed using a {ref}`Conda environment.yml file ` using either the default [conda-forge channel](https://conda-forge.org/feedstock-outputs/) or in one of the many [third-party channels](https://docs.conda.io/projects/conda/en/latest/user-guide/concepts/channels.html).
+In this case you should see if your packages can be installed using a {ref}`Conda environment.yml file ` using either the default [conda-forge channel](https://conda-forge.org/feedstock-outputs/) or in one of the many [third-party channels](https://docs.conda.io/projects/conda/en/latest/user-guide/concepts/channels.html).
Alternatively you can try installing the packages from source, using a {ref}`postBuild ` script.
diff --git a/docs/source/howto/export_environment.rst b/docs/source/howto/export_environment.rst
index 4aa87f980..0aa72fd25 100644
--- a/docs/source/howto/export_environment.rst
+++ b/docs/source/howto/export_environment.rst
@@ -1,7 +1,7 @@
.. _export-environment:
=============================================================================
-How to automatically create a ``environment.yml`` that works with repo2docker
+Export your environment to an ``environment.yml`` that works with repo2docker
=============================================================================
This how-to explains how to create a ``environment.yml`` that specifies all
@@ -16,12 +16,9 @@ This is the most robust for reproducibility, but it does bake in potential
platform-specific packages, so you can only use an exported environment on the
same platform.
-``repo2docker`` uses a linux based image as the starting point for every docker
-image it creates. However a lot of people use OSX or Windows as their day to
-day operating system. This means that the ``environment.yml`` created by a strict
-export will not work with error messages saying that certain packages can not
-be resolved (``ResolvePackageNotFound``).
-
+``repo2docker`` uses a Linux-based image as the starting point for every Docker image it creates.
+However a lot of people use OSX or Windows as their day to day operating system.
+This means that the ``environment.yml`` created by a strict export will not work with error messages saying that certain packages can not be resolved (``ResolvePackageNotFound``).
The solution
============
@@ -48,7 +45,7 @@ Strict version export
Follow this procedure to create a strict export of your environment that will
work with ``repo2docker`` and sites like `mybinder.org `_.
-We will launch a terminal inside a basic docker image, install the packages
+We will launch a terminal inside a basic Docker image, install the packages
you need and then perform a strict export of the environment.
#. install repo2docker on your computer by following :ref:`install`
diff --git a/docs/source/howto/index.rst b/docs/source/howto/index.rst
deleted file mode 100644
index 12ce3595c..000000000
--- a/docs/source/howto/index.rst
+++ /dev/null
@@ -1,19 +0,0 @@
-=============
-How-to Guides
-=============
-
-Short, actionable guides that cover specific topics with repo2docker.
-Select from the pages listed below to get started.
-
-.. toctree::
- :maxdepth: 2
- :caption: How-To guides
-
- user_interface
- languages
- export_environment
- lab_workspaces
- breaking_changes
- jupyterhub_images
- deploy
- base_image
diff --git a/docs/source/howto/languages.md b/docs/source/howto/languages.md
new file mode 100644
index 000000000..7661a39d7
--- /dev/null
+++ b/docs/source/howto/languages.md
@@ -0,0 +1,135 @@
+(languages)=
+
+# Choose languages for your environment
+
+You can define many different languages in your configuration files. This
+page describes how to use some of the more common ones.
+
+## Python
+
+Your environment will have Python (and specified dependencies) installed when
+you use one of the following configuration files:
+
+- `requirements.txt`
+- `environment.yml`
+
+By default, the environment will have {{ default_python }}.
+
+### Specify a version of Python
+
+To specify a specific version of Python, you have two options:
+
+- Use [environment.yml](#environment-yml). Conda environments let you define
+ the Python version in `environment.yml`.
+ To do so, add `python=X.X` to your dependencies section, like so:
+
+ ```
+ name: python 2.7
+ dependencies:
+ - python=2.7
+ - numpy
+ ```
+
+- Use [runtime.txt](#runtime-txt) with [requirements.txt](#requirements-txt).
+ If you are using `requirements.txt` instead of `environment.yml`,
+ you can specify the Python runtime version in a separate file called `runtime.txt`.
+ This file contains a single line of the following form:
+
+ ```
+ python-X.X
+ ```
+
+ For example:
+
+ ```
+ python-3.6
+ ```
+
+### Supported versions of Python
+
+Repo2docker officially supports the following versions of Python:
+
+- 3.11 (added in 2023)
+- 3.10 (added in 2022, default in 2023)
+- 3.9 (added in 2021)
+- 3.8 (added in 0.11)
+- 3.7 (added in 0.7, default in 0.8)
+- 3.6 (default in 0.7 and earlier)
+- 3.5
+- 2.7
+
+Additional versions may work, as long as the [base environment](https://github.com/jupyterhub/repo2docker/blob/HEAD/repo2docker/buildpacks/conda/environment.yml) can be installed for your version of Python.
+The most likely source of incompatibility is if one of the packages in the base environment is not packaged for your Python, either because the version of the package is too new and your chosen Python is too old, or vice versa.
+
+If an old version of Python is specified (3.6 or earlier in 2023), a separate environment for the kernel will be installed with your requested Python version.
+The notebook server will run in the default {{ default_python }} environment.
+That is, your _notebooks_ will run with Python 3.6, while your notebook _server_ will run with {{ default_python }}.
+
+These two environments can be distinguished with `$NB_PYTHON_PREFIX/bin/python` for the server and `$KERNEL_PYTHON_PREFIX/bin/python` for the kernel.
+Both of these environment variables are always defined, even when they are the same.
+
+Starting in 2023, the default version of Python used when Python version is unspecified will be updated more often.
+Python itself releases a new version every year now, and repo2docker will follow, with the default Python version generally trailing the latest stable version of Python itself by 1-2 versions.
+
+If you choose not to specify a Python version, your repository is _guaranteed_ to stop working, eventually.
+We **strongly** recommend specifying a Python version (in environment.yml, runtime.txt, Pipfile, etc.)
+
+## The R Language
+
+repo2docker supports R, the open source [RStudio IDE](https://www.rstudio.com/) as well
+as Jupyter support for R with the [IRKernel](https://irkernel.github.io/). To set it up,
+you need to create a `runtime.txt` file with the following format:
+
+> r-\-\-\-\
+
+This will provide you R of given version (such as 4.1, 3.6, etc), and a CRAN snapshot
+to install libraries from on the given date. You can install more R packages from CRAN
+by adding a [install.R](#install-R) file to your repo. RStudio and IRKernel are
+installed by default for all R versions.
+
+[packagemanager.posit.co](https://packagemanager.posit.co/client/#/)
+will be used to provide much faster installations via [binary packages](https://www.rstudio.com/blog/package-manager-v1-1-no-interruptions/).
+For _some_ packages, this might require you install underlying system libraries
+using [apt.txt](#apt-txt) - look at the page for the CRAN package you are interested in at
+[packagemanager.posit.co](https://packagemanager.posit.co/client/#/) to find
+a list.
+
+repo2docker stopped using the Microsoft mirror MRAN for older R versions after its shutdown in July, 2023.
+
+### Supported versions of R
+
+The default version of R is currently R 4.2. You can select the version of
+R you want to use by specifying it in the [runtime.txt](#runtime-txt)
+file.
+
+We support R versions 3.4, 3.5, 3.6, 4.0, 4.1 and 4.2.
+
+## Julia
+
+To build an environment with Julia, include a configuration file called
+`Project.toml`. The format of this file is documented at
+[the Julia Pkg.jl documentation](https://julialang.github.io/Pkg.jl/v1/).
+To specify a specific version of Julia to install, put a Julia version in the
+`[compat]` section of the `Project.toml` file, as described
+here: .
+
+### Supported versions of Julia
+
+All Julia versions since Julia 1.3 are supported via a [Project.toml](project-toml)
+file, and this is the recommended way to install Julia environments.
+
+Julia < 1.3 and the older Julia REQUIRE file is no longer supported because required infrastructure has been removed.
+
+## Languages not covered here
+
+If a language is not "officially" supported by a build pack, it can often be
+installed with a `postBuild` script. This will run arbitrary `bash` commands,
+and can be used to download / install a language.
+
+## Using multiple languages at once
+
+It may also be possible to combine multiple languages in a single environment.
+The details on how to accomplish this with all possible combinations are outside
+the scope of this guide. However we recommend that you take a look at the
+[Multi-Language Demo](https://github.com/binder-examples/multi-language-demo)
+repository for some inspiration.
diff --git a/docs/source/howto/languages.rst b/docs/source/howto/languages.rst
deleted file mode 100644
index af51a6fdf..000000000
--- a/docs/source/howto/languages.rst
+++ /dev/null
@@ -1,102 +0,0 @@
-.. _languages:
-
-=====================================
-Choose languages for your environment
-=====================================
-
-You can define many different languages in your configuration files. This
-page describes how to use some of the more common ones.
-
-Python
-======
-
-Your environment will have Python (and specified dependencies) installed when
-you use one of the following configuration files:
-
-* ``requirements.txt``
-* ``environment.yml``
-
-.. note::
-
- By default, the environment will have |default_python|.
-
-.. versionchanged:: 0.8
-
- Upgraded default Python from 3.6 to 3.7.
-
-
-Specifying a version of Python
-------------------------------
-
-To specify a specific version of Python, you have two options:
-
-* Use :ref:`environment.yml `. Conda environments let you define
- the Python version in ``environment.yml``.
- To do so, add ``python=X.X`` to your dependencies section, like so::
-
- name: python 2.7
- dependencies:
- - python=2.7
- - numpy
-
-* Use :ref:`runtime.txt ` with :ref:`requirements.txt `.
- If you are using ``requirements.txt`` instead of ``environment.yml``,
- you can specify the Python runtime version in a separate file called ``runtime.txt``.
- This file contains a single line of the following form::
-
- python-X.X
-
- For example::
-
- python-3.6
-
-
-The R Language
-==============
-
-repo2docker supports R, the open source `RStudio IDE `_ as well
-as Jupyter support for R with the `IRKernel `_. To set it up,
-you need to create a ``runtime.txt`` file with the following format:
-
- r----
-
-This will provide you R of given version (such as 4.1, 3.6, etc), and a CRAN snapshot
-to install libraries from on the given date. You can install more R packages from CRAN
-by adding a :ref:`install.R` file to your repo. RStudio and IRKernel are
-installed by default for all R versions.
-
-`packagemanager.posit.co `_
-will be used to provide much faster installations via `binary packages `_.
-For *some* packages, this might require you install underlying system libraries
-using :ref:`apt.txt` - look at the page for the CRAN package you are interested in at
-`packagemanager.posit.co `_ to find
-a list.
-
-repo2docker stopped using the Microsoft mirror MRAN for older R versions after its shutdown in July, 2023.
-
-
-Julia
-=====
-
-To build an environment with Julia, include a configuration file called
-``Project.toml``. The format of this file is documented at
-`the Julia Pkg.jl documentation `_.
-To specify a specific version of Julia to install, put a Julia version in the
-``[compat]`` section of the ``Project.toml`` file, as described
-here: https://julialang.github.io/Pkg.jl/v1/compatibility/.
-
-Languages not covered here
-==========================
-
-If a language is not "officially" supported by a build pack, it can often be
-installed with a ``postBuild`` script. This will run arbitrary ``bash`` commands,
-and can be used to download / install a language.
-
-Using multiple languages at once
-================================
-
-It may also be possible to combine multiple languages in a single environment.
-The details on how to accomplish this with all possible combinations are outside
-the scope of this guide. However we recommend that you take a look at the
-`Multi-Language Demo `_
-repository for some inspiration.
diff --git a/docs/source/index.md b/docs/source/index.md
new file mode 100644
index 000000000..43593c8a9
--- /dev/null
+++ b/docs/source/index.md
@@ -0,0 +1,91 @@
+# Welcome to `repo2docker`'s documentation
+
+`repo2docker` lets you **reproducibly build, run, and deploy user environment images for interactive computing and data workflows from source code repositories**.
+
+`repo2docker` can be used to explore a repository locally by building and executing the
+constructed image of the repository, or as a means of building images that
+are pushed to a Docker registry. It is the tool used by [BinderHub](https://binderhub.readthedocs.io) to build images on demand.
+
+::::{grid}
+:::{grid-item-card} 🔧 Build reproducible data science environments from repositories
+Build a reproducible data science environment as a Docker image and execute code interactively. Use many [configuration files](#config-files) to control language, tools, and setup instructions.
+:::
+:::{grid-item-card} 🚀 Deploy environments in JupyterHub or Binder
+Push environment images to a Docker registry for re-use in data science environment services like [JupyterHub](https://jupyterhub.readthedocs.io) or [a Binder instance](https://mybinder.org), or for other communities to build upon your base environment.
+:::
+:::{grid-item-card} ☁️ Host repositories in many providers
+Host repositories in a provider like GitHub, an open science repository like [Zenodo](https://zenodo.org) or [Figshare](https://figshare.com), a hosted data platform like a [Dataverse installation](https://dataverse.org/), an archive like the
+[Software Heritage Archive](https://archive.softwareheritage.org).
+:::
+::::
+
+## What is a user image and why would I build one with `repo2docker`?
+
+A **user image** contains the entire software environment that a user may access from an interactive data science session. For example, it might contain many **programming languages**, **software for data analysis**, or even **content files and datasets** available to anybody that accesses that environment. User images are built with [Docker](https://www.docker.com/), a standard open source tool for defining, building, and deploying images.
+
+Many data science platforms and services like [JupyterHub](https://jupyterhub.readthedocs.io) and [Binder](https://mybinder.org) launch interactive data science sessions **with a user image attached**, meaning that the user gains access to whatever is in the image. In short, this allows somebody to define and build the user image one time, in a way that users can reproducibly re-use many times.
+
+Dockerfiles are the standard way that Docker images are defined[^use-dockerfile].
+However, **building images with Dockerfiles takes expertise and time** that is outside the scope of a data scientist or researcher. `repo2docker` makes this workflow more accessible by using community-standard configuration files for computational reproducibility to build user images.
+
+In short, `repo2docker` is a tool to reproducibly build, run, and deploy these user environment Docker images for data science from source code repositories.
+
+[^use-dockerfile]: That said, you can still use a Dockerfile if you really want to!
+
+## How `repo2docker` works
+
+When you call `repo2docker` like so:
+
+```
+jupyter-repo2docker
+```
+
+It performs these steps:
+
+1. Inspects the repository for [configuration files](#config-files). These will be used to build the environment needed to run the repository.
+2. Builds a Docker image with an environment specified in these [configuration files](#config-files).
+3. Runs the image to let you explore the repository interactively via Jupyter notebooks, RStudio, or many other interfaces (this is optional).
+4. Pushes the images to a Docker registry so that it may be accessed remotely (this is optional).
+
+[swhid]: https://docs.softwareheritage.org/devel/swh-model/persistent-identifiers.html
+
+Please report [Bugs](https://github.com/jupyterhub/repo2docker/issues),
+[ask questions](https://gitter.im/jupyterhub/binder) or
+[contribute to the project](https://github.com/jupyterhub/repo2docker/blob/HEAD/CONTRIBUTING.md).
+
+## Get started with repo2docker
+
+This tutorial walks you through setting up repo2docker, building your first environment image, and running it locally with Docker.
+
+```{toctree}
+:maxdepth: 2
+
+start
+```
+
+## Learn how to use repo2docker
+
+Our user guide provides all of the information you need to configure and deploy the environment image you want.
+
+```{toctree}
+:maxdepth: 2
+use/index
+```
+
+## Contribute to repo2docker
+
+Our contributor guide describes how you can follow along with the project, learn how to collaborate with our open team, and learn developer workflows and information.
+
+```{toctree}
+:maxdepth: 2
+
+contribute/index
+```
+
+```{toctree}
+:caption: Changelog
+:maxdepth: 2
+:hidden: true
+
+changelog
+```
diff --git a/docs/source/index.rst b/docs/source/index.rst
deleted file mode 100644
index 09be2503f..000000000
--- a/docs/source/index.rst
+++ /dev/null
@@ -1,42 +0,0 @@
-jupyter-repo2docker
-===================
-
-``jupyter-repo2docker`` is a tool to **build, run, and push Docker
-images from source code repositories**.
-
-``repo2docker`` fetches a repository
-(from GitHub, GitLab, Zenodo, Figshare, Dataverse installations, a Git repository or a local directory)
-and builds a container image in which the code can be executed.
-The image build process is based on the configuration files found in the repository.
-
-``repo2docker`` can be
-used to explore a repository locally by building and executing the
-constructed image of the repository, or as a means of building images that
-are pushed to a Docker registry.
-
-``repo2docker`` is the tool used by `BinderHub `_
-to build images on demand.
-
-Please report `Bugs `_,
-`ask questions `_ or
-`contribute to the project `_.
-
-.. toctree::
- :maxdepth: 2
- :caption: Getting started with repo2docker
-
- getting-started/index
- howto/index
- configuration/index
-
-.. toctree::
- :maxdepth: 2
- :caption: Contribute to repo2docker
-
- contributing/index
-
-.. toctree::
- :maxdepth: 2
- :caption: Changelog
-
- changelog
diff --git a/docs/source/install.rst b/docs/source/install.rst
deleted file mode 100644
index be9c91b23..000000000
--- a/docs/source/install.rst
+++ /dev/null
@@ -1,79 +0,0 @@
-.. _install:
-
-Installing ``repo2docker``
-==========================
-
-repo2docker requires Python 3.6 or above on Linux and macOS. See
-:ref:`below ` for more information about Windows support.
-
-Prerequisite: Docker
---------------------
-
-Install `Docker `_ as it is required
-to build Docker images. The
-`Community Edition `_,
-is available for free.
-
-Recent versions of Docker are recommended.
-The latest version of Docker, ``18.03``, successfully builds repositories from
-`binder-examples `_.
-The `BinderHub `_ helm chart uses version
-``17.11.0-ce-dind``. See the
-`helm chart `_
-for more details.
-
-Optional: Mercurial
--------------------
-
-For `Mercurial `_ repositories, Mercurial and
-`hg-evolve `_ need to be
-installed. For example, on Debian based distributions, one can do::
-
- sudo apt install mercurial
- $(hg debuginstall --template "{pythonexe}") -m pip install hg-evolve --user
-
-To install Mercurial on other systems, see `here
-`_.
-
-Note that for old Mercurial versions, you may need to specify a version for
-hg-evolve. For example, ``hg-evolve==9.2`` for hg 4.5 (which is installed with
-`apt` on Ubuntu 18.4).
-
-Installing with ``pip``
------------------------
-
-We recommend installing ``repo2docker`` with the ``pip`` tool::
-
- python3 -m pip install jupyter-repo2docker
-
-for the latest release. To install the most recent code from the upstream repository, run::
-
- python3 -m pip install https://github.com/jupyterhub/repo2docker/archive/main.zip
-
-For information on using ``repo2docker``, see :ref:`usage`.
-
-Installing from source code
----------------------------
-
-Alternatively, you can install repo2docker from a local source tree,
-e.g. in case you are contributing back to this project::
-
- git clone https://github.com/jupyterhub/repo2docker.git
- cd repo2docker
- python3 -m pip install -e .
-
-That's it! For information on using ``repo2docker``, see
-:ref:`usage`.
-
-.. _windows:
-
-Windows support
----------------
-
-Windows support for ``repo2docker`` is still in the experimental stage.
-
-An article about `using Windows and the WSL`_ (Windows Subsytem for Linux or
-Bash on Windows) provides additional information about Windows and docker.
-
-
-.. _using Windows and the WSL: https://nickjanetakis.com/blog/setting-up-docker-for-windows-and-wsl-to-work-flawlessly
diff --git a/docs/source/specification.md b/docs/source/specification.md
new file mode 100644
index 000000000..fb7cc40c9
--- /dev/null
+++ b/docs/source/specification.md
@@ -0,0 +1,30 @@
+(specification)=
+
+# The Reproducible Execution Environment Specification
+
+repo2docker scans a repository for particular {ref}`config-files`, such
+as `requirements.txt` or `Project.toml`. The collection of files, their contents,
+and the resulting actions that repo2docker takes is known
+as the **Reproducible Execution Environment Specification** (or REES).
+
+The goal of the REES is to automate and encourage existing community best practices
+for reproducible computational environments. This includes installing packages using
+community-standard specification files and their corresponding tools,
+such as `requirements.txt` (with `pip`), `Project.toml` (with Julia), or
+`apt.txt` (with `apt`). While repo2docker automates the
+creation of the environment, a human should be able to look at a REES-compliant
+repository and reproduce the environment using common, clear steps without
+repo2docker software.
+
+Currently, the definition of the REE Specification is the following:
+
+> Any directory containing zero or more files from the {ref}`config-files` list is a
+> valid reproducible execution environment as defined by the REES. The
+> configuration files have to all be placed either in the root of the
+> directory, in a `binder/` sub-directory or a `.binder/` sub-directory.
+
+For example, the REES recognises `requirements.txt` as a valid config file.
+The file format is as defined by the `requirements.txt` standard of the Python
+community. A REES-compliant tool will install a Python interpreter (of unspecified version)
+and perform the equivalent action of `pip install -r requirements.txt` so that the
+user can afterwards run python and use the packages installed.
diff --git a/docs/source/specification.rst b/docs/source/specification.rst
deleted file mode 100644
index bd635b231..000000000
--- a/docs/source/specification.rst
+++ /dev/null
@@ -1,32 +0,0 @@
-.. _specification:
-
-====================================================
-The Reproducible Execution Environment Specification
-====================================================
-
-repo2docker scans a repository for particular :ref:`config-files`, such
-as ``requirements.txt`` or ``Project.toml``. The collection of files, their contents,
-and the resulting actions that repo2docker takes is known
-as the **Reproducible Execution Environment Specification** (or REES).
-
-The goal of the REES is to automate and encourage existing community best practices
-for reproducible computational environments. This includes installing packages using
-community-standard specification files and their corresponding tools,
-such as ``requirements.txt`` (with ``pip``), ``Project.toml`` (with Julia), or
-``apt.txt`` (with ``apt``). While repo2docker automates the
-creation of the environment, a human should be able to look at a REES-compliant
-repository and reproduce the environment using common, clear steps without
-repo2docker software.
-
-Currently, the definition of the REE Specification is the following:
-
- Any directory containing zero or more files from the :ref:`config-files` list is a
- valid reproducible execution environment as defined by the REES. The
- configuration files have to all be placed either in the root of the
- directory, in a ``binder/`` sub-directory or a ``.binder/`` sub-directory.
-
-For example, the REES recognises ``requirements.txt`` as a valid config file.
-The file format is as defined by the ``requirements.txt`` standard of the Python
-community. A REES-compliant tool will install a Python interpreter (of unspecified version)
-and perform the equivalent action of ``pip install -r requirements.txt`` so that the
-user can afterwards run python and use the packages installed.
diff --git a/docs/source/start.md b/docs/source/start.md
new file mode 100644
index 000000000..87d148fc9
--- /dev/null
+++ b/docs/source/start.md
@@ -0,0 +1,62 @@
+# Get started
+
+This tutorial guides you through installing `repo2docker` and building your first environment image.
+
+(install)=
+
+## Install `repo2docker`
+
+repo2docker requires Python 3.6 or above on Linux and macOS.
+
+:::{admonition} Windows support is experimental
+
+This [article about using Windows and the WSL](https://nickjanetakis.com/blog/setting-up-docker-for-windows-and-wsl-to-work-flawlessly) (Windows Subsystem for Linux or
+Bash on Windows) provides additional information about Windows and Docker.
+:::
+
+### Prerequisite: Install Docker
+
+Install [Docker](https://www.docker.com), as it is required to build Docker images.
+The [Community Edition](https://docs.docker.com/install/) is available for free.
+
+Recent versions of Docker are recommended.
+
+### Install `repo2docker` with `pip`
+
+We recommend installing `repo2docker` with the `pip` tool:
+
+```
+python3 -m pip install jupyter-repo2docker
+```
+
+(usage)=
+
+## Build a repository with `repo2docker`
+
+Now that you've installed Docker and `repo2docker`, we can build a repository.
+To do so, follow these steps.
+
+### Start Docker
+
+Follow the [instructions for starting Docker](https://docs.docker.com/engine/daemon/start/) to start a Docker process.
+
+### Build an image from a URL
+
+Next we'll build a reproducible image from a URL. We'll use the [Binder `requirements.txt` example](https://github.com/binder-examples/requirements), which installs a simple Python environment. Run the following command:
+
+```bash
+jupyter-repo2docker https://github.com/binder-examples/requirements
+```
+
+You'll see `repo2docker` take the following actions:
+
+1. Inspect the repository for [configuration files](#config-files). It will detect the `requirements.txt` file in the repository.
+2. Build a Docker image using the configuration files. In this case, the `requirements.txt` file will correspond to a Python environment.
+3. Run the image to let you explore the repository interactively.
+
+Click the link provided and you'll be taken to an interactive Jupyter Notebook interface where you can run commands interactively inside the environment.
+
+## Learn more
+
+This is a simple example building an environment image for your repository.
+To learn more about the kinds of source repositories, environments, and use-cases that repo2docker supports, see [the `repo2docker` user guide](./use/index.md).
diff --git a/docs/source/usage.rst b/docs/source/usage.rst
deleted file mode 100644
index e89eb8ff6..000000000
--- a/docs/source/usage.rst
+++ /dev/null
@@ -1,140 +0,0 @@
-.. _usage:
-
-=====================
-Using ``repo2docker``
-=====================
-
-.. note::
-
- `Docker `_ **must be running** in
- order to run ``repo2docker``. For more information on installing
- ``repo2docker``, see :ref:`install`.
-
-``repo2docker`` can build a reproducible computational environment for any repository that
-follows :ref:`specification`. repo2docker is called with the URL of a Git repository,
-a `DOI `_ from Zenodo or Figshare,
-a `Handle `_ or DOI from a Dataverse installation,
-a `SWHID`_ of a directory of a revision archived in the
-`Software Heritage Archive `_,
-or a path to a local directory.
-
-It then performs these steps:
-
-1. Inspects the repository for :ref:`configuration files `. These will be used to build
- the environment needed to run the repository.
-2. Builds a Docker image with an environment specified in these :ref:`configuration files `.
-3. Launches the image to let you explore the
- repository interactively via Jupyter notebooks, RStudio, or many other interfaces (optional)
-4. Pushes the images to a Docker registry so that it may be accessed remotely
- (optional)
-
-Calling repo2docker
-===================
-
-repo2docker is called with this command::
-
- jupyter-repo2docker
-
-where ```` is:
-
- * a URL of a Git repository (``https://github.com/binder-examples/requirements``),
- * a Zenodo DOI (``10.5281/zenodo.1211089``),
- * a SWHID_ (``swh:1:rev:999dd06c7f679a2714dfe5199bdca09522a29649``),
- * a URL of a CKAN_ dataset (``https://demo.ckan.org/dataset/sample-dataset-1``), or
- * a path to a local directory (``a/local/directory``)
-
-of the source repository you want to build.
-
-For example, the following command will build an image of Peter Norvig's
-Pytudes_ repository::
-
- jupyter-repo2docker https://github.com/norvig/pytudes
-
-Building the image may take a few minutes.
-
-Pytudes_
-uses a `requirements.txt file `_
-to specify its Python environment. Because of this, ``repo2docker`` will use
-``pip`` to install dependencies listed in this ``requirement.txt`` file, and
-these will be present in the generated Docker image. To learn more about
-configuration files in ``repo2docker`` visit :ref:`config-files`.
-
-When the image is built, a message will be output to your terminal::
-
- Copy/paste this URL into your browser when you connect for the first time,
- to login with a token:
- http://0.0.0.0:36511/?token=f94f8fabb92e22f5bfab116c382b4707fc2cade56ad1ace0
-
-Pasting the URL into your browser will open Jupyter Notebook with the
-dependencies and contents of the source repository in the built image.
-
-
-Building a specific branch, commit or tag
-=========================================
-
-To build a particular branch and commit, use the argument ``--ref`` and
-specify the ``branch-name`` or ``commit-hash``. For example::
-
- jupyter-repo2docker --ref 9ced85dd9a84859d0767369e58f33912a214a3cf https://github.com/norvig/pytudes
-
-.. tip::
- For reproducible builds, we recommend specifying a commit-hash to
- deterministically build a fixed version of a repository. Not specifying a
- commit-hash will result in the latest commit of the repository being built.
-
-
-.. _usage-config-file-location:
-
-Where to put configuration files
-================================
-
-``repo2docker`` will look for configuration files in:
-
-* A folder named ``binder/`` in the root of the repository.
-* A folder named ``.binder/`` in the root of the repository.
-* The root directory of the repository.
-
-Having both ``binder/`` and ``.binder/`` folders is not allowed.
-If one of these folders exists, only configuration files in that folder are considered, configuration in the root directory will be ignored.
-
-Check the complete list of :ref:`configuration files ` supported
-by ``repo2docker`` to see how to configure the build process.
-
-.. note::
-
- ``repo2docker`` builds an environment with Python 3.7 by default. If you'd
- like a different version, you can specify this in your
- :ref:`configuration files `.
-
-
-Debugging repo2docker with ``--debug`` and ``--no-build``
-=========================================================
-
-To debug the docker image being built, pass the ``--debug`` parameter:
-
- .. code-block:: bash
-
- jupyter-repo2docker --debug https://github.com/norvig/pytudes
-
-This will print the generated ``Dockerfile``, build it, and run it.
-
-To see the generated ``Dockerfile`` without actually building it,
-pass ``--no-build`` to the commandline. This ``Dockerfile`` output
-is for **debugging purposes** of ``repo2docker`` only - it can not
-be used by docker directly.
-
- .. code-block:: bash
-
- jupyter-repo2docker --no-build --debug https://github.com/norvig/pytudes
-
-
-Command line API
-================
-
-.. autoprogram:: repo2docker.__main__:argparser
- :prog: jupyter-repo2docker
-
-
-.. _Pytudes: https://github.com/norvig/pytudes
-.. _SWHID: https://docs.softwareheritage.org/devel/swh-model/persistent-identifiers.html
-.. _CKAN: https://ckan.org
diff --git a/docs/source/use/action.md b/docs/source/use/action.md
new file mode 100644
index 000000000..ec2df5649
--- /dev/null
+++ b/docs/source/use/action.md
@@ -0,0 +1,7 @@
+(github-action)=
+
+# Use GitHub Actions to build environment images
+
+The [`repo2docker` GitHub Action](https://github.com/jupyterhub/repo2docker-action) allows you to use `repo2docker` in your GitHub CI/CD to build environment images and push them to a registry. It is a useful way to automate building and pushing images to reduce deployment toil.
+
+See [the `repo2docker` GitHub Action README](https://github.com/jupyterhub/repo2docker-action) for documentation about how to use this tool.
diff --git a/docs/source/use/community-image.md b/docs/source/use/community-image.md
new file mode 100644
index 000000000..d499768c5
--- /dev/null
+++ b/docs/source/use/community-image.md
@@ -0,0 +1,54 @@
+# Use a community-maintained image in a JupyterHub
+
+The simplest way to deploy environment images to a JupyterHub or BinderHub is to _find and re-use a pre-existing image maintained by a community_.
+In this case, you won't have to maintain anything -- you'll simply use a community's image and contribute upstream if you wish.
+
+Here are some steps to follow.
+
+## First, find a community-maintained image for your workflow
+
+Many communities define their own user envionment images for re-use. Here are a few to look into:
+
+1. [jupyter Docker-stacks](https://jupyter-docker-stacks.readthedocs.io/) has a number of user images for general data science workflows.
+2. [pangeo Docker-stacks](https://github.com/pangeo-data/pangeo-docker-images) has images for geospatial workflows in the cloud
+3. [rocker](https://rocker-project.org/) has images meant for reproducibility with the R computing language.
+
+For example, the [Jupyter Docker Stacks images](https://jupyter-docker-stacks.readthedocs.io/en/latest/using/selecting.html#core-stacks) define images for several core datascience workflows. Each one points to a **hosted version of the image** on an image repository (such as [quay.io](https://quay.io)).
+
+## Pick a tag for the image
+
+Tags are a way to checkpoint an image so that you know exactly what is inside. They are similar to released versions of software. Tags will be listed in the image provider used by the community. For example:
+
+[Here's the list of tags for the `Docker-stacks-foundation` image](https://quay.io/repository/jupyter/docker-stacks-foundation?tab=tags).
+
+:::{admonition} Don't use the `latest` tag
+:class: warning
+`latest` is a special tag for "the latest tag to be released". This is generally a bad idea when using environment images for your interactive data science sessions. Here are a few reasons why:
+
+- The environment may regularly change without warning as new versions of the image are released.
+- The `JUPYTER_IMAGE` environment variable will not be useful because it's a generic `latest` variable.
+- If you use tools that _themselves_ use Docker, you might have different images on your interactive environment vs. the images the tools use, because they haven't all updated at once.
+
+Instead, we recommend periodically manually updating the tag you use over time.
+:::
+
+## Configure JupyterHub to use the tag
+
+Once you have an image and a tag, you can configure your JupyterHub to use it.
+
+Here's a sample configuration if you're using the [Zero to JupyterHub for Kubernetes distribution](https://z2jh.jupyter.org):
+
+```yaml
+singleuser:
+ image:
+ name: pangeo/pangeo-notebook
+ tag: 2023.04.15
+```
+
+Here's a sample configuration if you're using the [Dockerspawner for JupyterHub spawner](https://jupyterhub-dockerspawner.readthedocs.io/en/latest/):
+
+```python
+c.DockerSpawner.image = 'pangeo/pangeo-notebook:2023.04.15'
+```
+
+When you re-deploy your JupyterHub application, it should now spawn user environments from the image you've configured.
diff --git a/docs/source/use/extend-community-image.md b/docs/source/use/extend-community-image.md
new file mode 100644
index 000000000..f4bf5159d
--- /dev/null
+++ b/docs/source/use/extend-community-image.md
@@ -0,0 +1,91 @@
+# Extend a community image with your own packages
+
+If [using a community image alone](./community-image.md) doesn't provide the environment you need, you might get away with just installing one or two extra things on top of it.
+
+To do this, you should define a `Dockerfile` that "inherits" the community image, and "extend" it to add a few extra things.
+
+[Here's a repository that demonstrates how to do this using the `repo2docker` github action](https://github.com/2i2c-org/example-inherit-from-community-image). Below is a summary of the most important parts.
+
+A [local `environment.yml` file](https://github.com/2i2c-org/example-inherit-from-community-image/blob/main/environment.yml) defines **only the extra things to install** in addition to the upstream image.
+
+```{code-block}
+:caption: environment.yml
+channels:
+ - conda-forge
+
+dependencies:
+ - jupyterhub-singleuser>=3.0,<4.0
+
+ # Everyone wants to use nbgitpuller for everything, so let's do that
+ - nbgitpuller=1.1.*
+ # Install packages from pip
+ - pip
+ - pip:
+ - otter-grader
+```
+
+A local `Dockerfile` does the following things:
+
+- Inherits from the upstream image.
+- Copies your local `environment.yml` file into a directory in the image.
+- Updates the `conda` environment with it to install the extra packages.
+
+For example, the following inherits the `scipy-notebook` image with the `2023-05-01` tag and takes the above steps. (you could follow a similar workflow with other package managers).
+
+```{code-block} Dockerfile
+:caption: Dockerfile
+FROM jupyter/scipy-notebook:2023-05-01
+COPY environment.yml /tmp/environment.yml
+RUN mamba env update --prefix ${CONDA_DIR} --file /tmp/environment.yml
+COPY image-tests image-tests
+RUN ls
+```
+
+A [github workflow](https://github.com/2i2c-org/example-inherit-from-community-image/blob/main/.github/workflows/build.yaml) uses the [`repo2docker` GitHub action](#github-action) to build and push a Docker image using repo2docker. Here's an example of what the GitHub workflow looks like:
+
+````{dropdown} Example GitHub workflow code
+```{code-block} yaml
+:caption: .github/workflows/build.yaml
+name: Build and push container image
+
+on:
+ push:
+ branches:
+ - main
+
+jobs:
+ build-and-push:
+ runs-on: ubuntu-latest
+ steps:
+
+ # For biggish images, github actions runs out of disk space.
+ # So we cleanup some unwanted things in the disk image, and reclaim that space for our Docker use
+ # https://github.com/actions/virtual-environments/issues/2606#issuecomment-772683150
+ # and https://github.com/easimon/maximize-build-space/blob/b4d02c14493a9653fe7af06cc89ca5298071c66e/action.yml#L104
+ # This gives us a total of about 52G of free space, which should be enough for now
+ - name: cleanup disk space
+ run: |
+ sudo rm -rf /usr/local/lib/android /usr/share/dotnet /opt/ghc
+ df -h
+
+ - name: Checkout files in repo
+ uses: actions/checkout@main
+
+ - name: Build and push the image to quay.io
+ uses: jupyterhub/repo2docker-action@master
+ with:
+ # Make sure username & password/token pair matches your registry credentials
+ DOCKER_USERNAME: ${{ secrets.QUAY_USERNAME }}
+ DOCKER_PASSWORD: ${{ secrets.QUAY_PASSWORD }}
+ DOCKER_REGISTRY: "quay.io"
+ # Disable pushing a 'latest' tag, as this often just causes confusion
+ LATEST_TAG_OFF: true
+ #
+ # Uncomment and modify the following line with your image name, otherwise no push will happen
+ IMAGE_NAME: "yuvipanda/example-inherit-from-community-image"
+
+ # Lets us monitor disks getting full as images get bigger over time
+ - name: Show how much disk space is left
+ run: df -h
+```
+````
diff --git a/docs/source/use/index.md b/docs/source/use/index.md
new file mode 100644
index 000000000..7f8a953e0
--- /dev/null
+++ b/docs/source/use/index.md
@@ -0,0 +1,47 @@
+# User guide
+
+Short, actionable guides that cover specific topics with repo2docker.
+Select from the pages listed below to get started.
+
+Information about configuring your repository to work with repo2docker,
+and controlling elements of the built environment using configuration files.
+
+```{toctree}
+:caption: Image building basics
+:maxdepth: 1
+pathways
+../configuration/index
+repository
+../cli
+```
+
+```{toctree}
+:maxdepth: 1
+:caption: Customize your environment
+
+../howto/user_interface
+../howto/languages
+../howto/lab_workspaces
+../howto/export_environment
+../howto/base_image
+```
+
+```{toctree}
+:maxdepth: 1
+:caption: Deploy images into production
+
+../howto/deploy
+./community-image
+./extend-community-image
+../howto/jupyterhub_images
+./action
+../howto/breaking_changes
+```
+
+```{toctree}
+:maxdepth: 2
+:caption: Learn about repo2docker
+
+../faq
+../specification
+```
diff --git a/docs/source/use/pathways.md b/docs/source/use/pathways.md
new file mode 100644
index 000000000..5375152df
--- /dev/null
+++ b/docs/source/use/pathways.md
@@ -0,0 +1,102 @@
+# Four ways to use `repo2docker` images in a JupyterHub
+
+Many users of `repo2docker` primarily wish to define the environment for a Binder or a JupyterHub. Here are four ways to accomplish this, from least-to-most work.
+
+% TO EDIT THIS FIGURE: Load the linked SVG into excalidraw.com, make edits, and re-export.
+
+```{figure} ../_static/images/whentouse.svg
+A decision tree for the recommended way to get your reproducible user environment using `repo2docker` or a community image built with `repo2docker`.
+```
+
+## 1. Use a community maintained image
+
+The simplest thing to do is check whether another community already maintains and offers an image that you can simply re-use. This reduces the burden on you to keep the image up-to-date, and gives you an opportunity to collaborate and make contributions rather than building something yourself from scratch.
+
+See [](./community-image.md) for a brief how-to on using a community maintained image.
+
+::::{grid} 2
+:::{grid-item-card} Benefits
+
+- A community of experts maintains the image with you!
+- Don’t struggle alone with your problems, drag others along!
+- Good choices (base image, how python is installed, etc) made on your behalf
+- Updates happen without you needing to do much
+ :::
+ :::{grid-item-card} Drawbacks
+- Might have lots of stuff you don’t need
+- Large image sizes, slower pulling
+- Might get you 98% of the way there, but that isn't 100%
+- Architectural choices made might not fit your use case
+- Updates are not on your schedule
+ :::
+ ::::
+
+## 2. Inherit from a community maintained image and add a few extras
+
+Sometimes a community-maintained environment image has **almost** what you need, but there are a few extra packages you'd like to install yourself. In this case, it's easiest to **inherit and extend the community image**.
+
+This is not the same thing as forking the repository for the community image, modifying it, and re-building it from scratch. It's similar to depending on an upstream piece of software and then building upon it in your own tool.
+
+::::{grid} 2
+:::{grid-item-card} Benefits
+
+- Only need to maintain the changes you make
+- Update the FROM tag to keep up with upstream changes
+- You must manage a GH repo
+- Works well with mybinder.org
+ :::
+ :::{grid-item-card} Drawbacks
+- Need to understand how upstream image is built so you can customize
+- Documentation for this method currently sucks (but can be fixed!)
+- Removing existing packages might break things
+- You must manage a GH repo
+ :::
+ ::::
+
+See [](./extend-community-image.md) for a how-to guide.
+
+## 3. Use repo2docker to build your environment image
+
+If your needs differ far enough from the community-maintained images that you find, you can use `repo2docker` to build your environment image using [supported configuration files](#config-files).
+To learn how to do this, follow the [getting started with repo2docker guide](../start.md) and look at the [list of supported configuration files](#config-files).
+Check out the other sections under "Image building basics" for more useful how-tos.
+
+::::{grid} 2
+:::{grid-item-card} Benefits
+
+- Works well with mybinder.org
+- No need to learn Dockerfile syntax
+- Use language specific, well understood file formats
+- Only get whatever packages you want
+- Good defaults chosen by the community
+ :::
+ :::{grid-item-card} Drawbacks
+- Build time is slower
+- Images may be bigger
+- Might not support 100% of what you want, and at some point you might have to take the ramp-off to a Dockerfile
+ :::
+ ::::
+
+## 4. Use a full fledged custom Dockerfile
+
+If you need full control over the entire computational environment, you can always create your own `Dockerfile` from scratch, and repo2docker will build an image out of it.
+
+This is for advanced users only that know what they're doing - full guidance on how to use `Dockerfiles` is out of scope for this tutorial.
+
+[Here's an example repository that uses a `Dockerfile` to build with repo2docker](https://github.com/yuvipanda/example-use-dockerfile).
+
+::::{grid} 2
+:::{grid-item-card} Benefits
+
+- Get exactly what you want
+- Can be optimized for small image size & fast build times
+- Lots of existing documentation in the SRE world on how to use these
+
+:::
+:::{grid-item-card} Drawbacks
+
+- Requires a lot of knowledge for ongoing maintenance
+- Might have to solve problems yourself that were solved in upstream images / repo2docker
+- Need to adapt existing documentation from the SRE use case to interactive computing use case
+ :::
+ ::::
diff --git a/docs/source/use/repository.md b/docs/source/use/repository.md
new file mode 100644
index 000000000..0c9d95f0c
--- /dev/null
+++ b/docs/source/use/repository.md
@@ -0,0 +1,63 @@
+# Using source repositories
+
+(usage-config-file-location)=
+
+## Where to put configuration files in a repository
+
+`repo2docker` will look for configuration files in the following order:
+
+- The root directory of the repository.
+- A folder named `binder/` or `.binder/` in the root of the repository.
+
+ If one of these folders exists, only configuration files in that folder are considered, configuration in the root directory will be ignored.
+ Having both `binder/` and `.binder/` folders is not allowed.
+
+Check the complete list of [configuration files](#config-files) supported
+by `repo2docker` to see how to configure the build process.
+
+(repository-providers)=
+
+## Supported repository providers
+
+`repo2docker` can fetch repositories from a number of repositories. Here are the various types of _repository references_ we support:
+
+- A URL of a Git repository (`https://github.com/binder-examples/requirements`),
+- A Zenodo DOI (`10.5281/zenodo.1211089`),
+- A [SWHID] (`swh:1:rev:999dd06c7f679a2714dfe5199bdca09522a29649`),
+- A URL of a [CKAN] dataset (`https://demo.ckan.org/dataset/sample-dataset-1`)
+- A path to a local directory (`a/local/directory`)
+
+[swhid]: https://docs.softwareheritage.org/devel/swh-model/persistent-identifiers.html
+[ckan]: https://ckan.org
+
+In each case you can build from these repository sources like so:
+
+```bash
+jupyter-repo2docker
+```
+
+## Supported version control systems
+
+These Version Control Systems are supported by `repo2docker`.
+
+### Git
+
+Any `git` repository is supported with `repo2docker`.
+These are generally stored in [a git repository provider](#repository-providers) like [GitHub](https://github.com).
+
+### Mercurial
+
+For [Mercurial](https://www.mercurial-scm.org) repositories, Mercurial and
+[hg-evolve](https://www.mercurial-scm.org/doc/evolution/) need to be
+installed. For example, on Debian based distributions, one can do:
+
+```
+sudo apt install mercurial
+$(hg debuginstall --template "{pythonexe}") -m pip install hg-evolve --user
+```
+
+To install Mercurial on other systems, see [here](https://www.mercurial-scm.org/download).
+
+Note that for old Mercurial versions, you may need to specify a version for
+hg-evolve. For example, `hg-evolve==9.2` for hg 4.5 (which is installed with
+`apt` on Ubuntu 18.4).