Skip to content

Commit

Permalink
Add Sphinx docs and enable ReadTheDocs build (#13)
Browse files Browse the repository at this point in the history
* Remove WaterTAP-specific how-to pages

* Move existing files

* Add core files for Sphinx docs

* Add minimal installation instructions

* Add ReadTheDocs config file

* Reformat with Black

* Add basic GHA check for docs build

* Try resolving failures for Python 3.8

* Try fixing Codecov upload error

* Try fixing workflow syntax error

* Try with space

* Try downgrading Codecov CLI due to codecov/codecov-action#1487
  • Loading branch information
lbianchi-lbl committed Jun 21, 2024
1 parent 4bf6514 commit cf93acc
Show file tree
Hide file tree
Showing 15 changed files with 279 additions and 612 deletions.
21 changes: 16 additions & 5 deletions .github/workflows/checks.yml
Original file line number Diff line number Diff line change
Expand Up @@ -132,13 +132,24 @@ jobs:
- uses: actions/download-artifact@v4
with:
pattern: coverage-report-*
- name: Upload coverage report to Codecov
- name: Upload coverage report to Codecov (token)
uses: codecov/codecov-action@v4
with:
fail_ci_if_error: true
verbose: true
# NOTE: secrets are not available for pull_request workflows
# However, as of 2024-02-10, Codecov is still allowing tokenless upload from PRs
# but does require token for other workflows e.g. merge to `main`
# see https://github.com/codecov/codecov-action/issues/1274#issuecomment-1934437359
token: ${{ secrets.CODECOV_TOKEN }}
# downgrading after v0.7.0 broke tokenless upload
# see codecov/codecov-action#1487
version: v0.6.0

build-docs:
name: Build docs (Sphinx)
needs: [code-formatting]
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: '3.10'
- run: pip install -r requirements-dev.txt
- run: make -C docs html
19 changes: 19 additions & 0 deletions .readthedocs.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
# .readthedocs.yaml
# Read the Docs configuration file
# See https://docs.readthedocs.io/en/stable/config-file/v2.html for details

# Required
version: 2

# Build documentation in the docs/ directory with Sphinx
sphinx:
fail_on_warning: true

build:
os: ubuntu-22.04
tools:
python: "3.10"

python:
install:
- requirements: requirements-dev.txt
20 changes: 20 additions & 0 deletions docs/Makefile
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
# Minimal makefile for Sphinx documentation
#

# You can set these variables from the command line, and also
# from the environment for the first two.
SPHINXOPTS ?=
SPHINXBUILD ?= sphinx-build
SOURCEDIR = .
BUILDDIR = _build

# Put it first so that "make" without argument is like "make help".
help:
@$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)

.PHONY: help Makefile

# Catch-all target: route all unknown targets to Sphinx using the new
# "make mode" option. $(O) is meant as a shortcut for $(SPHINXOPTS).
%: Makefile
@$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
68 changes: 68 additions & 0 deletions docs/conf.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
from importlib import metadata

# Configuration file for the Sphinx documentation builder.
#
# For the full list of built-in configuration values, see the documentation:
# https://www.sphinx-doc.org/en/master/usage/configuration.html

# -- Project information -----------------------------------------------------
# https://www.sphinx-doc.org/en/master/usage/configuration.html#project-information

project = "Parameter Sweep"
copyright = "2024, WaterTAP"
author = "WaterTAP"

release = version = metadata.version("parameter-sweep")

# -- General configuration ---------------------------------------------------
# https://www.sphinx-doc.org/en/master/usage/configuration.html#general-configuration

extensions = [
"sphinx_rtd_theme",
"sphinx.ext.doctest",
"sphinx.ext.napoleon",
"autoapi.extension",
"sphinx.ext.intersphinx",
]
suppress_warnings = []

templates_path = ["_templates"]
exclude_patterns = ["_build", "Thumbs.db", ".DS_Store"]

# see https://sphinx-autoapi.readthedocs.io/en/latest/reference/config.html
autoapi_dirs = ["../src"]
autoapi_root = "technical_reference"
autoapi_options = [
"members",
"undoc-members",
"show-module-summary",
]
# set to True to save the autogenerated
autoapi_keep_files = False
autoapi_ignore = [
"*conftest.py",
"*test_*.py",
"*tests/*.py",
]
# the warnings come from parameter_sweep._compat
suppress_warnings += ["autoapi.python_import_resolution"]

intersphinx_mapping = {
"pyomo": ("https://pyomo.readthedocs.org/en/stable", None),
"numpy": ("https://numpy.org/doc/1.26", None),
}


def _skip_tests(app, what: str, name: str, obj, skip, options):
return skip


def setup(sphinx):
sphinx.connect("autoapi-skip-member", _skip_tests)


# -- Options for HTML output -------------------------------------------------
# https://www.sphinx-doc.org/en/master/usage/configuration.html#options-for-html-output

html_theme = "sphinx_rtd_theme"
html_static_path = ["_static"]
9 changes: 9 additions & 0 deletions docs/getting_started/index.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
Getting started
===============

.. toctree::
:maxdepth: 1

installation
parameter_sweep
parallel_manager
6 changes: 6 additions & 0 deletions docs/getting_started/installation.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
Installation
------------

.. code-block:: bash
pip install parameter-sweep
Original file line number Diff line number Diff line change
@@ -1,10 +1,5 @@
Parallel Manager
====================

.. index::
pair: parameter_sweep.parallel.parallel_manager;parallel_manager

.. currentmodule:: parameter_sweep.parallel.parallel_manager
================

Overview
--------
Expand Down Expand Up @@ -66,11 +61,11 @@ following statement.
The above code return 1 of 5 parallel manager classes. There are currently 5 parallel manager classes:

#. ``SingleProcessParallelManager`` : To enable serial implementation
#. ``MPIParallelManager`` : For MPI
#. ``ConcurrentFuturesParallelManager`` : For python concurrent futures
#. ``MultiprocessingParallelManager`` : For more advanced python multiprocessing
#. ``RayIoParallelManager`` : For using Ray
#. :class:`SingleProcessParallelManager` : To enable serial implementation
#. :class:`MPIParallelManager` : For MPI
#. :class:`ConcurrentFuturesParallelManager` : For python concurrent futures
#. :class:`MultiprocessingParallelManager` : For more advanced python multiprocessing
#. :class:`RayIoParallelManager` : For using Ray

Alternatively, the user can specify the parameter sweep class directly, for example

Expand Down
76 changes: 76 additions & 0 deletions docs/getting_started/parameter_sweep.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,76 @@

Parameter Sweep
===============

.. currentmodule:: parameter_sweep.parameter_sweep

Overview
--------

The parameter sweep tool systematically fixes variables or modifies
mutable parameters on a flowsheet (or Pyomo :class:`~pyomo.environ.ConcreteModel`), optimizes
the flowsheet, and reports user-specified results.
The parameter sweep tool can be operated in one of two ways: sweeping a
fixed set of parameters, or allowing for random samples from a distribution.
While different fixed or random sampling types can be combined, in a single
parameter sweep the user must use either all fixed or all random sampling types.
(This implementation detail may be relaxed in future releases.)

For the "fixed" sampling type, :class:`LinearSample`, the parameter sweep tool will evaluate the
cross product of all the specified parameters, whereas with the "random" sampling
types :class:`UniformSample` and :class:`NormalSample` the parameter sweep tool will evaluate
a fixed number of samples specified in :const:`num_samples`. With either sampling type,
the values of the Pyomo objects (:class:`~pyomo.environ.Var` s, :class:`~pyomo.environ.NamedExpression` s, etc.), that the user specifies
in `outputs`.

For each item the user wants to change, they specify a `sweep_params` dictionary.
The keys are "short" names, and the values are one of the included :class:`Sample` objects.
In all cases the :class:`Sample` objects are instantiated with the Pyomo object to be changed,
with additional arguements depending on the sampling type. For example, for the fixed
:class:`LinearSample` the user would also specify a lower limit, an upper limit, and the number
of elements to be sampled for this parameter between the lower limit and upper limit.
Each item should be sampled at least twice to capture the upper and lower
limit. The random :class:`UniformSample` requires a lower limit and upper limit, and the
:class:`NormalSample` requires a mean and standard deviation.

In addition to the parameters to sweep and the values to track for output,
the user must provide an :const:`optimize_function`, which takes the :const:`model` as an
attribute calls an optimization routine to solve it for the updated parameters.
Should the call to :const:`optimize_function` fail, and a :const:`reinitialize_function` is not
specified, the outputs will be reported as ``NaN`` for that parameter set.

The user can optionally specify a :const:`reinitialize_function` in case any piece
of the :const:`optimize_function` fails -- after the call to :const:`reinitialize_function`
the model should be in a state ready to optimize again. If the :const:`reinitialize_function`
or the second call to the :const:`optimize_function` fail for any reason, the outputs will
be reported as ``NaN`` for that parameter set.

The parameter sweep tool maintains the state of the flowsheet / Pyomo model between
calls to :const:`optimize_function` to take advantage of initializations provided by
earlier solutions. If this behavior is undesirable, the user should re-initialize
their flowsheet as part of their :const:`optimize_function`.

Finally, the user can specify a :const:`csv_results_file_name` and/or an :const:`h5_results_file_name`,
which will write the outputs to disk in a CSV and/or H5 format, respectively.
In the CSV results
file, each column specifies a fixed parameter or the associated output, and each row
is a single run with the specified parameters and resulting outputs. The H5 file
contains the parameter sweep inputs and the outputs stored in a dictionary-like format.
Additionally, when an H5 file is written, a companion text file is created with the name
:const:`h5_results_file_name` + ``".txt"``. This text file contains the metadata of the H5 results
file.

Parallel Usage
--------------

The parameter sweep tool can optionally utilize ``mpi4py`` to split the parameter
sweep over multiple processors. If ``mpi4py`` is installed in the user's conda environment,
a script utilizing the parameter sweep tool can be run in parallel, in this example
using two threads.

.. code:: bash
mpiexec -n 2 python parameter_sweep_script.py
For advanced users, the parameter sweep tool can optionally take a MPI communicator
as an argument.
Loading

0 comments on commit cf93acc

Please sign in to comment.