Begin re-organization of MOI docs (#1200)

* Begin re-organization of MOI docs

* Remove testing.md

* More re-organizing of documentation

* Various edits

Author: odow

docs/Project.toml

@@ -1,5 +1,6 @@
 [deps]
 Documenter = "e30172f5-a6a5-5a46-863b-614d45cd2de4"
+MathOptInterface = "b8f27783-ece8-5eb3-8dc8-9495eed66fee"
 
 [compat]
 Documenter = "0.25"

docs/make.jl

@@ -5,15 +5,24 @@ makedocs(
     format = Documenter.HTML(
         # See https://github.com/JuliaDocs/Documenter.jl/issues/868
         prettyurls = get(ENV, "CI", nothing) == "true",
-        mathengine = Documenter.MathJax()
+        mathengine = Documenter.MathJax2(),
+        collapselevel = 1,
     ),
     # See https://github.com/jump-dev/JuMP.jl/issues/1576
     strict = true,
     pages = [
         "Introduction" => "index.md",
-        "Manual" => "apimanual.md",
-        "Reference" => "apireference.md",
-        "Testing" => "testing.md",
+        "Manual" => [
+            "manual/basic_usage.md",
+            "manual/advanced_usage.md",
+            "manual/implementing.md",
+            "manual/Benchmarks.md",
+            "manual/Bridges.md",
+            "manual/FileFormats.md",
+            "manual/Test.md",
+            "manual/Utilities.md",
+        ],
+        "API Reference" => "apireference.md",
     ]
 )
 

docs/src/apimanual.md

@@ -1084,7 +1084,7 @@ Utilities.side_dimension_for_vectorized_dimension
 ## Benchmarks
 
 Functions to help benchmark the performance of solver wrappers. See
-[Benchmarking](@ref) for more details.
+[The Benchmarks submodule](@ref) for more details.
 
 ```@docs
 Benchmarks.suite
@@ -1095,7 +1095,7 @@ Benchmarks.compare_against_baseline
 ## File Formats
 
 Functions to help read and write MOI models to/from various file formats. See
-[File Formats](@ref) for more details.
+[The FileFormats submodule](@ref) for more details.
 
 ```@docs
 FileFormats.Model

docs/src/apireference.md

@@ -1,8 +1,84 @@
 # MathOptInterface
 
-[MathOptInterface.jl](https://github.com/jump-dev/MathOptInterface.jl) is a standardized API for mathematical optimization solvers.
+Each mathematical optimization solver API has its own concepts and data
+structures for representing optimization models and obtaining results.
+However, it is often desirable to represent an instance of an optimization
+problem at a higher level so that it is easy to try using different solvers.
 
-```@contents
-Pages = ["apimanual.md", "apireference.md", "testing.md"]
-Depth = 3
+[MathOptInterface.jl](https://github.com/jump-dev/MathOptInterface.jl) (MOI) is
+an abstraction layer designed to provide a unified interface to mathematical
+optimization solvers so that users do not need to understand multiple
+solver-specific APIs.
+
+MOI can be used directly, or through a higher-level modeling interface like
+[JuMP](https://github.com/jump-dev/JuMP.jl).
+
+## Background
+
+MOI has been designed to replace [MathProgBase](https://github.com/JuliaOpt/MathProgBase.jl),
+which was been used by modeling packages such as [JuMP](https://github.com/jump-dev/JuMP.jl)
+and [Convex.jl](https://github.com/jump-dev/Convex.jl).
+
+This second-generation abstraction layer addresses a number of limitations of
+MathProgBase. MOI is designed to:
+- Be simple and extensible, unifying linear, quadratic, and conic optimization,
+  and seamlessly facilitate extensions to essentially arbitrary constraints and
+  functions (e.g., indicator constraints, complementarity constraints, and
+  piecewise-linear functions)
+- Be fast by allowing access to a solver's in-memory representation of a problem
+  without writing intermediate files (when possible) and by using multiple
+  dispatch and avoiding requiring containers of nonconcrete types
+- Allow a solver to return multiple results (e.g., a pool of solutions)
+- Allow a solver to return extra arbitrary information via attributes (e.g.,
+  variable- and constraint-wise membership in an irreducible inconsistent subset
+  for infeasibility analysis)
+- Provide a greatly expanded set of status codes explaining what happened during
+  the optimization procedure
+- Enable a solver to more precisely specify which problem classes it supports
+- Enable both primal and dual warm starts
+- Enable adding and removing both variables and constraints by indices that are
+  not required to be consecutive
+- Enable any modification that the solver supports to an existing model
+- Avoid requiring the solver wrapper to store an additional copy of the problem
+  data
+
+## Sections of this documentation
+
+There are two main sections to this documentation.
+
+The manual introduces the concepts needed to understand MOI and gives a
+high-level picture of how all of the pieces fit together. The primary focus of
+[Basic usage](@ref) is on MOI from the perspective of a user of the interface.
+The section [Advanced usage](@ref) provides more detail on advanced topics
+such as [Duality](@ref). The manual also has a section on [Implementing a solver interface](@ref).
+
+In addition to the basic API, MathOptInterface.jl contains a number of
+submodules to help users interact with a model in MathOptInterface form. These
+include:
+ - [The `Benchmarks` submodule](@ref)
+ - [The `Bridges` submodule](@ref)
+ - [The `FileFormats` submodule](@ref)
+ - [The `Utilities` submodule](@ref)
+ - [The `Test` submodule](@ref)
+
+The [API Reference](@ref) page lists the complete API.
+
+## Further reading
+
+A [paper describing the design and features of MathOptInterface](https://arxiv.org/abs/2002.03447)
+is available on [arXiv](https://arxiv.org).
+
+If you find MathOptInterface useful in your work, we kindly request that you
+cite the following paper:
+```
+@misc{
+    legat2020mathoptinterface,
+    title = {MathOptInterface: a data structure for mathematical optimization problems},
+    author = {Beno{\^i}t Legat and Oscar Dowson and Joaquim Dias Garcia and Miles Lubin},
+    year = {2020},
+    eprint = {2002.03447},
+    archivePrefix = {arXiv},
+    primaryClass = {math.OC},
+    url = {https://arxiv.org/abs/2002.03447},
+}
 ```

docs/src/index.md

@@ -0,0 +1,52 @@
+```@meta
+CurrentModule = MathOptInterface
+DocTestSetup = quote
+    using MathOptInterface
+    const MOI = MathOptInterface
+end
+DocTestFilters = [r"MathOptInterface|MOI"]
+```
+
+# The `Benchmarks` submodule
+
+To aid the development of efficient solver wrappers, MathOptInterface provides
+benchmarking functionality. Benchmarking a wrapper follows a two-step process.
+
+First, prior to making changes, run and save the benchmark results on a given
+benchmark suite as follows:
+
+```julia
+using SolverPackage  # Replace with your choice of solver.
+
+using MathOptInterface
+const MOI = MathOptInterface
+
+suite = MOI.Benchmarks.suite() do
+    SolverPackage.Optimizer()
+end
+
+MOI.Benchmarks.create_baseline(
+    suite, "current"; directory = "/tmp", verbose = true
+)
+```
+Use the `exclude` argument to [`Benchmarks.suite`](@ref) to
+exclude benchmarks that the solver doesn't support.
+
+Second, after making changes to the package, re-run the benchmark suite and
+compare to the prior saved results:
+
+```julia
+using SolverPackage, MathOptInterface
+
+const MOI = MathOptInterface
+
+suite = MOI.Benchmarks.suite() do
+    SolverPackage.Optimizer()
+end
+
+MOI.Benchmarks.compare_against_baseline(
+    suite, "current"; directory = "/tmp", verbose = true
+)
+```
+
+This comparison will create a report detailing improvements and regressions.

docs/src/manual/Benchmarks.md

@@ -0,0 +1,81 @@
+```@meta
+CurrentModule = MathOptInterface
+DocTestSetup = quote
+    using MathOptInterface
+    const MOI = MathOptInterface
+end
+DocTestFilters = [r"MathOptInterface|MOI"]
+```
+
+# The `Bridges` submodule
+
+A constraint can often be written in a number of equivalent formulations. For
+example, the constraint ``l \le a^\top x \le u``
+([`ScalarAffineFunction`](@ref)-in-[`Interval`](@ref)) could be re-formulated as
+two constraints: ``a^\top x \ge l`` ([`ScalarAffineFunction`](@ref)-in-[`GreaterThan`](@ref))
+and ``a^\top x \le u`` (`ScalarAffineFunction`-in-`LessThan`). An alternative
+re-formulation is to add a dummy variable `y` with the constraints ``l \le y \le u``
+([`SingleVariable`](@ref)-in-[`Interval`](@ref)) and ``a^\top x - y = 0``
+([`ScalarAffineFunction`](@ref)-in-[`EqualTo`](@ref)).
+
+To avoid each solver having to code these transformations manually,
+MathOptInterface provides *bridges*. A bridge is a small transformation from one
+constraint type to another (potentially collection of) constraint type. Because
+these bridges are included in MathOptInterface, they can be re-used by any
+optimizer. Some bridges also implement constraint modifications and constraint
+primal and dual translations.
+
+For example, the `SplitIntervalBridge` defines the reformulation of a
+`ScalarAffineFunction`-in-`Interval` constraint into a
+`ScalarAffineFunction`-in-`GreaterThan` and a
+`ScalarAffineFunction`-in-`LessThan` constraint. `SplitInterval` is the
+bridge optimizer that applies the `SplitIntervalBridge` rewriting rule. Given
+an optimizer `optimizer` implementing `ScalarAffineFunction`-in-`GreaterThan`
+and `ScalarAffineFunction`-in-`LessThan`, the optimizer
+```jldoctest; setup=:(model = MOI.Utilities.Model{Float64}())
+optimizer = MOI.Bridges.Constraint.SplitInterval{Float64}(model)
+MOI.supports_constraint(
+    optimizer, MOI.ScalarAffineFunction{Float64}, MOI.Interval{Float64}
+)
+
+# output
+
+true
+```
+will additionally support [`ScalarAffineFunction`](@ref)-in-[`Interval`](@ref).
+Note that these [`Bridges.Constraint.SingleBridgeOptimizer`](@ref)s are mainly
+used for testing bridges.
+
+It is recommended to rather use [`Bridges.full_bridge_optimizer`](@ref) which
+automatically selects the appropriate constraint bridges for unsupported
+constraints.
+```julia
+optimizer = MOI.Bridges.full_bridge_optimizer(model, Float64)
+```
+
+### Variable reformulations
+
+A variable is often created constrained in a set unsupported by the solver while
+it could be parametrized by variables constrained in supported sets.
+For example, the [`Bridges.Variable.VectorizeBridge`](@ref) defines the
+reformulation of a constrained variable in [`GreaterThan`](@ref) into a
+constrained vector of one variable in [`Nonnegatives`](@ref).
+The `Bridges.Variable.Vectorize` is the bridge optimizer that applies the
+[`Bridges.Variable.VectorizeBridge`](@ref) rewriting rule. Given an optimizer
+`optimizer` implementing constrained variables in [`Nonnegatives`](@ref),
+the optimizer
+```jldoctest; setup=:(model = MOI.Utilities.Model{Float64}())
+optimizer = MOI.Bridges.Variable.Vectorize{Float64}(model)
+MOI.supports_add_constrained_variable(optimizer, MOI.GreaterThan{Float64})
+
+# output
+
+true
+```
+will additionally support constrained variables in [`GreaterThan`](@ref).
+Note that these [`Bridges.Variable.SingleBridgeOptimizer`](@ref) are mainly
+used for testing bridges.
+
+It is recommended to rather use [`Bridges.full_bridge_optimizer`](@ref), which
+automatically selects the appropriate bridges for unsupported constrained
+variables.

docs/src/manual/Bridges.md

@@ -0,0 +1,67 @@
+```@meta
+CurrentModule = MathOptInterface
+DocTestSetup = quote
+    using MathOptInterface
+    const MOI = MathOptInterface
+end
+DocTestFilters = [r"MathOptInterface|MOI"]
+```
+
+# The `FileFormats` submodule
+
+The `FileFormats` module provides functionality for reading and writing MOI
+models using [`write_to_file`](@ref) and [`read_from_file`](@ref).
+
+To write a model `src` to a MathOptFormat file, use:
+```julia
+src = # ...
+dest = FileFormats.Model(format = FileFormats.FORMAT_MOF)
+MOI.copy_to(dest, src)
+MOI.write_to_file(dest, "file.mof.json")
+```
+The list of supported formats is given by the [`FileFormats.FileFormat`](@ref)
+enum.
+
+Instead of the `format` keyword, you can also use the `filename` keyword
+argument to [`FileFormats.Model`](@ref). This will attempt to automatically
+guess the format from the file extension. For example:
+```julia
+src = # ...
+filename = "my_model.cbf.gz"
+dest = FileFormats.Model(filename = filename)
+MOI.copy_to(dest, src)
+MOI.write_to_file(dest, filename)
+
+src_2 = FileFormats.Model(filename = filename)
+MOI.read_from_file(src_2, filename)
+```
+Note how the compression format (GZip) is also automatically detected from the
+filename.
+
+In some cases `src` may contain constraints that are not supported by the file
+format (e.g., the CBF format supports integer variables but not binary). If so,
+you should copy `src` to a bridged model using [`Bridges.full_bridge_optimizer`](@ref):
+```julia
+src = # ... conic model ...
+dest = FileFormats.Model(format = FileFormats.FORMAT_CBF)
+bridged = MOI.Bridges.full_bridge_optimizer(dest, Float64)
+MOI.copy_to(bridged, src)
+MOI.write_to_file(dest, "my_model.cbf")
+```
+You should also note that even after bridging, it may still not be possible to
+write the model to file because of unsupported constraints (e.g., PSD variables
+in the LP file format).
+
+In addition to [`write_to_file`](@ref) and [`read_from_file`](@ref), you can
+read and write directly from `IO` streams using `Base.write` and `Base.read!`:
+```julia
+src = # ...
+io = IOBuffer()
+dest = FileFormats.Model(format = FileFormats.FORMAT_MPS)
+MOI.copy_to(dest, src)
+write(io, dest)
+
+seekstart(io)
+src_2 = FileFormats.Model(format = FileFormats.FORMAT_MPS)
+read!(io, src_2)
+```

docs/src/manual/FileFormats.md

@@ -1,4 +1,13 @@
-# Testing
+```@meta
+CurrentModule = MathOptInterface
+DocTestSetup = quote
+    using MathOptInterface
+    const MOI = MathOptInterface
+end
+DocTestFilters = [r"MathOptInterface|MOI"]
+```
+
+# The `Test` submodule
 
 All solvers use the tests in this repository as extra correctness tests for themselves.
 If we find a bug in one solver, instead of adding a test to that particular repository, we add it here so that all solvers can benefit.

docs/src/testing.md renamed to docs/src/manual/Test.md

@@ -0,0 +1,12 @@
+```@meta
+CurrentModule = MathOptInterface
+DocTestSetup = quote
+    using MathOptInterface
+    const MOI = MathOptInterface
+end
+DocTestFilters = [r"MathOptInterface|MOI"]
+```
+
+# The `Utilities` submodule
+
+TODO: document the utilities submodule