OSDOCS-1927: Add file-based catalog docs

_topic_map.yml

@@ -656,8 +656,13 @@ Topics:
   - Name: Basic tkn commands
     File: op-tkn-reference
 - Name: opm CLI
-  File: opm-cli
+  Dir: opm
   Distros: openshift-enterprise,openshift-origin
+  Topics:
+  - Name: Installing the opm CLI
+    File: cli-opm-install
+  - Name: opm CLI reference
+    File: cli-opm-ref
 - Name: Operator SDK
   Dir: osdk
   Distros: openshift-enterprise,openshift-origin
@@ -1254,7 +1259,7 @@ Topics:
   Topics:
   - Name: What are Operators?
     File: olm-what-operators-are
-  - Name: Packaging formats
+  - Name: Packaging format
     File: olm-packaging-format
   - Name: Common terms
     File: olm-common-terms

cli_reference/opm/cli-opm-install.adoc

@@ -0,0 +1,20 @@
+[id="cli-opm-install"]
+= Installing the opm CLI
+include::modules/common-attributes.adoc[]
+:context: cli-opm-install
+
+toc::[]
+
+include::modules/olm-about-opm.adoc[leveloffset=+1]
+.Additional resources
+
+* See xref:../../operators/understanding/olm-packaging-format.adoc#olm-bundle-format_olm-packaging-format[Operator Framework packaging format] for more information about the bundle format.
+* To create a bundle image using the Operator SDK, see
+xref:../../operators/operator_sdk/osdk-working-bundle-images.adoc#osdk-working-bundle-images[Working with bundle images].
+
+include::modules/olm-installing-opm.adoc[leveloffset=+1]
+
+[id="opm-addtl-resources"]
+== Additional resources
+
+* See xref:../../operators/admin/olm-managing-custom-catalogs.adoc#olm-managing-custom-catalogs[Managing custom catalogs] for `opm` procedures including creating, updating, and pruning catalogs.

cli_reference/opm/cli-opm-ref.adoc

@@ -0,0 +1,33 @@
+[id="cli-opm-ref"]
+= opm CLI reference
+include::modules/common-attributes.adoc[]
+:context: cli-opm-ref
+
+toc::[]
+
+The `opm` command-line interface (CLI) is a tool for creating and maintaining Operator catalogs.
+
+.`opm` CLI syntax
+[source,terminal]
+----
+$ opm <command> [<subcommand>] [<argument>] [<flags>]
+----
+
+.Global flags
+[options="header",cols="1,3"]
+|===
+|Flag |Description
+
+|`--skip-tls`
+|Skip TLS certificate verification for container image registries while pulling bundles or indexes.
+
+|===
+
+:FeatureName: The SQLite-based catalog format, including the related CLI commands,
+include::modules/deprecated-feature.adoc[]
+
+include::modules/opm-cli-ref-index.adoc[leveloffset=+1]
+include::modules/opm-cli-ref-init.adoc[leveloffset=+1]
+include::modules/opm-cli-ref-render.adoc[leveloffset=+1]
+include::modules/opm-cli-ref-validate.adoc[leveloffset=+1]
+include::modules/opm-cli-ref-serve.adoc[leveloffset=+1]

cli_reference/osdk/cli-osdk-install.adoc

@@ -7,6 +7,8 @@ toc::[]
 
 The Operator SDK provides a command-line interface (CLI) tool that Operator developers can use to build, test, and deploy an Operator. You can install the Operator SDK CLI on your workstation so that you are prepared to start authoring your own Operators.
 
+Operator authors with cluster administrator access to a Kubernetes-based cluster, such as {product-title}, can use the Operator SDK CLI to develop their own Operators based on Go, Ansible, or Helm. link:https://kubebuilder.io/[Kubebuilder] is embedded into the Operator SDK as the scaffolding solution for Go-based Operators, which means existing Kubebuilder projects can be used as is with the Operator SDK and continue to work.
+
 See xref:../../operators/operator_sdk/osdk-about.adoc#osdk-about[Developing Operators] for full documentation on the Operator SDK.
 
 [NOTE]

cli_reference/osdk/cli-osdk-ref.adoc

@@ -13,8 +13,6 @@ The Operator SDK command-line interface (CLI) is a development kit designed to m
 $ operator-sdk <command> [<subcommand>] [<argument>] [<flags>]
 ----
 
-Operator authors with cluster administrator access to a Kubernetes-based cluster (such as {product-title}) can use the Operator SDK CLI to develop their own Operators based on Go, Ansible, or Helm. link:https://kubebuilder.io/[Kubebuilder] is embedded into the Operator SDK as the scaffolding solution for Go-based Operators, which means existing Kubebuilder projects can be used as is with the Operator SDK and continue to work.
-
 See xref:../../operators/operator_sdk/osdk-about.adoc#osdk-about[Developing Operators] for full documentation on the Operator SDK.
 
 include::modules/osdk-cli-ref-bundle.adoc[leveloffset=+1]

modules/olm-about-opm.adoc

@@ -1,11 +1,11 @@
 // Module included in the following assemblies:
 //
-// * operators/understanding/olm/olm-packaging-formats.adoc
-// * cli_reference/opm-cli.adoc
+// * operators/understanding/olm-packaging-format.adoc
+// * cli_reference/opm/cli-opm-install.adoc
 
 [id="olm-about-opm_{context}"]
-= About opm
+= About the opm CLI
 
-The `opm` CLI tool is provided by the Operator Framework for use with the Operator bundle format. This tool allows you to create and maintain catalogs of Operators from a list of bundles, called an _index_, that are similar to software repositories. The result is a container image, called an _index image_, which can be stored in a container registry and then installed on a cluster.
+The `opm` CLI tool is provided by the Operator Framework for use with the Operator bundle format. This tool allows you to create and maintain catalogs of Operators from a list of Operator bundles that are similar to software repositories. The result is a container image which can be stored in a container registry and then installed on a cluster.
 
-An index contains a database of pointers to Operator manifest content that can be queried through an included API that is served when the container image is run. On {product-title}, Operator Lifecycle Manager (OLM) can use the index image as a catalog by referencing it in a `CatalogSource` object, which polls the image at regular intervals to enable frequent updates to installed Operators on the cluster.
+A catalog contains a database of pointers to Operator manifest content that can be queried through an included API that is served when the container image is run. On {product-title}, Operator Lifecycle Manager (OLM) can reference the image in a catalog source, defined by a `CatalogSource` object, which polls the image at regular intervals to enable frequent updates to installed Operators on the cluster.

modules/olm-bundle-format-dependencies-file.adoc

@@ -1,6 +1,6 @@
 // Module included in the following assemblies:
 //
-// * operators/understanding/olm/olm-packaging-format.adoc
+// * operators/understanding/olm-packaging-format.adoc
 // * operators/understanding/olm/olm-understanding-dependency-resolution.adoc
 
 [id="olm-bundle-format-dependencies_{context}"]

modules/olm-bundle-format.adoc

@@ -1,6 +1,6 @@
 // Module included in the following assemblies:
 //
-// * operators/understanding/olm/olm-packaging-format.adoc
+// * operators/understanding/olm-packaging-format.adoc
 
 [id="olm-bundle-format_{context}"]
 = Bundle format

modules/olm-creating-fb-catalog-image.adoc

@@ -0,0 +1,164 @@
+// Module included in the following assemblies:
+//
+// * operators/admin/olm-managing-custom-catalogs.adoc
+
+ifdef::openshift-origin[]
+:registry-image: quay.io/openshift/origin-operator-registry:4.9.0
+endif::[]
+ifndef::openshift-origin[]
+:registry-image: registry.redhat.io/openshift4/ose-operator-registry:v4.9
+endif::[]
+
+[id="olm-creating-fb-catalog-image_{context}"]
+= Creating a file-based catalog image
+
+You can create a catalog image that uses the plain text _file-based catalog_ format (JSON or YAML), which replaces the deprecated SQLite database format. The `opm` CLI provides tooling that helps initialize a catalog in the file-based format, render new records into it, and validate that the catalog is valid.
+
+.Prerequisites
+
+* `opm` version 1.18.0+
+* `podman` version 1.9.3+
+* A bundle image built and pushed to a registry that supports link:https://docs.docker.com/registry/spec/manifest-v2-2/[Docker v2-2]
+
+.Procedure
+
+. Initialize a catalog for a file-based catalog:
+
+.. Create a directory for the catalog:
++
+[source,terminal]
+----
+$ mkdir <operator_name>-index
+----
+
+.. Create a Dockerfile that can build a catalog image:
++
+.Example `<operator_name>-index.Dockerfile`
+[source,bash,subs="attributes+"]
+----
+# The base image is expected to contain
+# /bin/opm (with a serve subcommand) and /bin/grpc_health_probe
+FROM {registry-image}
+
+# Configure the entrypoint and command
+ENTRYPOINT ["/bin/opm"]
+CMD ["serve", "/configs"]
+
+# Copy declarative config root into image at /configs
+ADD <operator_name>-index /configs
+
+# Set DC-specific label for the location of the DC root directory
+# in the image
+LABEL operators.operatorframework.io.index.configs.v1=/configs
+----
++
+The Dockerfile must be in the same parent directory as the catalog directory that you created in the previous step:
++
+.Example directory structure
+[source,terminal]
+----
+.
+├── <operator_name>-index
+└── <operator_name>-index.Dockerfile
+----
+
+.. Populate the catalog with your package definition:
++
+[source,terminal]
+----
+$ opm init <operator_name> \ <.>
+    --default-channel=preview \ <.>
+    --description=./README.md \ <.>
+    --icon=./operator-icon.svg \ <.>
+    --output yaml \ <.>
+    > <operator_name>-index/index.yaml <.>
+----
+<.> Operator, or package, name.
+<.> Channel that subscription will default to if unspecified.
+<.> Path to the Operator's `README.md` or other documentation.
+<.> Path to the Operator's icon.
+<.> Output format: JSON or YAML.
+<.> Path for creating the catalog configuration file.
++
+This command generates an `olm.package` declarative config blob in the specified catalog configuration file.
+
+. Add a bundle to the catalog:
++
+[source,terminal]
+----
+$ opm render <registry>/<namespace>/<bundle_image_name>:<tag> \ <.>
+    --output=yaml \
+    >> <operator_name>-index/index.yaml <.>
+----
+<.> Pull spec for the bundle image.
+<.> Path to the catalog configuration file.
++
+The `opm render` command generates a declarative config blob from the provided catalog images and bundle images.
++
+[NOTE]
+====
+Channels must contain at least one bundle.
+====
+
+. Add a channel entry for the bundle. For example, modify the following example to your specifications, and add it to your `<operator_name>-index/index.yaml` file:
++
+.Example channel entry
+[source,yaml]
+----
+---
+schema: olm.channel
+package: <operator_name>
+name: preview
+entries:
+  - name: <operator_name>.v0.1.0 <.>
+----
+<.> Ensure that you include the period (`.`) after `<operator_name>` but before the `v` in the version. Otherwise, the entry will fail to pass the `opm validate` command.
+
+. Validate the file-based catalog:
+
+.. Run the `opm validate` command against the catalog directory:
++
+[source,terminal]
+----
+$ opm validate <operator_name>-index
+----
+
+.. Check that the error code is `0`:
++
+[source,terminal]
+----
+$ echo $?
+----
++
+.Example output
+[source,terminal]
+----
+0
+----
+
+. Build the catalog image:
++
+[source,terminal]
+----
+$ podman build . \
+    -f <operator_name>-index.Dockerfile \
+    -t <registry>/<namespace>/<catalog_image_name>:<tag>
+----
+
+. Push the catalog image to a registry:
+
+.. If required, authenticate with your target registry:
++
+[source,terminal]
+----
+$ podman login <registry>
+----
+
+.. Push the catalog image:
++
+[source,terminal]
+----
+$ podman push <registry>/<namespace>/<catalog_image_name>:<tag>
+----
+
+:!registry-image:

modules/olm-creating-index-image.adoc

@@ -3,13 +3,13 @@
 // * operators/admin/olm-managing-custom-catalogs.adoc
 
 [id="olm-creating-index-image_{context}"]
-= Creating an index image
+= Creating a SQLite-based index image
 
-You can create an index image using the `opm` CLI.
+You can create an index image based on the SQLite database format by using the `opm` CLI.
 
 .Prerequisites
 
-* `opm` version 1.12.3+
+* `opm` version 1.18.0+
 * `podman` version 1.9.3+
 * A bundle image built and pushed to a registry that supports link:https://docs.docker.com/registry/spec/manifest-v2-2/[Docker v2-2]
 

modules/olm-fb-catalogs-automation.adoc

@@ -0,0 +1,14 @@
+// Module included in the following assemblies:
+//
+// * operators/understanding/olm-packaging-format.adoc
+
+[id="olm-fb-catalogs-automation_{context}"]
+= Automation
+
+Operator authors and catalog maintainers are encouraged to automate their catalog maintenance with CI/CD workflows. Catalog maintainers can further improve on this by building GitOps automation to accomplish the following tasks:
+
+* Check that pull request (PR) authors are permitted to make the requested changes, for example by updating their package's image reference.
+* Check that the catalog updates pass the `opm validate` command.
+* Check that the updated bundle or catalog image references exist, the catalog images run successfully in a cluster, and Operators from that package can be successfully installed.
+* Automatically merge PRs that pass the previous checks.
+* Automatically rebuild and republish the catalog image.

modules/olm-fb-catalogs-example.adoc

@@ -0,0 +1,44 @@
+// Module included in the following assemblies:
+//
+// * operators/understanding/olm-packaging-format.adoc
+
+[id="olm-fb-catalogs-example_{context}"]
+= Example catalog
+
+With file-based catalogs, catalog maintainers can focus on Operator curation and compatibility. Because Operator authors have already produced Operator-specific catalogs for their Operators, catalog maintainers can build their catalog by rendering each Operator catalog into a subdirectory of the catalog's root directory.
+
+There are many possible ways to build a file-based catalog; the following steps outline a simple approach:
+
+. Maintain a single configuration file for the catalog, containing image references for each Operator in the catalog:
++
+.Example catalog configuration file
+[source,yaml]
+----
+name: community-operators
+repo: quay.io/community-operators/catalog
+tag: latest
+references:
+- name: etcd-operator
+  image: quay.io/etcd-operator/index@sha256:5891b5b522d5df086d0ff0b110fbd9d21bb4fc7163af34d08286a2e846f6be03
+- name: prometheus-operator
+  image: quay.io/prometheus-operator/index@sha256:e258d248fda94c63753607f7c4494ee0fcbe92f1a76bfdac795c9d84101eb317
+----
+
+. Run a script that parses the configuration file and creates a new catalog from its references:
++
+.Example script
+[source,sh]
+----
+name=$(yq eval '.name' catalog.yaml)
+mkdir "$name"
+yq eval '.name + "/" + .references[].name' catalog.yaml | xargs mkdir
+for l in $(yq e '.name as $catalog | .references[] | .image + "|" + $catalog + "/" + .name + "/index.yaml"' catalog.yaml); do
+  image=$(echo $l | cut -d'|' -f1)
+  file=$(echo $l | cut -d'|' -f2)
+  opm render "$image" > "$file"
+done
+opm alpha generate dockerfile "$name"
+indexImage=$(yq eval '.repo + ":" + .tag' catalog.yaml)
+docker build -t "$indexImage" -f "$name.Dockerfile" .
+docker push "$indexImage"
+----