nasa · jennyhliu · Jun 23, 2023 · Mar 13, 2023 · Mar 14, 2023 · Mar 14, 2023
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -6,6 +6,21 @@ The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/).
 
 ## Unreleased
 
+### Added
+
+- **CUMULUS-3201**
+  - Added support for sha512 as checksumType for LZARDs backup task.
+
+- **CUMULUS-3220**
+  - Added a send-pan-task that uploads a PAN response file to a specified path for a provider using HTTP/HTTPS protocol.
+
+### Changed
+
+- **CUMULUS-3165**
+  - Update example/cumulus-tf/orca.tf to use orca v6.0.3
+
+## [v15.0.0] 2023-03-10
+
 ### Breaking Changes
 
 - **CUMULUS-3147**
@@ -15,17 +30,22 @@ The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/).
   - Published new tag [`44` of `cumuluss/async-operation` to Docker Hub](https://hub.docker.com/layers/cumuluss/async-operation/44/images/sha256-8d757276714153e4ab8c24a2b7b6b9ffee14cc78b482d9924e7093af88362b04?context=explore).
   - The `async_operation_image` property of `cumulus` module must be updated to pull the ECR image for `cumuluss/async-operation:44`.
 
-## Changed
+### Changed
 
 - **CUMULUS-2997**
   - Migrate Cumulus Docs to Docusaurus v2 and DocSearch v3.
+- **CUMULUS-3044**
+  - Deployment section:
+    - Consolidate and migrate Cumulus deployment (public facing) content from wiki to Cumulus Docs in GitHub.
+    - Update links to make sure that the user can maintain flow between the wiki and GitHub deployment documentation.
+    - Organize and update sidebar to include categories for similar deployment topics.
 - **CUMULUS-3147**
   - Set example/cumulus-tf default async_operation_image_version to 44.
   - Set example/cumulus-tf default ecs_task_image_version to 1.9.0.
 - **CUMULUS-3166**
   - Updated example/cumulus-tf/thin_egress_app.tf to use tea 1.3.2
 
-## Fixed 
+### Fixed
 
 - **CUMULUS-3187**
   - Restructured Earthdata Login class to be individual methods as opposed to a Class Object
@@ -64,8 +84,8 @@ update the database cluster to use the new configuration.
 - **CUMULUS-3193**
   - Add a Python version file
 - **CUMULUS-3121**
-  - Added a map of variables in terraform for custom configuration of cloudwatch_log_groups' retention periods. 
-    Please refer to the [Cloudwatch-Retention] (https://nasa.github.io/cumulus/docs/configuration/cloudwatch-retention) 
+  - Added a map of variables in terraform for custom configuration of cloudwatch_log_groups' retention periods.
+    Please refer to the [Cloudwatch-Retention] (https://nasa.github.io/cumulus/docs/configuration/cloudwatch-retention)
     section of the Cumulus documentation in order for more detailed information and an example into how to do this.
 - **CUMULUS-3071**
   - Added 'PATCH' granules endpoint as an exact duplicate of the existing `PUT`
@@ -6879,7 +6899,8 @@ Note: There was an issue publishing 1.12.0. Upgrade to 1.12.1.
 
 ## [v1.0.0] - 2018-02-23
 
-[unreleased]: https://github.com/nasa/cumulus/compare/v14.1.0...HEAD
+[unreleased]: https://github.com/nasa/cumulus/compare/v15.0.0...HEAD
+[v15.0.0]: https://github.com/nasa/cumulus/compare/v14.1.0...v15.0.0
 [v14.1.0]: https://github.com/nasa/cumulus/compare/v14.0.0...v14.1.0
 [v14.0.0]: https://github.com/nasa/cumulus/compare/v13.4.0...v14.0.0
 [v13.4.0]: https://github.com/nasa/cumulus/compare/v13.3.2...v13.4.0

diff --git a/CONTRIBUTORS.md b/CONTRIBUTORS.md
@@ -30,6 +30,7 @@
 * Mark Schwab
 * Matt Savoie
 * Menno Van Diermen
+* Naga Nages
 * Nagavenkata Nageswaran
 * Nate Pauzenga
 * Pat Cappelaere
@@ -40,3 +41,5 @@
 * Slav Korolev
 * Snyk bot
 * Tobias Nießen
+* Vanh Khuyen Nguyen
+* Vanh Nguyen
diff --git a/bamboo/bootstrap-tf-deployment.sh b/bamboo/bootstrap-tf-deployment.sh
@@ -98,3 +98,5 @@ echo "Deploying Cumulus example to $DEPLOYMENT"
   -var "pdr_node_name_provider_bucket=$PDR_NODE_NAME_PROVIDER_BUCKET" \
   -var "rds_admin_access_secret_arn=$RDS_ADMIN_ACCESS_SECRET_ARN" \
   -var "orca_db_user_password=$ORCA_DATABASE_USER_PASSWORD" \
+  -var "orca_s3_access_key=$AWS_ACCESS_KEY_ID" \
+  -var "orca_s3_secret_key=$AWS_SECRET_ACCESS_KEY" \
diff --git a/docs/assets/pgadmin_create_server.png b/docs/assets/pgadmin_create_server.png
diff --git a/docs/assets/pgadmin_query_tool.png b/docs/assets/pgadmin_query_tool.png
diff --git a/docs/assets/pgadmin_retrieve_btn.png b/docs/assets/pgadmin_retrieve_btn.png
diff --git a/docs/assets/pgadmin_retrieve_values.png b/docs/assets/pgadmin_retrieve_values.png
diff --git a/docs/assets/pgadmin_server_connection.png b/docs/assets/pgadmin_server_connection.png
diff --git a/docs/assets/pgadmin_ssh_config.png b/docs/assets/pgadmin_ssh_config.png
diff --git a/docs/deployment/README.md b/docs/deployment/README.md
@@ -25,6 +25,12 @@ The process involves:
 - Creating resources for your Terraform backend
 - Using [Terraform](https://www.terraform.io) to deploy resources to AWS
 
+:::info
+
+Please note that internal and sensitive information is not in this public resource and you may have to visit our [Cumulus wiki](https://wiki.earthdata.nasa.gov/display/CUMULUS/Deployment) for NGAP access steps and other credentials.
+
+:::
+
 ---
 
 ## Requirements
@@ -74,7 +80,7 @@ Terraform v0.13.6
 ### Credentials
 
 - [CMR](https://earthdata.nasa.gov/about/science-system-description/eosdis-components/common-metadata-repository) username and password. CMR credentials must be provided if you are exporting metadata to CMR with Earthdata Login authentication.
-- [NASA Launchpad](https://launchpad.nasa.gov). Launchpad credentials must be provided if you are using Launchpad authentication to export metadata to CMR or to authenticate with the Cumulus API.
+- [NASA Launchpad](https://launchpad.nasa.gov). Launchpad credentials must be provided if you are using Launchpad authentication to export metadata to CMR or to authenticate with the Cumulus API. For more information on how to authenticate go to [Launchpad Authentication](https://wiki.earthdata.nasa.gov/display/CUMULUS/Launchpad+Authentication).
 - [Earthdata Login](https://earthdata.nasa.gov/about/science-system-description/eosdis-components/earthdata-login) username and password. User must have the ability to administer and/or create applications in URS. It's recommended to obtain an account in the test environment (UAT).
 
 ### Needed Git Repositories
@@ -86,7 +92,7 @@ Terraform v0.13.6
 
 ## Prepare Deployment Repository
 
- > _If you already are working with an existing repository that is configured appropriately for the version of Cumulus you intend to deploy or update, skip to [Prepare AWS configuration.](#prepare-aws-configuration)_
+ > _If you already are working with an existing repository that is configured appropriately for the version of Cumulus you intend to deploy or update, skip to [Prepare AWS Configuration.](#prepare-aws-configuration)_
 
 Clone the [`cumulus-template-deploy`](https://github.com/nasa/cumulus-template-deploy) repo and name appropriately for your organization:
 
@@ -108,7 +114,11 @@ We will return to [configuring this repo and using it for deployment below](#dep
 
 You can then [add/commit](https://help.github.com/articles/adding-a-file-to-a-repository-using-the-command-line/) changes as needed.
 
-> ⚠️ **Note**: If you are pushing your deployment code to a git repo, make sure to add `terraform.tf` and `terraform.tfvars` to `.gitignore`, **as these files will contain sensitive data related to your AWS account**.
+:::caution Update Your Gitignore File
+
+If you are pushing your deployment code to a git repo, make sure to add `terraform.tf` and `terraform.tfvars` to `.gitignore`, **as these files will contain sensitive data related to your AWS account**.
+
+:::
 
 </details>
 
@@ -140,7 +150,11 @@ You can create additional S3 buckets based on the needs of your workflows.
 
 These buckets do not need any non-default permissions to function with Cumulus; however, your local security requirements may vary.
 
-> ⚠️ **Note**: S3 bucket object names are global and must be unique across all accounts/locations/etc.
+:::caution naming S3 buckets
+
+S3 bucket object names are global and must be unique across all accounts/locations/etc.
+
+:::
 
 ### VPC, Subnets, and Security Group
 
@@ -165,9 +179,13 @@ This operation only needs to be done once per account, but it must be done for b
 
 ### Look Up ECS-optimized AMI (DEPRECATED)
 
-> ⚠️ **Note:** This step is unnecessary if you using the latest changes in the [`cumulus-template-deploy` repo which will automatically determine the AMI ID for you
+:::info
+
+This step is unnecessary if you using the latest changes in the [`cumulus-template-deploy` repo which will automatically determine the AMI ID for you
 based on your `deploy_to_ngap` variable](https://github.com/nasa/cumulus-template-deploy/commit/8472e2f3a7185d77bb68bf9e0f21a92a91b0cba9).
 
+:::
+
 Look up the recommended machine image ID for the Linux version and AWS region of your deployment. See [Linux Amazon ECS-optimized AMIs docs](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/ecs-optimized_AMI.html#ecs-optimized-ami-linux). The image ID, beginning with `ami-`, will be assigned to the `ecs_cluster_instance_image_id` variable for the [cumulus-tf module](https://github.com/nasa/cumulus/blob/master/tf-modules/cumulus/variables.tf).
 
 ### Set Up EC2 Key Pair (Optional)
@@ -188,7 +206,11 @@ Follow the directions on [how to register an application](https://wiki.earthdata
 
 ## Create Resources for Terraform State
 
-> _If you're re-deploying an existing Cumulus configuration you should skip to [Deploy the Cumulus instance](#deploy-the-cumulus-instance), as these values should already be configured._
+:::info
+
+If you're re-deploying an existing Cumulus configuration you should skip to [Deploy the Cumulus instance](#deploy-the-cumulus-instance), as these values should already be configured.
+
+:::
 
 The state of the Terraform deployment is stored in S3. In the following examples, it will be assumed that state is being stored in a bucket called `my-tf-state`. You can also use an existing bucket, if desired.
 
@@ -198,15 +220,25 @@ The state of the Terraform deployment is stored in S3. In the following examples
 aws s3api create-bucket --bucket my-tf-state
 ```
 
+:::tip
+
 In order to help prevent loss of state information, **it is strongly recommended that versioning be enabled on the state bucket**.
 
+:::
+
 ```shell
 aws s3api put-bucket-versioning \
     --bucket my-tf-state \
     --versioning-configuration Status=Enabled
 ```
 
-> ⚠️ **Note:** If your state information does become lost or corrupt, then deployment (via `terraform apply`) will have unpredictable results, including possible loss of data and loss of deployed resources. In order to reduce your risk of the corruption or loss of your Terraform state file, or otherwise corrupt your Cumulus deployment, please see the [Terraform Best Practices](terraform-best-practices.md) guide.
+:::danger important: terraform state
+
+In order to reduce your risk of the corruption or loss of your Terraform state file, or otherwise corrupt your Cumulus deployment, please see the [Terraform Best Practices](terraform-best-practices.md) guide.
+
+However, unfortunately, if your state information does become lost or corrupt, then deployment (via `terraform apply`) will have unpredictable results, including possible loss of data and loss of deployed resources.
+
+:::
 
 ### Create the Locks Table
 
@@ -227,7 +259,7 @@ $ aws dynamodb create-table \
 
 Cumulus requires a [PostgreSQL compatible database](../deployment/postgres-database-deployment.md) cluster deployed to AWS. We suggest utilizing [RDS](https://docs.aws.amazon.com/rds/index.html). For further guidance about what type of RDS database to use, please [see the guide on choosing and configuring your RDS database](./choosing_configuring_rds.md).
 
-Cumulus provides a default [template and RDS cluster module](postgres-database-deployment.md) utilizing Aurora Serverless.
+Cumulus provides a default [template and RDS cluster module](../deployment/postgres-database-deployment.md) utilizing Aurora Serverless.
 
 However, Core intentionally provides a "bring your own" approach, and any well-planned cluster setup should work, given the following:
 
@@ -272,8 +304,12 @@ Each of these modules have to be deployed independently and require their own Te
 
 ### Troubleshooting
 
+:::tip
+
 Please see our [troubleshooting documentation for any issues with your deployment](../troubleshooting/troubleshooting-deployment) when performing the upcoming steps.
 
+:::
+
 ### Configure and Deploy the `data-persistence-tf` Root Module
 
 These steps should be executed in the `data-persistence-tf` directory of the template deploy repo that you previously cloned. Run the following to copy the example files.
@@ -342,7 +378,11 @@ Your data persistence resources are now deployed.
 
 ### Deploy the Cumulus Message Adapter Layer (DEPRECATED)
 
-> ⚠️ **Note:** This step is unnecessary if you using the latest changes in the [`cumulus-template-deploy` repo which will automatically download the Cumulus Message Adapter and create the layer for you based on your `cumulus_message_adapter_version` variable](https://github.com/nasa/cumulus-template-deploy/commit/8472e2f3a7185d77bb68bf9e0f21a92a91b0cba9).
+:::info
+
+This step is unnecessary if you using the latest changes in the [`cumulus-template-deploy` repo which will automatically download the Cumulus Message Adapter and create the layer for you based on your `cumulus_message_adapter_version` variable](https://github.com/nasa/cumulus-template-deploy/commit/8472e2f3a7185d77bb68bf9e0f21a92a91b0cba9).
+
+:::
 
 The [Cumulus Message Adapter (CMA)](./../workflows/input_output.md#cumulus-message-adapter) is necessary for interpreting the input and output of Cumulus workflow steps. The CMA is now integrated with Cumulus workflow steps as a Lambda layer.
 
@@ -398,11 +438,21 @@ Consider [the sizing of your Cumulus instance](#cumulus-instance-sizing) when co
 
 ### Choose a Distribution API
 
-Cumulus can be configured to use either the Thin Egress App (TEA) or the Cumulus Distribution API. The default selection is the Thin Egress App if you're using the [Deployment Template](https://github.com/nasa/cumulus-template-deploy).
+#### Default Configuration
+
+If you are deploying from the Cumulus Deployment Template or a configuration based on that repo, the Thin Egress App (TEA) distribution app will be used by default.
+
+#### Configuration Options
+
+Cumulus can be configured to use either TEA or the Cumulus Distribution API. The default selection is the Thin Egress App if you're using the [Deployment Template](https://github.com/nasa/cumulus-template-deploy).
+
+:::note
 
-> ⚠️ **IMPORTANT:** If you already have a deployment using the TEA distribution and want to switch to Cumulus Distribution, there will be an API Gateway change. This means that there will be downtime while you update your CloudFront endpoint to use
+If you already have a deployment using the TEA distribution and want to switch to Cumulus Distribution, there will be an API Gateway change. This means that there will be downtime while you update your CloudFront endpoint to use
 the new API gateway.
 
+:::
+
 #### Configure the Thin Egress App
 
 TEA can be used for Cumulus distribution and is the default selection. It allows authentication using Earthdata Login. Follow the steps [in the TEA documentation](./thin_egress_app) to configure distribution in your `cumulus-tf` deployment.
@@ -430,7 +480,11 @@ distribution_redirect_uri = https://abc123.execute-api.us-east-1.amazonaws.com/D
 distribution_url = https://abc123.execute-api.us-east-1.amazonaws.com/DEV/
 ```
 
-> ⚠️ **Note:** Be sure to copy the redirect URLs because you will need them to update your Earthdata application.
+:::note
+
+Be sure to copy the redirect URLs because you will need them to update your Earthdata application.
+
+:::
 
 ### Update Earthdata Application
 
@@ -452,7 +506,11 @@ If you've lost track of the needed redirect URIs, they can be located on the [AP
 
 ### Dashboard Requirements
 
-Please note that the requirements are similar to the [Cumulus stack deployment requirements](#requirements). The installation instructions below include a step that will install/use the required node version referenced in the `.nvmrc` file in the Dashboard repository.
+:::note
+
+The requirements are similar to the [Cumulus stack deployment requirements](#requirements). The installation instructions below include a step that will install/use the required node version referenced in the `.nvmrc` file in the Dashboard repository.
+
+:::
 
 - git
 - [node 12.18](https://nodejs.org/en/) (use [nvm](https://github.com/creationix/nvm) to upgrade/downgrade)
@@ -506,7 +564,11 @@ If you do not have the correct version of node installed, replace `nvm use` with
 
 ### Building the Dashboard
 
-> ⚠️ **Note**: These environment variables are available during the build: `APIROOT`, `DAAC_NAME`, `STAGE`, `HIDE_PDR`. Any of these can be set on the command line to override the values contained in `config.js` when running the build below.
+:::caution
+
+These environment variables are available during the build: `APIROOT`, `DAAC_NAME`, `STAGE`, `HIDE_PDR`. Any of these can be set on the command line to override the values contained in `config.js` when running the build below.
+
+:::
 
 To configure your dashboard for deployment, set the `APIROOT` environment variable to your app's API root.[^2]
 

diff --git a/docs/deployment/api_gateway_logging.md b/docs/deployment/api_gateway_logging.md
@@ -44,9 +44,13 @@ This is a one time operation that must be performed on each AWS account to allow
 
 2. ### Create an account role to act as ApiGateway and write to CloudWatchLogs
 
-    > NASA users in NGAP: be sure to use your account's permission boundary.
+:::info in NGAP
 
-    ```sh
+**NASA users in NGAP**: Be sure to use your account's permission boundary.
+
+:::
+
+```sh
     aws iam create-role \
     --role-name ApiGatewayToCloudWatchLogs \
     [--permissions-boundary <permissionBoundaryArn>] \

diff --git a/docs/deployment/apis-introduction.mdx b/docs/deployment/apis-introduction.mdx
@@ -0,0 +1,23 @@
+---
+id: apis-introduction
+title: APIs
+hide_title: false
+---
+
+import DocCardList from '@theme/DocCardList';
+
+### Common Distribution APIs
+
+When deploying from the Cumulus Deployment Template or a configuration based on that repo, the Thin Egress App (TEA) distribution app will be used by default. However, you have the choice to use the Cumulus Distribution API as well.
+
+### Cumulus API Customization Use Cases
+
+Our Cumulus API offers you the flexibility to customize for your DAAC/organization. Below is a list of use cases that may help you with options:
+
+- [Cumulus API w/Launchpad Authentication](https://wiki.earthdata.nasa.gov/display/CUMULUS/Cumulus+API+with+Launchpad+Authentication)
+- [Using Cumulus with Private APIs](https://wiki.earthdata.nasa.gov/display/CUMULUS/Using+Cumulus+with+Private+APIs)
+- [Connecting to Cumulus Private APIs via socks5 proxy](https://wiki.earthdata.nasa.gov/display/CUMULUS/Connecting+to+Cumulus+Private+APIs+via+socks5+proxy)
+
+### Types of APIs
+
+<DocCardList />
diff --git a/docs/deployment/choosing_configuring_rds.md b/docs/deployment/choosing_configuring_rds.md
@@ -1,6 +1,6 @@
 ---
 id: choosing_configuring_rds
-title: Choosing and Configuration Your RDS Database
+title: "RDS: Choosing and Configuring Your Database Type"
 hide_title: false
 ---
 
@@ -72,3 +72,15 @@ the cluster minimum and maximum capacity, and more as seen in the [supported var
 
 Unfortunately, Terraform currently doesn't allow specifying the autoscaling timeout itself, so
 that value will have to be manually configured in the AWS console or CLI.
+
+## Optional: Manage RDS Database with pgAdmin
+
+### Setup SSM Port Forwarding
+
+:::note
+
+In order to perform this action you will need to deploy it within a VPC and have the credentials to access via NGAP protocols.
+
+:::
+
+For a walkthrough guide on how to utilize AWS's Session Manager for port forwarding to access the Cumulus RDS database go to the [Accessing Cumulus RDS database via SSM Port Forwarding](https://wiki.earthdata.nasa.gov/display/CUMULUS/Accessing+Cumulus+RDS+database+via+SSM+Port+Forwarding) article.
diff --git a/docs/deployment/create_bucket.md b/docs/deployment/create_bucket.md
@@ -22,7 +22,11 @@ $ aws s3api create-bucket \
 }
 ```
 
-> ⚠️ **Note:** The `region` and `create-bucket-configuration` arguments are only necessary if you are creating a bucket outside of the `us-east-1` region.
+:::info
+
+The `region` and `create-bucket-configuration` arguments are only necessary if you are creating a bucket outside of the `us-east-1` region.
+
+:::
 
 Please note security settings and other bucket options can be set via the options listed in the ``s3api`` documentation.
-Original file line number
+Diff line change
@@ Expand Up / @@ -22,7 +22,11 @@ $ aws s3api create-bucket \ @@
     }
     ```
-    > ⚠️ **Note:** The `region` and `create-bucket-configuration` arguments are only necessary if you are creating a bucket outside of the `us-east-1` region.
+    :::info
+    The `region` and `create-bucket-configuration` arguments are only necessary if you are creating a bucket outside of the `us-east-1` region.
+    :::
     Please note security settings and other bucket options can be set via the options listed in the ``s3api`` documentation.
@@ Expand Down @@