feat(quorum): introduce helm chart deployment capability

This commit enhances the deployment process for the Quorum Distributed Ledger Technology (DLT) network by exclusively leveraging Helm charts. Four new Helm charts have been introduced to streamline the deployment process: 1. quorum-genesis: Initializes the network with specific requirements. 2. quorum-node: Starts the network and allows adding new nodes. 3. quorum-tessera-node: Facilitates privacy by enabling transaction encryption using Tessera. 4. quorum-tlscert-gen: Enables secure access to the node via HTTPS. Each chart comes with a README to help users customize the network. These changes make deploying and managing the Quorum DLT network smoother with Helm. Other Improvements: - Added error handling to ensure clean uninstallation of the besu-genesis and besu-tlscert-gen charts. - Updated the README's API Call section. fixes #2484 Signed-off-by: saurabhkumarkardam <[email protected]>
hyperledger-bevel · Mar 4, 2024 · 7e4e834 · 7e4e834
1 parent e43b478
commit 7e4e834
Show file tree

Hide file tree

Showing 91 changed files with 3,804 additions and 4,258 deletions.
diff --git a/platforms/hyperledger-besu/charts/README.md b/platforms/hyperledger-besu/charts/README.md
@@ -110,24 +110,48 @@ helm install genesis ./besu-genesis --namespace carrier-bes --values ./values/pr
 helm install carrier ./besu-node --namespace carrier-bes --values ./values/proxy-and-vault/txnode-sec.yaml --set global.proxy.p2p=15016 --set node.besu.identity="O=Carrier,OU=Carrier,L=51.50/-0.13/London,C=GB"
 ```
 
-### API Calls
-Once deployed, services are available as follows on the address as provided in your `global.proxy.externalUrlSuffix`.
+### API call
 
-```bash
-# HTTP RPC API
-curl -v -X POST -H "Content-Type: application/json" --data '{"jsonrpc":"2.0","method":"eth_blockNumber","params":[],"id":1}' http://supplychainrpc.test.yourdomain.com
-
-# which should return (confirming that the node running the JSON-RPC service is syncing):
-{
-  "jsonrpc" : "2.0",
-  "id" : 1,
-  "result" : "0x64"
-}
-```
+Once your services are deployed, they can be accessed using the domain name provided in your `global.proxy.externalUrlSuffix`.
+
+1. **Retrieve the Source Host for Your Node**
+
+   Run the following command to get the mapping for your node:
+
+   ```bash
+   kubectl get mapping --namespace supplychain-bes
+   ```
+
+   From the output, copy the source host for your node.
+
+2. **Make HTTP RPC API Calls**
+
+   You can interact with your node using HTTP RPC API calls. Here's an example of how to do it:
+
+   ```bash
+   curl -X POST -H "Content-Type: application/json" --data '{"jsonrpc":"2.0","method":"eth_blockNumber","params":[],"id":1}' http://<source-host>
+   ```
+
+   Replace `<source-host>` with the source host you copied earlier.
+
+3. **Verify the Node Syncing Status**
+
+   If the node running the JSON-RPC service is syncing correctly, the previous command should return the following:
+
+   ```json
+   {
+     "jsonrpc" : "2.0",
+     "id" : 1,
+     "result" : "0x64"
+   }
+   ```
+
+   This confirms that your node is syncing as expected.
 
 ### Clean-up
 
-To clean up, just uninstall the helm releases.
+To clean up, simply uninstall the Helm releases. It's important to uninstall the genesis Helm chart at the end to prevent any cleanup failure.
+
 ```bash
 helm uninstall --namespace supplychain-bes validator-1
 helm uninstall --namespace supplychain-bes validator-2
@@ -145,4 +169,5 @@ To deploy the proposed validator chart, we need to deploy the Besu node chart fi
 
 ```bash
 helm install validator-5 ./besu-propose-validator --namespace supplychain-bes --values besu-propose-validator/values.yaml
+>>>>>>> upstream/develop
 ```
diff --git a/platforms/hyperledger-besu/charts/besu-genesis/templates/genesis-job-cleanup.yaml b/platforms/hyperledger-besu/charts/besu-genesis/templates/genesis-job-cleanup.yaml
@@ -41,11 +41,14 @@ spec:
             - |
 
 {{- if .Values.settings.removeGenesisOnDelete }}
+              if kubectl get configmap --namespace {{ .Release.Namespace }} besu-genesis &> /dev/null; then
+                echo "Deleting genesis configmap in k8s ..."
+                kubectl delete configmap --namespace {{ .Release.Namespace }} besu-genesis
+              fi
 
-              echo "Deleting genesis configmap in k8s ..."
-              kubectl delete configmap --namespace {{ .Release.Namespace }} besu-genesis
-
-              echo "Deleting node-enodes configmap in k8s ..."
-              kubectl delete configmap --namespace {{ .Release.Namespace }} besu-peers
+              if kubectl get configmap --namespace {{ .Release.Namespace }} besu-peers &> /dev/null; then
+                echo "Deleting node-enodes configmap in k8s ..."
+                kubectl delete configmap --namespace {{ .Release.Namespace }} besu-peers
+              fi
 
 {{- end}} 
diff --git a/platforms/hyperledger-besu/charts/besu-node/templates/node-statefulset.yaml b/platforms/hyperledger-besu/charts/besu-node/templates/node-statefulset.yaml
@@ -172,10 +172,9 @@ spec:
               --identity={{ .Values.node.besu.identity | quote }} --miner-enabled=false \
               --Xdns-enabled=true --Xdns-update-enabled=true --Xnat-kube-service-name={{ include "besu-node.fullname" . }} \
               --min-gas-price=0
-
         livenessProbe:
           httpGet:
-            path: /liveness
+            path: /
             port: 8545
           initialDelaySeconds: 180
           periodSeconds: 60

diff --git a/platforms/hyperledger-besu/charts/besu-tlscert-gen/templates/job-cleanup.yaml b/platforms/hyperledger-besu/charts/besu-tlscert-gen/templates/job-cleanup.yaml
@@ -43,5 +43,7 @@ spec:
             - -c
           args:
             - |
-              echo "Deleting tls-certs secret in k8s ..."
-              kubectl delete secret --namespace {{ .Release.Namespace }} {{ include "besu-tlscert-gen.name" . }}-tls-certs
+              if kubectl get secret --namespace {{ .Release.Namespace }} {{ include "besu-tlscert-gen.name" . }}-tls-certs &>/dev/null; then
+                echo "Deleting tls-certs secret in k8s ..."
+                kubectl delete secret --namespace {{ .Release.Namespace }} {{ include "besu-tlscert-gen.name" . }}-tls-certs
+              fi
diff --git a/platforms/hyperledger-besu/charts/besu-tlscert-gen/templates/job.yaml b/platforms/hyperledger-besu/charts/besu-tlscert-gen/templates/job.yaml
@@ -95,8 +95,7 @@ spec:
           cert=$(echo ${VAULT_SECRET} | jq -r '.["ambassadorcrt"]')
 
           # If the cert is null, empty, or contains a parse error, then the certificates do not exist in Vault
-          if [ "$cert" == "null" ] || [[ "$cert" = "parse error"* ]] || [ "$cert" = "" ]
-          then
+          if [ "$cert" == "null" ] || [[ "$cert" = *"error"* ]] || [ "$cert" = "" ]; then
             # Create a file to indicate that the ambassador TLS certificates are absent
             echo "Certficates absent in vault. Ignore error warning"
             touch ${OUTPUT_PATH}/ambassadortls_absent.txt
@@ -313,8 +312,7 @@ spec:
                 # Check if any of the certificate and key fields are missing, empty or having any kind of error
                 for field in "$CA_PEM" "$CA_KEY" "$AMBASSADORCRT" "$AMBASSADORKEY" "$KEYSTORE" "$PASSWORD" "$KNOWNSERVER"
                 do
-                  if [ "$field" = "null"  ] || [[ "$field" = "parse error"* ]] || [ "$field" = "" ]
-                  then
+                  if [ "$field" = "null" ] || [[ "$field" = *"error"* ]] || [ "$field" = "" ]; then
                     AMBASSADORTLS_CERT_WRITTEN=false
                     break
                   else
@@ -334,8 +332,7 @@ spec:
                 # Check if any of the certificate and key fields are missing, empty or having any kind of error
                 for field in "$CA_PEM" "$CA_KEY" "$AMBASSADORCRT" "$AMBASSADORKEY"
                 do
-                  if [ "$field" = "null"  ] || [[ "$field" = "parse error"* ]] || [ "$field" = "" ]
-                  then
+                  if [ "$field" = "null" ] || [[ "$field" = *"error"* ]] || [ "$field" = "" ]; then
                     AMBASSADORTLS_CERT_WRITTEN=false
                     break
                   else

diff --git a/platforms/quorum/charts/README.md b/platforms/quorum/charts/README.md
@@ -3,20 +3,220 @@
 [//]: # (SPDX-License-Identifier: Apache-2.0)
 [//]: # (##############################################################################################)
 
-# Charts for Quorum components
+# Charts for Goquorum components
 
 ## About
-This folder contains helm charts which are used by the ansible playbooks for the deployment of the Quorum components. Each chart folder contain a folder for templates, chart file and the corresponding value file. 
-
-## Example Folder Structure ###
-```
-/quorum-member-node
-|-- templates
-|   |-- _helpers.tpl
-|   |-- configmap.yaml
-|   |-- deployment.yaml
-|   |-- ingress.yaml
-|   |-- service.yaml
-|-- Chart.yaml
-|-- values.yaml
+This folder contains the helm charts which are used for the deployment of the Hyperledger Goquorum components. Each helm that you can use has the following keys and you need to set them. The `global.cluster.provider` is used as a key for the various cloud features enabled. Also you only need to specify one cloud provider, **not** both if deploying to cloud. As of writing this doc, AWS is fully supported.
+
+```yaml
+global:
+  serviceAccountName: vault-auth
+  cluster:
+    provider: aws   # choose from: minikube | aws
+    cloudNativeServices: false  # future: set to true to use Cloud Native Services 
+    kubernetesUrl: "https://yourkubernetes.com" # Provide the k8s URL, ignore if not using Hashicorp Vault
+  vault:
+    type: hashicorp # choose from hashicorp | kubernetes
+    network: quorum   # must be quorum for these charts
+    # Following are necessary only when hashicorp vault is used.
+    address: http://vault.url:8200
+    authPath: supplychain
+    secretEngine: secretsv2
+    secretPrefix: "data/supplychain"
+    role: vault-role
+```
+
+## Usage
+
+### Pre-requisites
+
+- Kubernetes Cluster (either Managed cloud option like EKS or local like minikube)
+- Accessible and unsealed Hahsicorp Vault (if using Vault)
+- Configured Ambassador AES (if using Ambassador as proxy)
+- Update the dependencies
+  ```
+  helm dependency update quorum-genesis
+  helm dependency update quorum-node
+  ```
+
+
+## `Without Proxy and Vault`
+
+### 1. Install Genesis Node
+```bash
+# Install the genesis node
+helm install genesis ./quorum-genesis --namespace supplychain-quo --create-namespace --values ./values/noproxy-and-novault/genesis.yaml
+```
+
+### 2. Install Validator Nodes
+```bash
+# Install validator nodes
+helm install validator-1 ./quorum-node --namespace supplychain-quo --values ./values/noproxy-and-novault/validator.yaml
+helm install validator-2 ./quorum-node --namespace supplychain-quo --values ./values/noproxy-and-novault/validator.yaml
+helm install validator-3 ./quorum-node --namespace supplychain-quo --values ./values/noproxy-and-novault/validator.yaml
+helm install validator-4 ./quorum-node --namespace supplychain-quo --values ./values/noproxy-and-novault/validator.yaml
+```
+
+### 3. Deploy Member and Tessera Node Pair
+```bash
+# Deploy Quorum and Tessera node pair
+helm install member-1 ./quorum-node --namespace supplychain-quo --values ./values/noproxy-and-novault/txnode.yaml
+```
+
+### Setting Up Another Member in a Different Namespace
+
+```bash
+# Get the genesis and static nodes from existing member and and place them in the directory 'besu-genesis/files'
+cd ./quorum-genesis/files/
+kubectl --namespace supplychain-quo get configmap quorum-peers -o jsonpath='{.data.static-nodes\.json}' > static-nodes.json
+kubectl --namespace supplychain-quo get configmap quorum-genesis  -o jsonpath='{.data.genesis\.json}' > genesis.json
+
+# Install secondary genesis node
+helm install genesis ./quorum-genesis --namespace carrier-quo --values ./values/noproxy-and-novault/genesis-sec.yaml
+
+# Install secondary member node
+helm install member-2 ./quorum-node --namespace carrier-quo --values ./values/noproxy-and-novault/txnode-sec.yaml
+```
+
+---
+
+## `With Ambassador Proxy and Vault`
+
+### 1. Create Namespace and Secret
+```bash
+# Create a namespace
+kubectl create namespace supplychain-quo
+
+# Create the roottoken secret
+kubectl -n supplychain-quo create secret generic roottoken --from-literal=token=<VAULT_ROOT_TOKEN>
+```
+
+### 2. Install Genesis Node
+```bash
+# Install the genesis node
+helm install genesis ./quorum-genesis --namespace supplychain-quo --values ./values/proxy-and-vault/genesis.yaml
+```
+
+### 3. Install Validator Nodes
+```bash
+# Install validator nodes
+helm install validator-1 ./quorum-node --namespace supplychain-quo --values ./values/proxy-and-vault/validator.yaml --set global.proxy.p2p=15011
+helm install validator-2 ./quorum-node --namespace supplychain-quo --values ./values/proxy-and-vault/validator.yaml --set global.proxy.p2p=15012
+helm install validator-3 ./quorum-node --namespace supplychain-quo --values ./values/proxy-and-vault/validator.yaml --set global.proxy.p2p=15013
+helm install validator-4 ./quorum-node --namespace supplychain-quo --values ./values/proxy-and-vault/validator.yaml --set global.proxy.p2p=15014
+```
+
+### 4. Deploy Member and Tessera Node Pair
+```bash
+# Deploy Quorum and Tessera node pair
+helm install supplychain ./quorum-node --namespace supplychain-quo --values ./values/proxy-and-vault/txnode.yaml --set global.proxy.p2p=15015
+```
+
+### Setting Up Another Member in a Different Namespace
+
+```bash
+# Get the genesis and static nodes from existing member and and place them in the directory 'besu-genesis/files'
+cd ./quorum-genesis/files/
+kubectl --namespace supplychain-quo get configmap quorum-peers -o jsonpath='{.data.static-nodes\.json}' > static-nodes.json
+kubectl --namespace supplychain-quo get configmap quorum-genesis  -o jsonpath='{.data.genesis\.json}' > genesis.json
+
+# Create a new namespace
+kubectl create namespace carrier-quo
+
+# Create the roottoken secret
+kubectl -n carrier-quo create secret generic roottoken --from-literal=token=<VAULT_ROOT_TOKEN>
+
+# Install secondary genesis node
+helm install genesis ./quorum-genesis --namespace carrier-quo --values ./values/proxy-and-vault/genesis-sec.yaml
+
+# Install secondary member node
+helm install carrier ./quorum-node --namespace carrier-quo --values ./values/proxy-and-vault/txnode-sec.yaml --set global.proxy.p2p=15016
+```
+
+## `API call`
+
+Once your services are deployed, they can be accessed using the domain name provided in your `global.proxy.externalUrlSuffix`.
+
+1. **Retrieve the Source Host for Your Node**
+
+   Run the following command to get the mapping for your node:
+
+   ```bash
+   kubectl get mapping --namespace supplychain-quo
+   ```
+
+   From the output, copy the source host for your node.
+
+2. **Make HTTP RPC API Calls**
+
+   You can interact with your node using HTTP RPC API calls. Here's an example of how to do it:
+
+   ```bash
+   curl -X POST -H "Content-Type: application/json" --data '{"jsonrpc":"2.0","method":"eth_blockNumber","params":[],"id":1}' http://<source-host>
+   ```
+
+   Replace `<source-host>` with the source host you copied earlier.
+
+3. **Verify the Node Syncing Status**
+
+   If the node running the JSON-RPC service is syncing correctly, the previous command should return the following:
+
+   ```json
+   {
+     "jsonrpc" : "2.0",
+     "id" : 1,
+     "result" : "0x64"
+   }
+   ```
+
+   This confirms that your node is syncing as expected.
+
+## `Managing IBFT Validators Deployment`
+
+To deploy the proposed validator chart for IBFT, you first need to set up the Quorum DLT network. Below are the steps you can follow:
+
+1. **Deploy Quorum DLT Network**:
+   You have two options for deploying the Quorum DLT network:
+
+   - **With Vault and Proxy**
+   - **Without Vault and Proxy**
+
+   Choose the appropriate method based on your requirements.
+
+2. **Install Validator Chart**:
+   Utilize Helm for installing the validator chart. Ensure to adjust values accordingly:
+
+   ```bash
+   helm install validator-5 ./quorum-propose-validator --namespace supplychain-quo --values quorum-propose-validator/values.yaml
+   ```
+
+   This chart facilitates the addition or removal of validators through majority voting.
+
+3. **Verify Validator Status**:
+   Confirm the validator status by executing:
+
+   ```bash
+   curl -X POST -H "Content-Type: application/json" --data '{"jsonrpc":"2.0","method":"istanbul_getValidators","params":["latest"],"id":1}' http://<SOURCE-HOST>
+   ```
+
+	This command retrieves the current list of validators for the latest block.
+
+	Replace `<SOURCE-HOST>` with the appropriate host address.
+
+
+## `Clean-up`
+
+To clean up, simply uninstall the Helm releases. It's important to uninstall the genesis Helm chart at the end to prevent any cleanup failure.
+
+```bash
+helm uninstall --namespace supplychain-quo validator-1
+helm uninstall --namespace supplychain-quo validator-2
+helm uninstall --namespace supplychain-quo validator-3
+helm uninstall --namespace supplychain-quo validator-4
+helm uninstall --namespace supplychain-quo validator-5
+helm uninstall --namespace supplychain-quo supplychain
+helm uninstall --namespace supplychain-quo genesis
+
+helm uninstall --namespace carrier-quo carrier
+helm uninstall --namespace carrier-quo genesis
 ```
diff --git a/platforms/quorum/charts/quorum-genesis/Chart.yaml b/platforms/quorum/charts/quorum-genesis/Chart.yaml
@@ -0,0 +1,22 @@
+# apiVersion: v2
+apiVersion: v1
+name: quorum-genesis
+description: "Quorum: This Helm chart deploys genesis."
+type: application
+version: 1.0.0
+appVersion: latest
+keywords:
+  - bevel
+  - ethereum
+  - quorum
+  - hyperledger
+  - enterprise
+  - blockchain
+  - deployment
+  - accenture
+home: https://hyperledger-bevel.readthedocs.io/en/latest/
+sources:
+  - https://github.com/hyperledger/bevel
+maintainers:
+  - name: Hyperledger Bevel maintainers
+    email: [email protected]