Skip to content

Latest commit

 

History

History
262 lines (174 loc) · 11.7 KB

README.md

File metadata and controls

262 lines (174 loc) · 11.7 KB

Kubernetes LocalNet

This guide shows how to deploy a LocalNet using pocket-operator.

TL;DR

If you feel adventurous, and you know what you're doing, here is a rapid guide to start LocalNet:

  1. Install Docker
  2. Install Kind (e.g., brew install kind)
  3. Create Kind cluster (e.g., kind create cluster)
  4. Start LocalNet (e.g., make localnet_up)

Otherwise, please continue with the full guide.

Dependencies

All necessary dependencies, except Docker and Kubernetes cluster, are installed automatically when running make install_cli_deps. The following dependencies are required for LocalNet to function:

  1. tilt
  2. Docker or Docker Desktop
  3. Kubernetes cluster: refer to Choosing Kubernetes Distribution section for more details.
  4. kubectl: CLI is required and should be configured to access the cluster. This should happen automatically if using Docker Desktop, Rancher Desktop, k3s, k3d, minikube, etc.
  5. helm: required to template the YAML manifests for the dependencies (e.g., Postgres, Grafana). Installation instructions available.
  6. rsync: required to for some extensions used with Tilt; https://github.com/tilt-dev/tilt-extensions/tree/master/syncback#usage

Choosing Kubernetes Distribution

While any Kubernetes distribution should work, we've had success and recommend to use Kind. To run Kind Kubernetes clusters, you need to have Docker installed.

How to create Kind Kubernetes cluster

Kind depends solely on Docker. To install Kind, follow this page.

Create a new cluster:

kind create cluster

Make sure your Kubernetes context is pointed to new kind cluster:

kubectl config current-context

should return kind-kind. Now you can start LocalNet!

LocalNet

Starting LocalNet

make localnet_up

This action will create a file called localnet_config.yaml in the root of the repo if it doesn't exist. The default configuration can be found in Tiltfile.

Viewing Logs

The developer can then view logs either from a browser or terminal.

Terminal Logs

  • make localnet_logs_validators - shows prior logs
  • make localnet_logs_validators_follow - shows prior logs and follows the new log lines as validators do their work

Tilt Web UI Logs

tilt UI

Scaling Actors

When starting a k8s LocalNet, localnet_config.yaml is generated (with default configs) in the root of the repo if doesn't already exist.

The config file can be modified to scale the number of actors up/down. As long as localnet_up is running, the changes should be automatically applied within seconds.

demo-scaling.mp4

Stopping & Clean Resources

make localnet_down

The command stops LocalNet and cleans up all the resources, including the postgres database.

Interacting w/ LocalNet

As the workloads run in Kubernetes, you can see and modify any resources on your local kubernetes by a tool of your choice (k9s, Lens, VSCode extension, etc), but note that Tilt will change manifests back eventually.

Make Targets

Open a shell in the pod that has client cli available. It gets updated automatically whenever the code changes:

make localnet_shell

Open a client debug cli. It allows to interact with blockchain, e.g. change pace maker mode, reset to genesis, etc. It gets updated automatically whenever the code changes (though you would need to stop/start the binary to execute the new code):

make localnet_client_debug

Addresses and keys on LocalNet

You can find private keys and addresses for all actors in the private-keys.yaml file. They have been pre-generated and follow a specific pattern – they start with pre-determined numbers for easier troubleshooting and debugging.

Addresses begin with YYYXX number, where YYY is a number of an actor and XX is a type of actor.

The current mapping for XX is:

  • 01 - Application
  • 02 - Servicer
  • 03 - Fisherman
  • 04 - Validator

For example:

  • 420043b854e78f2d5f03895bba9ef16972913320 is a validator #420.
  • 66603bc4082281b7de23001ffd237da62c66a839 is a fisherperson #666.
  • 0010297b55fc9278e4be4f1bcfe52bf9bd0443f8 is a servicer #001.
  • 314019dbb7faf8390c1f0cf4976ef1215c90b7e4 is an application #314.

Applications staked on LocalNet

Applications with the following addresses are staked on LocalNet, through the applications field of the genesis.json in the LocalNet configuration

  • 00001fff518b1cdddd74c197d76ba5b5dedc0301
  • 00101f2ff54811e84df2d767c661f57a06349b7e

These addresses can be used for e.g. testing the CLI.

Servicers staked on LocalNet

Servicers with the following addresses are staked on LocalNet, through the servicers field of the genesis.json in the LocalNet configuration

  • 00002b8cea1bcc3dadc72ebecf95564ceb9c2e2a
  • 001022b138896c4c5466ac86b24a9bbe249905c2

How to change configuration files

Configurations can be changed in helm charts where network protocol actor configs are maintained. You can find them in this directory.

config.json file is created using the config section content in values.yaml. For example, you can find the configuration for a validator here.

If you need to add a new parameter – feel free to modify the section in place. Some of the parameters that contain secrets (e.g. private key), are stored in Secrets object and injected as environment variables.

Please refer to helm charts documentation for more details.

Overriding default values for localnet with Tilt

You may also create a overrides YAML file in the charts/pocket directory and override the default values for the various actors.

Override files supported:

  • pocket-fisherman-overrides.yaml
  • pocket-servicer-overrides.yaml
  • pocket-validator-overrides.yaml
touch charts/pocket/pocket-validator-overrides.yaml

For example, to enable CORS for the RPC server, you can run the following command. Note, this overrides config.json for all validators in localnet.

cat <<EOF > charts/pocket/pocket-validator-overrides.yaml
config:
  rpc:
    use_cors: true
EOF

How does it work?

tilt reads the Tiltfile, where LocalNet configs are specified. Tiltfile is written in Starlark, a dialect of Python.

The k8s manifests that tilt submits to the cluster can be found in this directory. Please refer to code structure for more details where different parts are located.

Tilt continuously monitors files on local filesystem in specific directories, and it rebuilds the binary and distributes it to the pods on every code change. This allows developers to iterate on the code and see the changes immediately (i.e. hot-reloading).

Troubleshooting

Why?

Developers might experience issues with running LocalNet on Kubernetes.

Issues might be related to the fact different developers run different clusters/OS/environments.

Machines going to sleep and that might not play well with virtual machines, postgres or pocket client.

Force Trigger an Update

Visit the tilt web UI by pressing space in the shell where you started tilt or by visiting this webpage.

If you see any errors, you can click Trigger Update on a resource that has issues to restart the service or retry a command.

Force Restart

If force triggering an update didn't work, try the following:

  1. make localnet_down
  2. make localnet_up

Full Cleanup

If a force restart didn't help, try rebuilding local kubernetes cluster using the tool you're managing your cluster with (e.g. Docker Desktop, Rancher Desktop, k3s, k3d, minikube, etc).

Code Structure

build/localnet
├── README.md # This file
├── Tiltfile # File outlining tilt process
├── dependencies # Helm charts that install all the dependencies needed to run and observe LocalNet
│   ├── Chart.yaml # Main file of the helm chart, contains metadata
│   ├── dashboards # Directory with all the dashboards that are automatically imported to Grafana
│   │   ├── README.md # README file with instructions on how to add a new dashboard
│   │   └── raintree-telemetry-graphs.json # Raintree Telemetry dashboard
│   ├── requirements.yaml # Specifies dependencies of the chart, this allows us to install all the dependencies with a single command
│   ├── templates # Additional Kubernetes manifests that we need to connect different dependencies together
│   │   ├── _helpers.tpl
│   │   ├── dashboards.yml
│   │   └── datasources.yml
│   └── values.yaml # Configuration values that override the default values of the dependencies, this allows us to connect dependencies together and make them available to our LocalNet services
└── manifests # Static YAML Kubernetes manifests that are consumed by `tilt`
    ├── cli-client.yaml # Pod that has the latest binary of the pocket client. Makefile targets run CLI in this pod.
    ├── configs.yaml # Location of the config files (default configs for all validators and a genesis file) that are shared across all actors
    ├── network.yaml # Networking configuration that is shared between different actors, currently a Service that points to all validators
    └── private-keys.yaml # Pre-generated private keys with a semantic format for easier development