Skip to content

Latest commit

 

History

History
177 lines (149 loc) · 10.2 KB

development.md

File metadata and controls

177 lines (149 loc) · 10.2 KB
Raw

Development

This document will walk you through how to set up a local development environment, build Akri component containers, and test Akri using your newly built containers.

Required Tools

To develop, you'll need:

  • A Linux environment whether on amd64 or arm64v8
  • Rust - version 1.41.0 which the build system uses can be installed using: 
    sudo curl https://sh.rustup.rs -sSf | sh -s -- -y --default-toolchain=1.41.0
cargo version
  • .NET - the ONVIF broker is written in .NET, which can be installed according to .NET instructions

In order to cross-build containers for both ARM and x64, several tools are leveraged:

  • The Cross tool can be installed with this command: cargo install cross.

  • qemu can be installed with:

    sudo apt-get install -y qemu qemu qemu-system-misc qemu-user-static qemu-user binfmt-support

    For qemu to be fully configured on Ubuntu 18.04, after running apt-get install, run these commands:

      sudo mkdir -p /lib/binfmt.d
  sudo sh -c 'echo :qemu-arm:M::\\x7fELF\\x01\\x01\\x01\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x02\\x00\\x28\\x00:\\xff\\xff\\xff\\xff\\xff\\xff\\xff\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\xfe\\xff\\xff\\xff:/usr/bin/qemu-arm-static:F > /lib/binfmt.d/qemu-arm-static.conf'
  sudo sh -c 'echo :qemu-aarch64:M::\\x7fELF\\x02\\x01\\x01\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x02\\x00\\xb7\\x00:\\xff\\xff\\xff\\xff\\xff\\xff\\xff\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\xfe\\xff\\xff\\xff:/usr/bin/qemu-aarch64-static:F > /lib/binfmt.d/qemu-aarch64-static.conf'
  sudo systemctl restart systemd-binfmt.service

Build and Test

Local builds and tests

  1. Navigate to the repo's top folder (where this README is)
  2. Install prerequisites 
    ./build/setup.sh
  3. Build Controller, Agent, and udev broker 
    cargo build
  4. Build ONVIF broker 
    cd ./samples/brokers/onvif-video-broker
dotnet build

There are unit tests for all of the Rust code. To run all unit tests, simply navigate to the repo's top folder (where this README is) and type cargo test

To locally run the controller as part of a k8s cluster, follow these steps:

  1. Create or provide access to a valid cluster configuration by setting KUBECONFIG (can be done in the commandline) ... for the sake of this, the config is assumed to be in ~/test.cluster.config
  2. Build the repo by running cargo build
  3. Run the desired component (in this case, looking at info-level logging and running the controller locally): RUST_LOG=info KUBECONFIG=~/test.cluster.config ./target/debug/controller

To build containers

Makefile has been created to help with the more complicated task of building the Akri components and containers for the various supported platforms.

Establish a container repository

Containers for Akri are currently hosted in ghcr.io/deislabs/akri using the new GitHub container registry. Any container repository can be used for private containers. If you want to enable GHCR, you can follow the getting started guide.

To build containers, log into the desired repository:

CONTAINER_REPOSITORY=<repo>
sudo docker login $CONTAINER_REPOSITORY

Build intermediate containers

To ensure quick builds, we have created a number of intermediate containers that rarely change.

By default, Makefile will try to create containers with tag following this format: <repo>/$USER/<component>:<label> where

  • <component> = rust-crossbuild | opencv-base
  • <repo> = devcaptest.azurecr.io
    • <repo> can be overridden by setting REGISTRY=<desired repo>
  • $USER = the user executing Makefile (could be root if using sudo)
    • <repo>/$USER can be overridden by setting PREFIX=<desired container path>
  • <label> = the label is defined in ../build/intermediate-containers.mk
Rust cross-build containers

These containers are used by the cross tool to crossbuild the Akri Rust code. There is a container built for each supported platform and they contain any required dependencies for Akri components to build. The dockerfile can be found here: build/containers/intermediate/Dockerfile.rust-crossbuild-*

# To make all of the Rust cross-build containers:
PREFIX=$CONTAINER_REPOSITORY make rust-crossbuild
# To make specific platform(s):
PREFIX=$CONTAINER_REPOSITORY BUILD_AMD64=1 BUILD_ARM32=1 BUILD_ARM64=1 make rust-crossbuild

After building the cross container(s), update Cross.toml to point to your intermediate container(s).

.NET OpenCV containers

These containers allow the ONVIF broker to be created without rebuilding OpenCV for .NET each time. There is a container built for AMD64 and it is used to crossbuild to each supported platform. The dockerfile can be found here: build/containers/intermediate/Dockerfile.opencvsharp-build.

# To make all of the OpenCV base containers:
PREFIX=$CONTAINER_REPOSITORY make opencv-base
# To make specific platform(s):
PREFIX=$CONTAINER_REPOSITORY BUILD_AMD64=1 BUILD_ARM32=1 BUILD_ARM64=1 make opencv-base

Build Akri component containers

By default, Makefile will try to create containers with tag following this format: <repo>/$USER/<component>:<label> where

  • <component> = controller | agent | etc
  • <repo> = devcaptest.azurecr.io
    • <repo> can be overridden by setting REGISTRY=<desired repo>
  • $USER = the user executing Makefile (could be root if using sudo)
    • <repo>/$USER can be overridden by setting PREFIX=<desired container path>
  • <label> = v$(cat version.txt)
    • <label> can be overridden by setting LABEL_PREFIX=<desired label>

Note: Before building these final component containers, make sure you have built any necessary intermediate containers. In particular, to build any of the rust containers (the Controller, Agent, or udev broker), you must first build the cross-build containers.

# To make all of the Akri containers:
PREFIX=$CONTAINER_REPOSITORY make akri
# To make a specific component:
PREFIX=$CONTAINER_REPOSITORY make akri-controller
PREFIX=$CONTAINER_REPOSITORY make akri-agent
PREFIX=$CONTAINER_REPOSITORY make akri-udev
PREFIX=$CONTAINER_REPOSITORY make akri-onvif
PREFIX=$CONTAINER_REPOSITORY make akri-opcua-monitoring
PREFIX=$CONTAINER_REPOSITORY make akri-anomaly-detection
PREFIX=$CONTAINER_REPOSITORY make akri-streaming

# To make a specific component on specific platform(s):
PREFIX=$CONTAINER_REPOSITORY BUILD_AMD64=1 BUILD_ARM32=1 BUILD_ARM64=1 make akri-streaming

# To make a specific component on specific platform(s) with a specific label:
PREFIX=$CONTAINER_REPOSITORY LABEL_PREFIX=latest BUILD_AMD64=1 BUILD_ARM32=1 BUILD_ARM64=1 make akri-streaming

NOTE: If your docker install requires you to use sudo, this will conflict with the cross command. This flow has helped:

sudo -s
source /home/$SUDO_USER/.cargo/env

# run make commands that crossbuild the Rust

exit

Install Akri with newly built containers

When installing Akri using helm, you can set the imagePullSecrets, image.repository and image.tag Helm values to point to your newly created containers. For example, to install Akri with with custom Controller and Agent containers, run the following, specifying the image.tag version to reflect version.txt:

kubectl create secret docker-registry <your-secret-name> --docker-server=ghcr.io  --docker-username=<your-github-alias> --docker-password=<your-github-token>
helm repo add akri-helm-charts https://deislabs.github.io/akri/
helm install akri akri-helm-charts/akri-dev \
    --set imagePullSecrets[0].name="<your-secret-name>" \
    --set agent.image.repository="ghcr.io/<your-github-alias>/agent" \
    --set agent.image.tag="v<akri-version>-amd64" \
    --set controller.image.repository="ghcr.io/<your-github-alias>/controller" \
    --set controller.image.tag="v<akri-version>-amd64"

More information about the Akri Helm charts can be found in the user guide.

Other useful Helm Commands

Helm Package

If you make changes to anything in the helm folder, you will probably need to create a new Helm chart for Akri. This can be done using the helm package command. To create a chart using the current state of the Helm templates and CRDs, run (from one level above the Akri directory) helm package akri/deployment/helm/. You will see a tgz file called akri-<akri-version>.tgz at the location where you ran the command. Now, install Akri using that chart:

helm install akri akri-<akri-version>.tgz \
    --set useLatestContainers=true

Helm Template

When you install Akri using Helm, Helm creates the DaemonSet, Deployment, and Configuration yamls for you (using the values set in the install command) and applies them to the cluster. To inspect those yamls before installing Akri, you can use helm template. For example, you will see the image in the Agent DaemonSet set to image: "ghcr.io/<your-github-alias>/agent:v<akri-version>-amd64" if you run the following:

helm template akri deployment/helm/ \
  --set imagePullSecrets[0].name="<your-secret-name>" \
  --set agent.image.repository="ghcr.io/<your-github-alias>/agent" \
  --set agent.image.tag="v<akri-version>-amd64"

Helm Get Manifest

Run the following to inspect an already running Akri installation in order to see the currently applied yamls such as the Configuration CRD, Instance CRD, protocol Configurations, Agent DaemonSet, and Controller Deployment:

helm get manifest akri | less

Helm Upgrade

To modify an Akri installation to reflect a new state, you can use helm upgrade. See the Customizing an Akri Installation document for further explanation.