-
Notifications
You must be signed in to change notification settings - Fork 146
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Allow conditional compilation of agent protocols #196
Conversation
@@ -35,19 +34,23 @@ serde_json = "1.0.45" | |||
serde_yaml = "0.8.11" | |||
serde_derive = "1.0.104" | |||
akri-shared = { path = "../shared" } | |||
sxd-document = "0.3.0" | |||
sxd-xpath = "0.4.0" | |||
tempfile = "3.1.0" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I removed these deps and mime
which didn't seem to be used anywhere
@kate-goldenring Actually I think the PR is ready, I don't have anything else on my mind. The biggest annoyance is that one still needs to manually remove Nevertheless I've been using the changes in the last days while developing the CoRE broker and I haven't noticed any issue. I also don't think that writing a dump agent to allow testing locally is necessary anymore, because of the transition to a gRPC/REST agent in #198 Let me know what do you think. After #196 is merged I can update the version and the PR should be ready to go. |
agent/src/protocols/mod.rs
Outdated
} | ||
|
||
#[derive(Default)] | ||
pub struct NoopDiscoveryHandler {} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
can this be removed?
agent/Cargo.toml
Outdated
|
||
onvif = ["xml-rs", "yaserde", "yaserde_derive"] | ||
opcua = ["opcua-client"] | ||
udevf = ["pest", "pest_derive", "udev"] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
does this need to be udevf
or can it be udev
? (not sure if the 'f' is intentional because of a naming conflict or if it is accidental)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
see comments around 'udevf' and 'noop'
This is a great addition!
The development doc explains how to run the controller locally by passing it the kubeconfig file. You can also run the agent locally, by also specifying some additional runtime parameters needed for slot reconciliation. For example (for MicroK8s): sudo RUST_LOG=info KUBECONFIG=~/.kube/config AGENT_NODE_NAME=kagold HOST_CRICTL_PATH=/usr/local/bin/crictl HOST_RUNTIME_ENDPOINT=/var/snap/microk8s/common/run/containerd.sock HOST_IMAGE_ENDPOINT==/var/snap/microk8s/common/run/containerd.sock ./target/debug/agent
kubectl apply -f configuration.yaml This would be great to add to the documentation, but can also be added in another PR. Also, currently the Agent is unix only due to the underlying rust UnixStreamer and UnixListener libraries used for the Device Plugins |
@bfjelds replaced noop with panic and renamed @kate-goldenring Sorry I wasn't clear in the description. I meant to run the agent locally without being in the cluster, although running in Microk8s would be a good compromise I guess. Personally I'd prefer to avoid for now. Every k8s distro has its own quirks because of different architectures, such as Kind vs Microk8s vs Minikube vs K3s. I'd prefer sticking with my k3s cluster which I'm getting to know better 😄 Btw IIRC Microk8s doesn't run on MacOS, it's Linux-only. |
If you want to use K3s, I believe you should only need to substitute into the command above |
agent/Cargo.toml
Outdated
|
||
onvif = ["xml-rs", "yaserde", "yaserde_derive"] | ||
opcua = ["opcua-client"] | ||
udevfeat = ["pest", "pest_derive", "udev"] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
i haven't used features before, would it make sense to be consistent (even if it is not needed):
default = ["onvif-feature", "opcua-feature", "udev-feature"]
onvif-feature = ...
opcua-feature = ...
udev-feature = ...
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sure, I changed to -feat
suffix. Usually the feature name doesn't mention the word "feature", e.g. smoltcp/Cargo.toml, but consistency >> convention 😄.
@kate-goldenring I can't do it because In the initial description, I meant writing the code on MacOS and also testing it on MacOS before deploying on the cluster. And actually I've been able to do so by using unit tests for the agent. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is going to be a great addition! We are working on updating our workflow files so that the tests are no longer failing. When #202 merges, could you rebase? Then, hopefully the x's will disappear!
@kate-goldenring this one is also ready to go 🎉 |
* test latest k8s versions [SAME VERSION] (#188) * update changelog for 0.1.5 release (#189) * add more description and simplify commands in e2e demo (#191) * Various typos fixed on new OPCUA documentation (#195) * Various typos fixed on new OPCUA documentation Signed-off-by: Didier Durand <[email protected]> * Fixes on extensibility documentation Signed-off-by: Didier Durand <[email protected]> * Make end-to-end demo on Raspberry Pi simpler and more descriptive (#193) * mirror updates in e2e demo in rasp-pi demo * syntax changes, capacity specification, building kernel module * fix syntax issues * Get Agent logs before testing their contents [SAME VERSION] (#197) * add additional checks to see why getting logs failing * log information when get crictl error * use os.system to log info * try using shell with subprocess when doing complex commands * Fix failing to query container runtime in test workflow on MicroK8s [SAME VERSION] (#202) * loop query containerd several times instead of sleeping * shorten agent log query * Allow overwriting controller.nodeSelectors (#194) * Allow overwriting controller.nodeSelectors * Add note about `sed` for MacOS contributors * Allow conditional compilation of agent protocols (#196) * Allow conditional compilation of agent protocols * Bump patch version Co-authored-by: Jiayi Hu <[email protected]> * Prometheus (#190) * add frame count metric to udev broker * add prometheus metrics * format metrics.rs * update Helm charts for Prometheus support * Prometheus documentation * increase supported rust version to latest * fix metrics chart * use PodMonitor instead of ServiceMonitor for Akri Agent and Controller * update tarpaulin to latest version to fix protobuf issue * remove debug echo shared setting * update version * Use anyhow instead of the deprecated failure crate (#201) * Switch to use anyhow crate instead of failure crate Include Cargo.lock updates * Rebase and increment patch version * Merge conflict * Add missing version updates after rebase * [Helm] Added CRD variables (#205) * Added CRD variables * `version.sh` updates Helm `crds.version` * Bump version * Document naming convention (#204) * Initial commit * Remove references to `http` example * Added `udev-broker` * Added `akri-udev` * Rephrased * Typo * naming-guidelines >> development.md * r/udev-broker/udev-video-broker * punctuation * r/Resources/resources * r/Compute/Computer (#210) * fix instructions for cloning and building v4l2 (#209) * Update hyper version and other dependencies * Update patch version * Explicitly specify h2 patch version dependency to avoid future accidentl updates by cargo-update or similar * Remove redundant unnecessary hyper dependencies * Add security audit workflow * Move security audit to existing check-rust workflow * Simplify mockall usage (#214) * Simplify mockall usage * Bump patch version * add project logo and art [SAME VERSION] (#149) * add akri branding and artwork Signed-off-by: flynnduism <[email protected]> * edit artwork README Signed-off-by: flynnduism <[email protected]> * re-render dark logos Signed-off-by: flynnduism <[email protected]> * redraw the svg icon to fix glitchy render Signed-off-by: flynnduism <[email protected]> * update logo.svg Signed-off-by: flynnduism <[email protected]> * fix the mixed up color hex values Signed-off-by: flynnduism <[email protected]> * update branding with inconsolata typeface Signed-off-by: flynnduism <[email protected]> * use correct casing for mentions of Akri Signed-off-by: flynnduism <[email protected]> * remove default documentation to use latest containers (#218) * Image pull policy (#207) * change default image pull policy to IfNotPresent * set ImagePullPolicy to default to empty * increase version * only add ImagePullPolicy if specified * Add a workflow to automated dependencies update * fix the instruction of the end to end demo and the rspi4 demo. (#217) * Add comment for cargo-audit action behavior * use akri bot PAT for the autoupdate workflow * update token secret name * Patch anomaly detection app (#229) * close connection to grpc service each loop * update version * use local subchannel pool * fix title typo * Add logo to README [SAME VERSION] (#220) * Auto-updated dependencies * Auto-updated dependencies * Allow unreachable code introduced with feature-driven protocols [SAME VERSION] (#232) * Allow unreachable code introduced with feature-driven protocols [SAME VERSION] The catchall match is there in case someone turns off one of the handlers via features ... but at runtime the disabled feature is configured. * update for cargo fmt * add comment explanation * fix cargo fmt * Improve development experience with timestamped container labels (#234) Using labels based on version.txt can make it difficult to iterate during development. Creating timestamped labels for the local build experience should make it easier to differentiate between development builds. * Reduce frequency of dependency update [SAME VERSION] (#236) * Reduce frequency of dependency update Our dependencies update on a daily basis ... not sure it makes sense for us to update our version each time someone else's do. I chose a monthly cadence because it would, at most, cause us to update our version 12x per year, which doesn't seem too bad. * Update auto-update-dependencies.yml * update code owners * Lazy eval of makefile timestamp breaks local build [SAME VERSION] [IGNORE INTERMEDIATE BUILDS] (#242) * Update hyper crate for security risk * patch version update * force override kubernetes package to deal with installation errors * Fix typo * Simplify matrix; Use all versions 1.16+ for all distros (#244) * Simplify matrix; Use all versions 1.16+ for all distros * Remove non-existent webhook test * Remove `microk8s.enable helm3` appears redundant and fails * Link Kubernetes and crictl for K3s|MicroK8s * Per: #206 (comment) * Typo * fix matrix and crictl path * Consistency and double-quote to expand `${PWD}` * Temporarily (!) block K3s 1.16 due to hard-coded device plugs (k3s-io/k3s#1390) * Test mapping K3s 1.16 device plugins to default K8s location * Create kubectl path in expected location if not present * Consistent w/ MicroK8s step * Wait until cluster has stabilized before wrangling device-plugins * Initial documentation for `run-test-cases.yml` * Correct indentation Co-authored-by: bfjelds <[email protected]> * add microk8s inspect to installation * try waiting repeatedly for microk8s to be ready * Cleanup streaming app sample (#173) * Initial version * Create logical classes * Fixes and optimizations * Use naming guideline for method names * Version bump * Version bump Co-authored-by: Brian Fjeldstad <[email protected]> * Admission Controller (Webhook) for Akri Configuration(s) (#206) * Helm Chart updates * Add 'webhooks' to the project * Commented Helm Chart `webhook-configuration` * Template `webhook-configuration` * Webhook service * Dockerfile (draft) * GitHub Action (draft) * Force build * Trying again * And again * Added GHCR secret to fork * Grrr * Grrr * Issue w/ GitHub Action build * Undo * Consistency * Revised Rust Webhook to not require these * Preliminary README * Update `version.sh` and run `./version.sh -u -s` * Added `webhook-configuration` * Does this resolve Helm Linting concern? * r/AMD32V7/AMR32V7 * Are these required? * Follow pattern * Follow Akri pattern * r/webhook-configuration/webhookConfiguration * correct path references * Cross-build Rust * Exceptions no longer required * Assume `./target/release/webhook-configuration` * Expose x-plat targets to docker * Omission * Improve handling of flags * Tidy * Correct indendation * Version bump * Update `CHANGELOG` * `cargo.lock` versions now bumped too * Temporarily (!) bumping build timeouts * Corrected component reference * Missed `build-arg=CROSS_BUILD_TARGET=...` * Avoid Helm issue with un-`named` `Lists` * Revert `CHANGELOG` * Added unit tests * Filters Akri Configuration manifest so that it validates * Added test for filtered `generation` * Added test for extended (complete) Config * To avoid GitHub Actions build failure * End-to-End Webhook tests * Integration tests require `actix-rt` * Include `webhook-configuration` container image * Learns all the things! * Typo * Added `webhook` to `test-case`s * Removed `get_agent_and_controller` reference * Simplified Actions `matrix`; Updated Webhook certs * Override `webhookConfiguration.image.tag` to `pr-amd64` * Add debugging * Typo * sudo'ing `kubectl` commands * Try subprocess; reduce test matrix * Add `shell=True` * More `describe`s * Correct `kubectl --selector` commands * Reflect r/ENTRYPOINT/CMD in Dockerfile * Try matrix * Debugging Kubenernetes 1.16 * Additional MicroK8s 1.16+1.17 and debugging * Generate: `*-webhook-log` from GitHub Actions * Add some debugging statements * K8s 1.16 expects AdmissionReview to be v1beta1 * Checking K3s 1.16+1.17 * Ordering may be significant * Reverted * Don't delete CRDs; Remove redundant debugging * Limit to distro-versions known to work for E2Es * Bumped OpenAPI-generated sources due to Akri security audit issues * Documents `run-webook.py` * Typo * `expect` only when providing useful message * fix Makefile formatting for image push [SAME VERSION] * add updates to lock file fir admission controller webhook dependencies (#254) * proposal for logging standards * Fix typo * Simplify containers makefile (#256) * simplify makefile for containers * add for all containers * fix indent * bump version * try to fix indentation * fix manifest creation error (#257) * Fix onvif * changes to the particular makefiles should invoke container workflow (#258) * Add more docs for akri build (#260) * add docs for akri build * address feedback * add local instructions * fix mistakes * Auto-updated dependencies * Allow dependencies auto update to be manually run * Auto-updated dependencies * increase timeout for tarpaulin and check-rust workflows (#266) * Increase docker create timeout to 600 minutes and document this hidden timeout (#267) * Try same env var for action and container timeout * output script to verify * try reusing timeout-minutes directly * Try moving timeout to step and access that directly * try fixing env var creation * try using env var again https://docs.github.com/en/actions/reference/context-and-expression-syntax-for-github-actions#job-context suggests timeout-minutes can be set from env * Use step.timeout-minutes because it can pull from env var * Update run-tarpaulin.yml * just make the docker keep alive script sleep 600 minutes * remove env var * fix typo * typos, typos, typos * create certs for webhook e2e tests with longer lasting CA (#271) * Enable Discovery Handlers as Pods (#252) * Move security audit checks to a daily schedule instead of on PRs * Fix typo * Link to specific github run url from github context * rename rust jobs * remove unnecessary job * only run on main and not forks * Update security-audit.yml Restrict the action to main repo and not forks * Serialize Configuration Properties for Webhook (#276) * don't skip serializing configuration properties * increase version * rust fmt fixes * Try to ONLY build rust once per platform (#270) * build rust once/platform, share with artifacts * timeout => timeout-minutes * trigger when build-rust-code changes * only run container builds after successful rust build * install rust toolchain to build rust * add crossbuild qemu * reduce retention to minimum * make test timeout longer * update version * Fix artifact name [SAME VERSION] (#280) * fix artifact name [SAME VERSION] * only run for main * Add Workflow to Cancel Previous PR Workflows [SAME VERSION] (#281) * add workflow to cancel all previous workflows on a pr * remove security workflow and cancel workflow from list * Build both slim and full Agents [IGNORE INTERMEDIATE BUILDS] (#279) * specify component name in agent multiarch build (#285) * Build discovery handler images [SAME VERSION] (#286) * Enable Canceling Previous Workflows on PRs from Forks [SAME VERSION] (#284) * use workflow_run trigger in cancel workflow * only run cancel workflow explicitly for pull_request events * Increase e2e test timeout [SAME VERSION] (#287) * increase e2e test timeout * increase timeout further * Ensure Discovery Handler is marked as available again after Configuration deletions (#282) * Make sure `shared` environment variable is set in debug echo Discovery Handler (#289) * Overwrite latest tag when user specifies a tag (#293) * overwrite latest tag when user supplies tag * increase version * Auto-updated dependencies * Reference correct Agent features for embedding Discovery Handlers [IGNORE INTERMEDIATE BUILDS] (#295) * Debug echo properties (#288) * Add Documentation on using Debug Echo [SAME VERSION] (#296) * make sure webhook-configuration version updated in lock file (#300) * Remove un-utilized CRD field (#290) * remove rbac property from Instance CRD * bump minor version to 0.5.0 for breaking change * Rename CRD fields to be more descriptive (#297) * change Configuration.properties to Configuration.brokerProperties * change Instance.metadata to Instance.brokerProperties * increase minor version due to CRD breaking change * fix errors in user guide (#302) * Remove udev directory mount from Agent (#304) * Extend documentation (#303) * Documentation fixes (#307) A few simple fixes to the Debug Echo documentation. Signed-off-by: Hernan Gatta <[email protected]> * fix opc ua configuration table in docs (#309) * use sudo when getting logs in e2e tests (#312) * suggest background reading before discovery handler development (#310) * Update cluster setup documentation (#308) * lint helm chart with values (#306) * Add documentation on setting a Configuration's `brokerProperties` (#311) * Fix end-to-end tests (#301) * Reduce K3s and MicroK8s eviction limits for tests [SAME VERSION] (#313) * reduce K3s eviction limits for tests * reduce MicroK8s limits * Rename configuration templates (#315) * rename configuration templates * increase version * Set compute resource limits and requests for all Akri components (#305) * Extensibility example fixes [SAME VERSION] (#321) * Use latest Rust and Tarpaulin (#318) * use latest tarpaulin in workflow * use latest rust in workflows * increase version * use previous tarpaulin version due to segfault issue * Increase dependency minor versions where possible [SAME VERSION] (#323) * increase cfg-if, env_logger, mockall_double, prometheus, udev, opcua-client, and rand crate minor versions * increase version * extend sleep in test case workflow to 45 minutes for component builds to complete (#326) * Test e2e tests on latest K8s version [SAME VERSION] (#317) * reference Akri's release Helm chart in documentation (#325) * Update CHANGELOG for v0.6.5 release (#319) * [Extensibility] HTTP protocol (branch: http-extensibility) (#135) * Initial commit * Working * Correct errors & revise for Device|Discovery v2 * Working * Should (!) work * Typo * Enrich * Refine * Fixes: #102 * Moves * Resolved * Revised for `dnsPolicy` * Dockerfiles * Added gRPC solution * Replaced * Kubernetes specs * Merge * Correct Kubernetes config filenames * Typo * Removed duplicate * Standalone broker exposes no ports * end-to-end-demo-gce.md: fixing 1 typo (#131) Hi, just removing the superfluous 's' Didier * link to codecov.io and use codecov action (#134) * Update end-to-end-demo.md (#112) * Update end-to-end-demo.md Hi there I extended the documentatio with the installation of linux-modules-extra because I had to add this package on Ubuntu 20.04 LTS (on a GCE instance): it seems that it doesn't come by default on distros / images aimed at backend servers. v4l loopback won't load if this other module is not present. Didier * Update end-to-end-demo.md fixing initial text according to revision notes in PR review * Update end-to-end-demo.md fixing style * Update end-to-end-demo.md simplifying text, based on PR review by Kate * rebase to finalize PR * Fix codecov for pull_request_target [SAME VERSION] (#138) * fix codecov for pull_request_target * add anti-pull-request-* conditional * Do not upload to codecov on PRs (#140) * end-to-end-demo-rpi4: fixing 1 typo (#143) Hello, Title says it all Didier * end-to-end demo : fixing 1 typo (#141) Hi there, The title says it all Didier * Minor e2e demo doc flow change (#139) * Minor flow change * Cleanup * Expand udev rule support (#130) * add support for SUBSYSTEMS, KERNELS, ATTRIBUTES, DRIVERS, and TAGS fields * fix formatting and add comments * add recursive search for subsystems * update udev configuration document with newly supported fields * check child device for field before looking up hierarchy * update version * document how to filter for only devices with devnodes when testing * reuse shared code * rust fmt fixes * add inline documentation for regex matching * Update end-to-end-demo.md (#113) Hi there, I added a comment regarding the importance of the version of v4l2loopback to avoid my personal initial mistake to others: I did not realize at start that the version was so important. So I went through this ticket with the dev: umlaeute/v4l2loopback#370 Didier Co-authored-by: bfjelds <[email protected]> * Add sudo into udev configuration doc (#144) * Use arm7 * Reference arm7 * Reference branch on clone * `REPO="akri"` * Helm template for Agent already revised * Consistency * Use `./version.txt` for Akri version * Automate IMAGE updates * Build|Push gRPC images; Add'l consistency * Add Action to build HTTP Broker containers * I guess `http-extensibility` not `main` * Enable PRs to test changes to helm charts [SAME VERSION] (#146) * enable PRs to test changes to helm charts * handle manual job, fix push and release conditional * trigger e2e for helm changes * Updates references to `armv7` * Removed `shared` from HTTP Broker Dockerfiles * Correct version reference for image tags * Typo * Adds Golang Device|Discovery and alt-gRPC * s/println!/trace! * Consistency * Reflect relocation of apps * Simplify environment * Typo * Consistency on repo names for HTTP apps * Rename HTTP apps gRPC images * GitHub Actions to build HTTP apps containers * Configure broker security context (#148) * make udev broker security context configurable * update version * change group permissions to be default in example * fix comment with group id * Remove privileged security context from Akri Agent (#147) * Remove (redundant!?) aliasing * Remove healthchecks (`/healthz`) * Removed `akri-shared` dep * Simplified using `strategy.matrix` * Revised to use `strategy.matrix` * reference macro using fully-qualified module name * r|Microsoft|deislabs/akri * Remove unnecessary include * Add precision on arch being built * Consistent image naming and v1 * Remove `/v2/` references * Explantion of different broker implementations * Cleanup Workflows [IGNORE INTERMEDIATE BUILDS][SAME VERSION] (#155) * don't persist github creds and remove setup.sh from workflows * Update CHANGELOG for release v0.0.44 (#153) * Update CHANGELOG for release v0.0.44 * update release to better describe helm changes and supported flavors * add note about known k3s incompat * Pin CHANGELOG changes to the expected label * Fix CHANGELOG typo * Optimize build and test all supported versions (#157) * build once, test all supported versions * test step needs build step * fix copy-paste error * use startsWith instead of '==' to check kube-runtime * for download, specifying path introduces an unwanted directory * load the artifact containers into docker for Kubernetes * microk8s ctr images import is mysteriously failing with 'context deadline exceeded' * avoid k3s' crictl, add more testing versions for kubernetes & k3s * allow downgrades for kubernetes install, try to understand microk8s image import failure * swallow the return code of version checks * use persist-creds=false * disable microk8s 1.19 for now * microk8s 1.19 requires small sleep before it is ready for ctr requests * to ensure sequential running, we cannot skip the build-containers job (just its steps) (#159) * Update user guide with first release tag for Akri release chart (#160) * update user guide with first release tag for akri release chart * reference akri release chart instead of dev chart in docs * Remove pull_request_target triggers and minimize credential use [SAME VERSION] [IGNORE INTERMEDIATE BUILDS] (#161) * remove dockerhub credentials from workflows * add link to repo to intermediate containers * remove pull_request_target * only log into docker when pushing changes to ghcr * Typos * Added existing "Extensibility" preamble * Tidy * Replace "Extensibility" * attempt to merge extensibility.md i focused on the standalone to cut down on the size (the file is still 800+ lines) with a thought towards linking to some md files in http-extensibility branch for more details (on the rust grpc and golang grpc). * tighten up to reduce size combine device and discovery into one http server/pod, remove some comments from quoted code * provide a few more explanations * fix typo * Use matrix for build-http-broker-containers.yaml * Replace backup md file with extensibility-http-grpc.md Create document for deeper dive into gRPC broker/client code. This can be referenced from extensibility.md. * update extensibility.md to be consistent 'HTTP' * create links to http-extensibility branch * after walkthrough, make a couple fixes * use different curl container busyboxplus seems to have a dns problem that manifests on k3s * remove workflows for http-extensibility * remove new workflows in extensibility branch Co-authored-by: Didier Durand <[email protected]> Co-authored-by: bfjelds <[email protected]> Co-authored-by: Jiri Appl <[email protected]> Co-authored-by: Kate Goldenring <[email protected]> * add http discovery handler to the discovery handler modules folder Co-authored-by: Didier Durand <[email protected]> Co-authored-by: Jiayi Hu <[email protected]> Co-authored-by: Jiayi Hu <[email protected]> Co-authored-by: Roaa <[email protected]> Co-authored-by: Daz Wilkin <[email protected]> Co-authored-by: Ronan Flynn-Curran <[email protected]> Co-authored-by: romoh <[email protected]> Co-authored-by: Chao Zheng <[email protected]> Co-authored-by: bfjelds <[email protected]> Co-authored-by: Jiri Appl <[email protected]> Co-authored-by: Hernan Gatta <[email protected]>
* test latest k8s versions [SAME VERSION] (project-akri#188) * update changelog for 0.1.5 release (project-akri#189) * add more description and simplify commands in e2e demo (project-akri#191) * Various typos fixed on new OPCUA documentation (project-akri#195) * Various typos fixed on new OPCUA documentation Signed-off-by: Didier Durand <[email protected]> * Fixes on extensibility documentation Signed-off-by: Didier Durand <[email protected]> * Make end-to-end demo on Raspberry Pi simpler and more descriptive (project-akri#193) * mirror updates in e2e demo in rasp-pi demo * syntax changes, capacity specification, building kernel module * fix syntax issues * Get Agent logs before testing their contents [SAME VERSION] (project-akri#197) * add additional checks to see why getting logs failing * log information when get crictl error * use os.system to log info * try using shell with subprocess when doing complex commands * Fix failing to query container runtime in test workflow on MicroK8s [SAME VERSION] (project-akri#202) * loop query containerd several times instead of sleeping * shorten agent log query * Allow overwriting controller.nodeSelectors (project-akri#194) * Allow overwriting controller.nodeSelectors * Add note about `sed` for MacOS contributors * Allow conditional compilation of agent protocols (project-akri#196) * Allow conditional compilation of agent protocols * Bump patch version Co-authored-by: Jiayi Hu <[email protected]> * Prometheus (project-akri#190) * add frame count metric to udev broker * add prometheus metrics * format metrics.rs * update Helm charts for Prometheus support * Prometheus documentation * increase supported rust version to latest * fix metrics chart * use PodMonitor instead of ServiceMonitor for Akri Agent and Controller * update tarpaulin to latest version to fix protobuf issue * remove debug echo shared setting * update version * Use anyhow instead of the deprecated failure crate (project-akri#201) * Switch to use anyhow crate instead of failure crate Include Cargo.lock updates * Rebase and increment patch version * Merge conflict * Add missing version updates after rebase * [Helm] Added CRD variables (project-akri#205) * Added CRD variables * `version.sh` updates Helm `crds.version` * Bump version * Document naming convention (project-akri#204) * Initial commit * Remove references to `http` example * Added `udev-broker` * Added `akri-udev` * Rephrased * Typo * naming-guidelines >> development.md * r/udev-broker/udev-video-broker * punctuation * r/Resources/resources * r/Compute/Computer (project-akri#210) * fix instructions for cloning and building v4l2 (project-akri#209) * Update hyper version and other dependencies * Update patch version * Explicitly specify h2 patch version dependency to avoid future accidentl updates by cargo-update or similar * Remove redundant unnecessary hyper dependencies * Add security audit workflow * Move security audit to existing check-rust workflow * Simplify mockall usage (project-akri#214) * Simplify mockall usage * Bump patch version * add project logo and art [SAME VERSION] (project-akri#149) * add akri branding and artwork Signed-off-by: flynnduism <[email protected]> * edit artwork README Signed-off-by: flynnduism <[email protected]> * re-render dark logos Signed-off-by: flynnduism <[email protected]> * redraw the svg icon to fix glitchy render Signed-off-by: flynnduism <[email protected]> * update logo.svg Signed-off-by: flynnduism <[email protected]> * fix the mixed up color hex values Signed-off-by: flynnduism <[email protected]> * update branding with inconsolata typeface Signed-off-by: flynnduism <[email protected]> * use correct casing for mentions of Akri Signed-off-by: flynnduism <[email protected]> * remove default documentation to use latest containers (project-akri#218) * Image pull policy (project-akri#207) * change default image pull policy to IfNotPresent * set ImagePullPolicy to default to empty * increase version * only add ImagePullPolicy if specified * Add a workflow to automated dependencies update * fix the instruction of the end to end demo and the rspi4 demo. (project-akri#217) * Add comment for cargo-audit action behavior * use akri bot PAT for the autoupdate workflow * update token secret name * Patch anomaly detection app (project-akri#229) * close connection to grpc service each loop * update version * use local subchannel pool * fix title typo * Add logo to README [SAME VERSION] (project-akri#220) * Auto-updated dependencies * Auto-updated dependencies * Allow unreachable code introduced with feature-driven protocols [SAME VERSION] (project-akri#232) * Allow unreachable code introduced with feature-driven protocols [SAME VERSION] The catchall match is there in case someone turns off one of the handlers via features ... but at runtime the disabled feature is configured. * update for cargo fmt * add comment explanation * fix cargo fmt * Improve development experience with timestamped container labels (project-akri#234) Using labels based on version.txt can make it difficult to iterate during development. Creating timestamped labels for the local build experience should make it easier to differentiate between development builds. * Reduce frequency of dependency update [SAME VERSION] (project-akri#236) * Reduce frequency of dependency update Our dependencies update on a daily basis ... not sure it makes sense for us to update our version each time someone else's do. I chose a monthly cadence because it would, at most, cause us to update our version 12x per year, which doesn't seem too bad. * Update auto-update-dependencies.yml * update code owners * Lazy eval of makefile timestamp breaks local build [SAME VERSION] [IGNORE INTERMEDIATE BUILDS] (project-akri#242) * Update hyper crate for security risk * patch version update * force override kubernetes package to deal with installation errors * Fix typo * Simplify matrix; Use all versions 1.16+ for all distros (project-akri#244) * Simplify matrix; Use all versions 1.16+ for all distros * Remove non-existent webhook test * Remove `microk8s.enable helm3` appears redundant and fails * Link Kubernetes and crictl for K3s|MicroK8s * Per: project-akri#206 (comment) * Typo * fix matrix and crictl path * Consistency and double-quote to expand `${PWD}` * Temporarily (!) block K3s 1.16 due to hard-coded device plugs (k3s-io/k3s#1390) * Test mapping K3s 1.16 device plugins to default K8s location * Create kubectl path in expected location if not present * Consistent w/ MicroK8s step * Wait until cluster has stabilized before wrangling device-plugins * Initial documentation for `run-test-cases.yml` * Correct indentation Co-authored-by: bfjelds <[email protected]> * add microk8s inspect to installation * try waiting repeatedly for microk8s to be ready * Cleanup streaming app sample (project-akri#173) * Initial version * Create logical classes * Fixes and optimizations * Use naming guideline for method names * Version bump * Version bump Co-authored-by: Brian Fjeldstad <[email protected]> * Admission Controller (Webhook) for Akri Configuration(s) (project-akri#206) * Helm Chart updates * Add 'webhooks' to the project * Commented Helm Chart `webhook-configuration` * Template `webhook-configuration` * Webhook service * Dockerfile (draft) * GitHub Action (draft) * Force build * Trying again * And again * Added GHCR secret to fork * Grrr * Grrr * Issue w/ GitHub Action build * Undo * Consistency * Revised Rust Webhook to not require these * Preliminary README * Update `version.sh` and run `./version.sh -u -s` * Added `webhook-configuration` * Does this resolve Helm Linting concern? * r/AMD32V7/AMR32V7 * Are these required? * Follow pattern * Follow Akri pattern * r/webhook-configuration/webhookConfiguration * correct path references * Cross-build Rust * Exceptions no longer required * Assume `./target/release/webhook-configuration` * Expose x-plat targets to docker * Omission * Improve handling of flags * Tidy * Correct indendation * Version bump * Update `CHANGELOG` * `cargo.lock` versions now bumped too * Temporarily (!) bumping build timeouts * Corrected component reference * Missed `build-arg=CROSS_BUILD_TARGET=...` * Avoid Helm issue with un-`named` `Lists` * Revert `CHANGELOG` * Added unit tests * Filters Akri Configuration manifest so that it validates * Added test for filtered `generation` * Added test for extended (complete) Config * To avoid GitHub Actions build failure * End-to-End Webhook tests * Integration tests require `actix-rt` * Include `webhook-configuration` container image * Learns all the things! * Typo * Added `webhook` to `test-case`s * Removed `get_agent_and_controller` reference * Simplified Actions `matrix`; Updated Webhook certs * Override `webhookConfiguration.image.tag` to `pr-amd64` * Add debugging * Typo * sudo'ing `kubectl` commands * Try subprocess; reduce test matrix * Add `shell=True` * More `describe`s * Correct `kubectl --selector` commands * Reflect r/ENTRYPOINT/CMD in Dockerfile * Try matrix * Debugging Kubenernetes 1.16 * Additional MicroK8s 1.16+1.17 and debugging * Generate: `*-webhook-log` from GitHub Actions * Add some debugging statements * K8s 1.16 expects AdmissionReview to be v1beta1 * Checking K3s 1.16+1.17 * Ordering may be significant * Reverted * Don't delete CRDs; Remove redundant debugging * Limit to distro-versions known to work for E2Es * Bumped OpenAPI-generated sources due to Akri security audit issues * Documents `run-webook.py` * Typo * `expect` only when providing useful message * fix Makefile formatting for image push [SAME VERSION] * add updates to lock file fir admission controller webhook dependencies (project-akri#254) * proposal for logging standards * Fix typo * Simplify containers makefile (project-akri#256) * simplify makefile for containers * add for all containers * fix indent * bump version * try to fix indentation * fix manifest creation error (project-akri#257) * Fix onvif * changes to the particular makefiles should invoke container workflow (project-akri#258) * Add more docs for akri build (project-akri#260) * add docs for akri build * address feedback * add local instructions * fix mistakes * Auto-updated dependencies * Allow dependencies auto update to be manually run * Auto-updated dependencies * increase timeout for tarpaulin and check-rust workflows (project-akri#266) * Increase docker create timeout to 600 minutes and document this hidden timeout (project-akri#267) * Try same env var for action and container timeout * output script to verify * try reusing timeout-minutes directly * Try moving timeout to step and access that directly * try fixing env var creation * try using env var again https://docs.github.com/en/actions/reference/context-and-expression-syntax-for-github-actions#job-context suggests timeout-minutes can be set from env * Use step.timeout-minutes because it can pull from env var * Update run-tarpaulin.yml * just make the docker keep alive script sleep 600 minutes * remove env var * fix typo * typos, typos, typos * create certs for webhook e2e tests with longer lasting CA (project-akri#271) * Enable Discovery Handlers as Pods (project-akri#252) * Move security audit checks to a daily schedule instead of on PRs * Fix typo * Link to specific github run url from github context * rename rust jobs * remove unnecessary job * only run on main and not forks * Update security-audit.yml Restrict the action to main repo and not forks * Serialize Configuration Properties for Webhook (project-akri#276) * don't skip serializing configuration properties * increase version * rust fmt fixes * Try to ONLY build rust once per platform (project-akri#270) * build rust once/platform, share with artifacts * timeout => timeout-minutes * trigger when build-rust-code changes * only run container builds after successful rust build * install rust toolchain to build rust * add crossbuild qemu * reduce retention to minimum * make test timeout longer * update version * Fix artifact name [SAME VERSION] (project-akri#280) * fix artifact name [SAME VERSION] * only run for main * Add Workflow to Cancel Previous PR Workflows [SAME VERSION] (project-akri#281) * add workflow to cancel all previous workflows on a pr * remove security workflow and cancel workflow from list * Build both slim and full Agents [IGNORE INTERMEDIATE BUILDS] (project-akri#279) * specify component name in agent multiarch build (project-akri#285) * Build discovery handler images [SAME VERSION] (project-akri#286) * Enable Canceling Previous Workflows on PRs from Forks [SAME VERSION] (project-akri#284) * use workflow_run trigger in cancel workflow * only run cancel workflow explicitly for pull_request events * Increase e2e test timeout [SAME VERSION] (project-akri#287) * increase e2e test timeout * increase timeout further * Ensure Discovery Handler is marked as available again after Configuration deletions (project-akri#282) * Make sure `shared` environment variable is set in debug echo Discovery Handler (project-akri#289) * Overwrite latest tag when user specifies a tag (project-akri#293) * overwrite latest tag when user supplies tag * increase version * Auto-updated dependencies * Reference correct Agent features for embedding Discovery Handlers [IGNORE INTERMEDIATE BUILDS] (project-akri#295) * Debug echo properties (project-akri#288) * Add Documentation on using Debug Echo [SAME VERSION] (project-akri#296) * make sure webhook-configuration version updated in lock file (project-akri#300) * Remove un-utilized CRD field (project-akri#290) * remove rbac property from Instance CRD * bump minor version to 0.5.0 for breaking change * Rename CRD fields to be more descriptive (project-akri#297) * change Configuration.properties to Configuration.brokerProperties * change Instance.metadata to Instance.brokerProperties * increase minor version due to CRD breaking change * fix errors in user guide (project-akri#302) * Remove udev directory mount from Agent (project-akri#304) * Extend documentation (project-akri#303) * Documentation fixes (project-akri#307) A few simple fixes to the Debug Echo documentation. Signed-off-by: Hernan Gatta <[email protected]> * fix opc ua configuration table in docs (project-akri#309) * use sudo when getting logs in e2e tests (project-akri#312) * suggest background reading before discovery handler development (project-akri#310) * Update cluster setup documentation (project-akri#308) * lint helm chart with values (project-akri#306) * Add documentation on setting a Configuration's `brokerProperties` (project-akri#311) * Fix end-to-end tests (project-akri#301) * Reduce K3s and MicroK8s eviction limits for tests [SAME VERSION] (project-akri#313) * reduce K3s eviction limits for tests * reduce MicroK8s limits * Rename configuration templates (project-akri#315) * rename configuration templates * increase version * Set compute resource limits and requests for all Akri components (project-akri#305) * Extensibility example fixes [SAME VERSION] (project-akri#321) * Use latest Rust and Tarpaulin (project-akri#318) * use latest tarpaulin in workflow * use latest rust in workflows * increase version * use previous tarpaulin version due to segfault issue * Increase dependency minor versions where possible [SAME VERSION] (project-akri#323) * increase cfg-if, env_logger, mockall_double, prometheus, udev, opcua-client, and rand crate minor versions * increase version * extend sleep in test case workflow to 45 minutes for component builds to complete (project-akri#326) * Test e2e tests on latest K8s version [SAME VERSION] (project-akri#317) * reference Akri's release Helm chart in documentation (project-akri#325) * Update CHANGELOG for v0.6.5 release (project-akri#319) * [Extensibility] HTTP protocol (branch: http-extensibility) (project-akri#135) * Initial commit * Working * Correct errors & revise for Device|Discovery v2 * Working * Should (!) work * Typo * Enrich * Refine * Fixes: project-akri#102 * Moves * Resolved * Revised for `dnsPolicy` * Dockerfiles * Added gRPC solution * Replaced * Kubernetes specs * Merge * Correct Kubernetes config filenames * Typo * Removed duplicate * Standalone broker exposes no ports * end-to-end-demo-gce.md: fixing 1 typo (project-akri#131) Hi, just removing the superfluous 's' Didier * link to codecov.io and use codecov action (project-akri#134) * Update end-to-end-demo.md (project-akri#112) * Update end-to-end-demo.md Hi there I extended the documentatio with the installation of linux-modules-extra because I had to add this package on Ubuntu 20.04 LTS (on a GCE instance): it seems that it doesn't come by default on distros / images aimed at backend servers. v4l loopback won't load if this other module is not present. Didier * Update end-to-end-demo.md fixing initial text according to revision notes in PR review * Update end-to-end-demo.md fixing style * Update end-to-end-demo.md simplifying text, based on PR review by Kate * rebase to finalize PR * Fix codecov for pull_request_target [SAME VERSION] (project-akri#138) * fix codecov for pull_request_target * add anti-pull-request-* conditional * Do not upload to codecov on PRs (project-akri#140) * end-to-end-demo-rpi4: fixing 1 typo (project-akri#143) Hello, Title says it all Didier * end-to-end demo : fixing 1 typo (project-akri#141) Hi there, The title says it all Didier * Minor e2e demo doc flow change (project-akri#139) * Minor flow change * Cleanup * Expand udev rule support (project-akri#130) * add support for SUBSYSTEMS, KERNELS, ATTRIBUTES, DRIVERS, and TAGS fields * fix formatting and add comments * add recursive search for subsystems * update udev configuration document with newly supported fields * check child device for field before looking up hierarchy * update version * document how to filter for only devices with devnodes when testing * reuse shared code * rust fmt fixes * add inline documentation for regex matching * Update end-to-end-demo.md (project-akri#113) Hi there, I added a comment regarding the importance of the version of v4l2loopback to avoid my personal initial mistake to others: I did not realize at start that the version was so important. So I went through this ticket with the dev: umlaeute/v4l2loopback#370 Didier Co-authored-by: bfjelds <[email protected]> * Add sudo into udev configuration doc (project-akri#144) * Use arm7 * Reference arm7 * Reference branch on clone * `REPO="akri"` * Helm template for Agent already revised * Consistency * Use `./version.txt` for Akri version * Automate IMAGE updates * Build|Push gRPC images; Add'l consistency * Add Action to build HTTP Broker containers * I guess `http-extensibility` not `main` * Enable PRs to test changes to helm charts [SAME VERSION] (project-akri#146) * enable PRs to test changes to helm charts * handle manual job, fix push and release conditional * trigger e2e for helm changes * Updates references to `armv7` * Removed `shared` from HTTP Broker Dockerfiles * Correct version reference for image tags * Typo * Adds Golang Device|Discovery and alt-gRPC * s/println!/trace! * Consistency * Reflect relocation of apps * Simplify environment * Typo * Consistency on repo names for HTTP apps * Rename HTTP apps gRPC images * GitHub Actions to build HTTP apps containers * Configure broker security context (project-akri#148) * make udev broker security context configurable * update version * change group permissions to be default in example * fix comment with group id * Remove privileged security context from Akri Agent (project-akri#147) * Remove (redundant!?) aliasing * Remove healthchecks (`/healthz`) * Removed `akri-shared` dep * Simplified using `strategy.matrix` * Revised to use `strategy.matrix` * reference macro using fully-qualified module name * r|Microsoft|deislabs/akri * Remove unnecessary include * Add precision on arch being built * Consistent image naming and v1 * Remove `/v2/` references * Explantion of different broker implementations * Cleanup Workflows [IGNORE INTERMEDIATE BUILDS][SAME VERSION] (project-akri#155) * don't persist github creds and remove setup.sh from workflows * Update CHANGELOG for release v0.0.44 (project-akri#153) * Update CHANGELOG for release v0.0.44 * update release to better describe helm changes and supported flavors * add note about known k3s incompat * Pin CHANGELOG changes to the expected label * Fix CHANGELOG typo * Optimize build and test all supported versions (project-akri#157) * build once, test all supported versions * test step needs build step * fix copy-paste error * use startsWith instead of '==' to check kube-runtime * for download, specifying path introduces an unwanted directory * load the artifact containers into docker for Kubernetes * microk8s ctr images import is mysteriously failing with 'context deadline exceeded' * avoid k3s' crictl, add more testing versions for kubernetes & k3s * allow downgrades for kubernetes install, try to understand microk8s image import failure * swallow the return code of version checks * use persist-creds=false * disable microk8s 1.19 for now * microk8s 1.19 requires small sleep before it is ready for ctr requests * to ensure sequential running, we cannot skip the build-containers job (just its steps) (project-akri#159) * Update user guide with first release tag for Akri release chart (project-akri#160) * update user guide with first release tag for akri release chart * reference akri release chart instead of dev chart in docs * Remove pull_request_target triggers and minimize credential use [SAME VERSION] [IGNORE INTERMEDIATE BUILDS] (project-akri#161) * remove dockerhub credentials from workflows * add link to repo to intermediate containers * remove pull_request_target * only log into docker when pushing changes to ghcr * Typos * Added existing "Extensibility" preamble * Tidy * Replace "Extensibility" * attempt to merge extensibility.md i focused on the standalone to cut down on the size (the file is still 800+ lines) with a thought towards linking to some md files in http-extensibility branch for more details (on the rust grpc and golang grpc). * tighten up to reduce size combine device and discovery into one http server/pod, remove some comments from quoted code * provide a few more explanations * fix typo * Use matrix for build-http-broker-containers.yaml * Replace backup md file with extensibility-http-grpc.md Create document for deeper dive into gRPC broker/client code. This can be referenced from extensibility.md. * update extensibility.md to be consistent 'HTTP' * create links to http-extensibility branch * after walkthrough, make a couple fixes * use different curl container busyboxplus seems to have a dns problem that manifests on k3s * remove workflows for http-extensibility * remove new workflows in extensibility branch Co-authored-by: Didier Durand <[email protected]> Co-authored-by: bfjelds <[email protected]> Co-authored-by: Jiri Appl <[email protected]> Co-authored-by: Kate Goldenring <[email protected]> * add http discovery handler to the discovery handler modules folder Co-authored-by: Didier Durand <[email protected]> Co-authored-by: Jiayi Hu <[email protected]> Co-authored-by: Jiayi Hu <[email protected]> Co-authored-by: Roaa <[email protected]> Co-authored-by: Daz Wilkin <[email protected]> Co-authored-by: Ronan Flynn-Curran <[email protected]> Co-authored-by: romoh <[email protected]> Co-authored-by: Chao Zheng <[email protected]> Co-authored-by: bfjelds <[email protected]> Co-authored-by: Jiri Appl <[email protected]> Co-authored-by: Hernan Gatta <[email protected]>
What this PR does / why we need it:
This is an initial draft to allow avoiding to compile all the deps when they are not actually needed by one's use case. This has two main advantages that comes to my mind:
The agent can't run locally on MacOS yet since it needs to be inside a k8s node, but this could be a first step towards that direction. I could maybe add a dumb agent that just calls the discovery handler in order to test the behaviour. In the worst case it's still helpful to have quick build errors instead of relying on
cross
for development.Once the protocol handler is locally tested, one can deploy to the cluster for better testing. Otherwise I can't even write unit tests because the cargo can't compile the dependencies without
libudev
and in future there might be other unique env dependencies based on the protocol.Let me know what you think about the idea 😄
Special notes for your reviewer:
If applicable:
cargo fmt
)cargo build
)cargo clippy
)cargo test
)cargo doc
)./version.sh
)