samples: Add a pod manifest for uvm test container #330
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This adds a GitHub Action workflow to check that the policy samples have been updated with each PR. We'll add it as a required check to catch policy errors earlier in the dev process. I also tweaked the update script to have better error handling, and I simplified the PR template to make it more readable. Signed-off-by: Aurélien Bombo <[email protected]>
ci: add check for policy samples
Allow all users to pull images by using containerd instead of having to worry about this tool potentially creating files that the caller of update_policy_samples.py doesn't own. Granting all users access to containerd is also consistent with the behavior of upstream Kata CI. If any users really need to execute genpolicy as root, hopefully executing this entire script as root is good enough for them. Signed-off-by: Dan Mihai <[email protected]>
…cket samples: don't execute genpolicy as root user
Update azure-file-secrets.yaml ans use better azureFile shareName Signed-off-by: Saul Paredes <[email protected]>
update docker sample annotation for latest image version Signed-off-by: Saul Paredes <[email protected]>
Save under /tmp the policy text when updating sample YAML files - just in case someone needs to read/debug these policy files. Signed-off-by: Dan Mihai <[email protected]>
Update deployment-docker-busybox.yaml policy to match the latest version of the busybox container. Signed-off-by: Dan Mihai <[email protected]>
samples: update docker sample and shareName
This branch starts introducing additional scripting to build, deploy and evaluate the components used in AKS' Pod Sandboxing and Confidential Containers preview features. This includes the capability to build the IGVM file and its reference measurement file for remote attestation. Signed-off-by: Manuel Huber <[email protected]>
tools: Add initial igvm-builder and node-builder/azure-linux scripting
samples: save decoded policy text
Enable genpolicy's dm-verity root hash cache when updating the sample files, to potentially achieve faster update policy script execution. Also, this is an additional opportunity to test multiple instances of genpolicy synchronizing their access to the cache files. Signed-off-by: Dan Mihai <[email protected]>
samples: use the dm-verity root hash cache
Update the sev parameter to sev_snp in the cloud-hypervisor.yaml file to match cloud-hypervisor 38.0.72. See https://github.com/microsoft/cloud-hypervisor/blob/msft/v38.0.72/vmm /src/config.rs#L687 Signed-off-by: Saul Paredes <[email protected]>
virtcontainers: update sev_snp param serialization
This change adds support for volume claim templates for stateful sets. This is useful for stateful sets that require persistent storage, using cc-azurefile-csi, cc-azuredisk-csi or other storage providers. Signed-off-by: Archana Choudhary <[email protected]>
This patch updates the samples to include the support for persistant storage for statefulsets. Signed-off-by: Archana Choudhary <[email protected]>
genpolicy: add persistent storage support for stateful sets
Add common images category Signed-off-by: Saul Paredes <[email protected]>
The two samples define Pod overhead values. When testing these samples on clusters where different podOverhead values are defined by the respective runtime classes, the tests fail as the values need to match, see: /plugin/pkg/admission/runtimeclass/admission.go in the kubernetes GitHub repository. Hence, removing the overhead definitions. Re-ran update_policy_samples.py. No change. Signed-off-by: Manuel Huber <[email protected]>
samples: add common images category
genpolicy: update policy samples
For docker-based builds only install Rust when necessary. Further, execute the detect Rust version check only when intending to install Rust. As of today, this is the case when we intend to build the agent during rootfs build. Signed-off-by: Manuel Huber <[email protected]>
We set the VERSION variable consistently across Makefiles to 'unknown' if the file is empty or not present. We also use git commands consistently for calculating the COMMIT, COMMIT_NO variables, not erroring out when building outside of a git repository. In create_summary_file we also account for a missing/empty VERSION file. This makes e.g. the UVM build process in an environment where we build outside of git with a minimal/reduced set of files smoother. Signed-off-by: Manuel Huber <[email protected]>
Avoid hitting docker.io by using mcr.microsoft.com/acc/samples/acc-perl:1.0. Signed-off-by: Dan Mihai <[email protected]>
samples: change job.yaml to an mcr container image
Add the new category for tests we want to run genpolicy against, but not run them in deployments (as the name suggests, we cannot run those). Signed-off-by: Manuel Huber <[email protected]>
samples: introduce incomplete_init category
Cherry-pick upstream PR kata-containers#9825: osbuilder: allow rootfs builds w/o git or version file deps
Using a symlink would create a cycle after calling this script again when copying the final configuration at line 74 so we just use cp instead. Also, I moved this block to the end of the file to properly override the final config file. Signed-off-by: Aurélien Bombo <[email protected]>
Instead, require the user to define a validation regex on the settings if not present and use that regex to validate in the rules. Signed-off-by: Saul Paredes <[email protected]>
Add regex validation in our settings for variables in our samples that need it. Signed-off-by: Saul Paredes <[email protected]>
Update samples Signed-off-by: Saul Paredes <[email protected]>
agent: Fix race condition with cgroup watchers
genpolicy: fix env variables that are always allowed
JOB_COMPLETION_INDEX might be empty, so the regex should be changed to match the empty string. Signed-off-by: Saul Paredes <[email protected]>
Update samples Signed-off-by: Saul Paredes <[email protected]>
tests: fix regex used in settings for testing genpolicy
The storage destination is already validated in allow_mount_point(). Signed-off-by: Aurélien Bombo <[email protected]>
After validating the storage source and destination, validate that the mount source == storage destination. Signed-off-by: Aurélien Bombo <[email protected]>
Trivial validation. Signed-off-by: Aurélien Bombo <[email protected]>
Update samples after improving storage validation. Signed-off-by: Aurélien Bombo <[email protected]>
genpolicy: Harden storage validation
PolicyCreateContainerRequest will be a wrapper to CreateContainerRequest and will allow the agent to transform the input such that is easier to validate in the policy. Signed-off-by: Saul Paredes <[email protected]>
- Add environment variable map to the engine on the agent - Add environment variable map to the policy on genpolicy - Validate this environemnt variable map on the rules - Restrain some environment variable values that we expect to look like a subdomain and use a regex to validate them Signed-off-by: Saul Paredes <[email protected]>
- Mark old arg validation methods as deprecated - Mark old arg policy data field as deprecated Signed-off-by: Saul Paredes <[email protected]>
Add a new rule that validates args by substituting env variables received from the input. This new rule ensures that the args received from the input are the same as the args received from the policy. Signed-off-by: Saul Paredes <[email protected]>
Update samples Signed-off-by: Saul Paredes <[email protected]>
…tion policy: improve args and env variables validation
node-builder: Fix SHIM_USE_DEBUG_CONFIG behavior
Signed-off-by: Ankita Pareek <[email protected]>
manuelh-dev
reviewed
Mar 5, 2025
| value: "4" | ||
| - name: TEST_MEMORY_EXPECTED_MEMORY_MB | ||
| value: "1982" | ||
| securityContext: |
There was a problem hiding this comment.
When I tested it manually, sudo was always needed. So I thought probably it would be required here as well.
There was a problem hiding this comment.
Let's please clarify which test exactly requires this and only enable when needed. We will want this if we run this container on the host as a runc container so that we can e.g. detect if /dev/mshv is present
manuelh-dev
reviewed
Mar 7, 2025
| env: | ||
| - name: ENABLED_TESTS | ||
| value: "cpu,memory" | ||
| - name: TEST_CPU_EXPECTED_VCPU_COUNT |
There was a problem hiding this comment.
This won't succeed without memory or cpu limits
There was a problem hiding this comment.
@ms-mahuber What is meant by memory and cpu limits here? Is it something we need to set up explicitly while creating the VMs?
There was a problem hiding this comment.
in the pod manifest resources->limits->cpu|memory
manuelh-dev
reviewed
Mar 7, 2025
| value: "cpu,memory" | ||
| - name: TEST_CPU_EXPECTED_VCPU_COUNT | ||
| value: "4" | ||
| - name: TEST_MEMORY_EXPECTED_MEMORY_MB |
There was a problem hiding this comment.
We internally also run this pod YAML against the vanilla handler where the observed memory values may diverge between kata and kata-cc - let's talk.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Merge Checklist
upstream/missinglabel (orupstream/not-needed) has been set on the PR.Summary
Test Methodology