Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

tests: add OS_IMAGE_TYPE setting to allow for minimal tests #5682

Conversation

blackboxsw
Copy link
Collaborator

Proposed Commit Message

tests: add OS_IMAGE_TYPE setting to allow for minimal tests

Allow setting CLOUD_INIT_OS_IMAGE_TYPE="mimimal" to support
running the full integration test suite against Ubuntu minimal
images.

Required for SC-1750: Daily jenkins ubuntu minimal test runner for
Ubuntu Focal, Jammy and Noble releases.

Additional Context

https://warthogs.atlassian.net/browse/SC-1750
Depends on canonical/pycloudlib#410
After the above this PR and the pyclluldlib PR merge, a serserver-jenkins-jobs matrix will be configured to run LXDContainer jobs against ubuntu-minimal:(focal, jammy, and noble)

Test Steps

CLOUD_INIT_OS_IMAGE_TYPE=minimal tox -e integration-tests -- tests/integration-tests/test_paths.py

Merge type

  • Squash merge using "Proposed Commit Message"
  • Rebase and merge unique commits. Requires commit messages per-commit each referencing the pull request number (#<PR_NUM>)

@blackboxsw blackboxsw marked this pull request as draft September 10, 2024 12:12
# image-type lookup for daily_image and released_images.
#
# One of the following:
# - generic
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where do these names come from? I assume that this is a pycloudlib construct? Can we please include a pointer to wherever in pycloudlib this is documented to ease future maintenance of this?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These come from the ImageType enum in pycloudlib. (pycloudlib.cloud.ImageType)

Copy link
Collaborator

@a-dubs a-dubs left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work on this @blackboxsw! I left a fix inline to address @holmanb's question.

Also, would you mind updating the integration docs to cover this new configuration option? Here's a patch to get you started (or if you think it is sufficient, just ship it 🫡)

diff --git a/doc/rtd/development/integration_tests.rst b/doc/rtd/development/integration_tests.rst
index aecb02244..a51657797 100644
--- a/doc/rtd/development/integration_tests.rst
+++ b/doc/rtd/development/integration_tests.rst
@@ -284,6 +284,31 @@ variable to be the desired image specification.
 
             OS_IMAGE = 'jammy'
 
+To run integration tests on a specific type/family of image, modify the
+``OS_IMAGE_TYPE`` variable to be the desired image type. This comes from
+pycloudlib's ImageType enum, which can take the following values:
+
+- "generic"
+- "minimal"
+- "Pro"
+- "Pro FIPS"
+
+.. tab-set::
+
+    .. tab-item:: Inline environment variable
+
+        .. code-block:: bash
+
+            CLOUD_INIT_OS_IMAGE_TYPE='minimal' tox -e integration_tests
+
+    .. tab-item:: user_settings.py file
+
+        .. code-block:: python
+
+            OS_IMAGE_TYPE = 'minimal'
+
+Note: Not all clouds and images support all image types
+
 
 Image setup
 ===========

tests/integration_tests/integration_settings.py Outdated Show resolved Hide resolved
# image-type lookup for daily_image and released_images.
#
# One of the following:
# - generic
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These come from the ImageType enum in pycloudlib. (pycloudlib.cloud.ImageType)

Copy link
Member

@TheRealFalcon TheRealFalcon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I know this is still draft, but I left an inline comment.

# - minimal
# - Pro
# - Pro FIPS
OS_IMAGE_TYPE = "generic"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We keep talking about additional OS support, and this one comes across as being Ubuntu-specific in a way that might encourage a bunch of OS-specific fields to proliferate across various OSes. We could try to encode this into OS_IMAGE, but I'm also wondering if it might be time for a new abstraction for all the various ways of specifying an image (e.g., differentiating amd64/arm is on the horizon too).

I think I'm ok with this now, but if we have to add another dimension to our image specification, I think we need a better solution.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@TheRealFalcon: from the looks of other distribution vendors , there are similar alternative "offers" such as BYOS (bring your own subscription) images, HA and minimal images. I wonder if it is worth representing an optional "::offering" segment in
OS_IMAGE of the format
<image_id>[::<os>::<release>::<version>[::<offer-name>]]

This way we'd still have the ability to configure a known offer stream of images for Ubuntu, without bloating integration-settings.py options with yet another config param.

In the absence of offer-name, we'd assume generic, otherwise any string could be provided. From Ubuntu perspective, we could provide (generic|minimal|Pro|Pro FIPS). In the future, other vendors could provide BYOS, minimal, OSA(openshift app), etc.

For other distro support, I think we may have to rethink how we use pycloudlib.cloud.ImageType and maybe rename that OfferType to better align with how clouds label separate product images from vendors.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After spending some time on this proposal. I can reject my suggestion for optional <offer_name> suffix in OS_IMAGE for two reasons:

  1. offer names are actually platform/cloud specific. A "minimal" image offer in EC2 will not have the same offer/product name in Azure or GCP. The pycloudlib.cloud.ImageType interprets image types per cloud to provide the right mapping of ImageType to cloud offering/product/image stream so that the integration test runners don't have to have detailed knowledge of a certain flavor/offer name or image per-platform.
  2. providing an optional <offer_name> suffix in OS_IMAGE also requires making <image_id>:: a leading optional value of None or null in order to provide a trailing <offer_name>. This leads to complexity of image specification where it can take either 4 or 5 parts, and significant documentation around when to use the optional leading None::

For these reasons, I think it makes sense to stick with a new OS_IMAGE_TYPE config value (or we could call it IMAGE_FLAVOR) to better represent this unique image stream.

Allow setting CLOUD_INIT_OS_IMAGE_TYPE="mimimal" to support
running the full integration test suite against Ubuntu minimal
images.

Required for SC-1750: Daily jenkins ubuntu minimal test runner for
Ubuntu Focal, Jammy and Noble releases.
@blackboxsw blackboxsw force-pushed the SC-1750-lxd-minimal-integration-testing branch from 746a75a to 5af25f0 Compare October 1, 2024 12:57
@github-actions github-actions bot added the documentation This Pull Request changes documentation label Oct 1, 2024
@blackboxsw blackboxsw marked this pull request as ready for review October 1, 2024 12:57
Copy link
Member

@TheRealFalcon TheRealFalcon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Sorry for the churn on this!

@blackboxsw blackboxsw merged commit 2e866d3 into canonical:main Oct 2, 2024
21 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
documentation This Pull Request changes documentation
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants