Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

tests: add OS_IMAGE_TYPE setting to allow for minimal tests #5682

Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 29 additions & 1 deletion doc/rtd/development/integration_tests.rst
Original file line number Diff line number Diff line change
Expand Up @@ -263,7 +263,7 @@ the form: ``<image_id>[::<os>::<release>::<version>]`` where ``image_id`` is a
cloud's image ID, ``os`` is the OS name, and ``release`` is the OS
release name. So, for example, Ubuntu 24.04 LTS (Noble Numbat) on LXD is
``ubuntu:noble::ubuntu::noble::24.04`` or RHEL8 on Amazon is
``ami-justanexample::rhel::8``. When a full specification is given,
``ami-justanexample::rhel::9::9.3``. When a full specification is given,
only tests which are intended for use on that OS and release will be
executed.

Expand All @@ -285,6 +285,33 @@ variable to be the desired image specification.
OS_IMAGE = 'jammy'


To run integration tests on a specific type/family of image, modify the
``OS_IMAGE_TYPE`` variable to be the desired image type. This comes from
`pycloudlib's ImageType enum`_, which can take the following values:

- "generic"
- "minimal"
- "Pro"
- "Pro FIPS"

.. tab-set::

.. tab-item:: Inline environment variable

.. code-block:: bash

CLOUD_INIT_PLATFORM=lxd_container CLOUD_INIT_OS_IMAGE=noble CLOUD_INIT_OS_IMAGE_TYPE=minimal tox -e integration_tests

.. tab-item:: user_settings.py file

.. code-block:: python

OS_PLATFORM = 'lxd_container'
OS_IMAGE = 'noble'
OS_IMAGE_TYPE = 'minimal'

Note: Not all clouds and OSes support all image types

Image setup
===========

Expand Down Expand Up @@ -427,3 +454,4 @@ Customizing the launch arguments before launching an instance manually:
.. _Pytest marks: https://github.com/canonical/cloud-init/blob/af7eb1deab12c7208853c5d18b55228e0ba29c4d/tests/integration_tests/conftest.py#L220-L224
.. _IntegrationCloud: https://github.com/canonical/cloud-init/blob/af7eb1deab12c7208853c5d18b55228e0ba29c4d/tests/integration_tests/clouds.py#L102
.. _pycloudlib configuration documentation: https://pycloudlib.readthedocs.io/en/latest/configuration.html
.. _pycloudlib's ImageType enum: https://github.com/canonical/pycloudlib/blob/1!10.0.0/pycloudlib/cloud.py#L28
2 changes: 1 addition & 1 deletion integration-requirements.txt
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
# PyPI requirements for cloud-init integration testing
# https://docs.cloud-init.io/en/latest/development/integration_tests.html
#
pycloudlib>=1!6.7.0,<1!9.3
pycloudlib>=1!9.1.0,<1!10

# Avoid breaking change in `testpaths` treatment forced
# test/unittests/conftest.py to be loaded by our integration-tests tox env
Expand Down
5 changes: 5 additions & 0 deletions tests/integration_tests/clouds.py
Original file line number Diff line number Diff line change
Expand Up @@ -275,6 +275,11 @@ class _LxdIntegrationCloud(IntegrationCloud):
instance_tag: str
cloud_instance: _BaseLXD

def _get_initial_image(self, **kwargs) -> str:
return super()._get_initial_image(
image_type=self._image_type, **kwargs
)

def _get_or_set_profile_list(self, release):
return None

Expand Down
12 changes: 10 additions & 2 deletions tests/integration_tests/conftest.py
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@
from typing import Dict, Generator, Iterator, List, Type

import pytest
from pycloudlib.cloud import ImageType
from pycloudlib.lxd.instance import LXDInstance

from tests.integration_tests import integration_settings
Expand Down Expand Up @@ -80,8 +81,15 @@ def session_cloud() -> Generator[IntegrationCloud, None, None]:
f"{integration_settings.PLATFORM} is an invalid PLATFORM "
f"specified in settings. Must be one of {list(platforms.keys())}"
)

cloud = platforms[integration_settings.PLATFORM]()
image_types = [member.value for member in ImageType.__members__.values()]
try:
image_type = ImageType(integration_settings.OS_IMAGE_TYPE)
except ValueError:
raise ValueError(
f"{integration_settings.OS_IMAGE_TYPE} is an invalid OS_IMAGE_TYPE"
f" specified in settings. Must be one of {image_types}"
)
cloud = platforms[integration_settings.PLATFORM](image_type=image_type)
cloud.emit_settings_to_log()
yield cloud
cloud.destroy()
Expand Down
12 changes: 12 additions & 0 deletions tests/integration_tests/integration_settings.py
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,17 @@
# to this format internally; in this case, to "None::ubuntu::focal::20.04".)
OS_IMAGE = "focal"


# Determines unique image type or flavor to exercise if the cloud supports
# image-type lookup for daily_image and released_images.
#
# One of the following pycloudlib.cloud.ImageType values:
# - generic
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where do these names come from? I assume that this is a pycloudlib construct? Can we please include a pointer to wherever in pycloudlib this is documented to ease future maintenance of this?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These come from the ImageType enum in pycloudlib. (pycloudlib.cloud.ImageType)

# - minimal
# - Pro
# - Pro FIPS
OS_IMAGE_TYPE = "generic"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We keep talking about additional OS support, and this one comes across as being Ubuntu-specific in a way that might encourage a bunch of OS-specific fields to proliferate across various OSes. We could try to encode this into OS_IMAGE, but I'm also wondering if it might be time for a new abstraction for all the various ways of specifying an image (e.g., differentiating amd64/arm is on the horizon too).

I think I'm ok with this now, but if we have to add another dimension to our image specification, I think we need a better solution.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@TheRealFalcon: from the looks of other distribution vendors , there are similar alternative "offers" such as BYOS (bring your own subscription) images, HA and minimal images. I wonder if it is worth representing an optional "::offering" segment in
OS_IMAGE of the format
<image_id>[::<os>::<release>::<version>[::<offer-name>]]

This way we'd still have the ability to configure a known offer stream of images for Ubuntu, without bloating integration-settings.py options with yet another config param.

In the absence of offer-name, we'd assume generic, otherwise any string could be provided. From Ubuntu perspective, we could provide (generic|minimal|Pro|Pro FIPS). In the future, other vendors could provide BYOS, minimal, OSA(openshift app), etc.

For other distro support, I think we may have to rethink how we use pycloudlib.cloud.ImageType and maybe rename that OfferType to better align with how clouds label separate product images from vendors.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After spending some time on this proposal. I can reject my suggestion for optional <offer_name> suffix in OS_IMAGE for two reasons:

  1. offer names are actually platform/cloud specific. A "minimal" image offer in EC2 will not have the same offer/product name in Azure or GCP. The pycloudlib.cloud.ImageType interprets image types per cloud to provide the right mapping of ImageType to cloud offering/product/image stream so that the integration test runners don't have to have detailed knowledge of a certain flavor/offer name or image per-platform.
  2. providing an optional <offer_name> suffix in OS_IMAGE also requires making <image_id>:: a leading optional value of None or null in order to provide a trailing <offer_name>. This leads to complexity of image specification where it can take either 4 or 5 parts, and significant documentation around when to use the optional leading None::

For these reasons, I think it makes sense to stick with a new OS_IMAGE_TYPE config value (or we could call it IMAGE_FLAVOR) to better represent this unique image stream.


# Populate if you want to use a pre-launched instance instead of
# creating a new one. The exact contents will be platform dependent
EXISTING_INSTANCE_ID: Optional[str] = None
Expand All @@ -47,6 +58,7 @@
# IMAGE GENERATION SETTINGS
##################################################################


# Depending on where we are in the development / test / SRU cycle, we'll want
# different methods of getting the source code to our SUT. Because of
# this there are a number of different ways to initialize
Expand Down