Add example of integration with kubevirt (#3972)

Greetings,

In our team, we handle important Ansible roles to configure various
aspects of RHEL systems, such as sshd, firewalld, grub, crypto-policies,
and more. Although we have implemented automated testing with Molecule
and Podman, there are instances where manual testing on our RHEL hosts
becomes necessary before we proceed with deployment. I propose the
integration of Molecule with Kubevirt as a potential solution to this
issue. Such an integration would allow us to quickly and effectively
test our Ansible roles on actual VMs, enhancing our testing procedures
and providing more reliable results.

Some benefits of testing on VMs:

1. **System-level realism**: Ephemeral VMs provide a complete, isolated
guest OS environment, much like your production environment. Containers
share the host's kernel and are not fully isolated. This difference can
occasionally lead to inconsistencies between testing and production
environments. With VM-based testing, you can ensure the roles will work
as expected on the actual operating system.

2. **Broader Compatibility**: Not all applications or configurations are
container-friendly, especially when they interact with the system at a
low level. VMs provide broader compatibility for testing as they offer a
full-fledged OS environment.

3. **Improved Debugging**: Since VMs provide an entire guest operating
system, it is often easier to debug issues related to system services,
kernel modules, and other low-level components.

4. **Greater Variety of Testing**: VMs can run different kernel
versions, different operating systems, or different system-level
configurations. In contrast, containers are somewhat limited by the
features and configurations of the host kernel.

In this pull request, I have utilized ephemeral VMs as a part of my
example to showcase the benefits of this approach. By harnessing the
capabilities of ephemeral VMs, we can test our Ansible roles in a highly
realistic environment that closely mirrors our actual production
environment, taking full advantage of the benefits outlined above.

This example serves as a practical guide to illustrate how a real-world
application of Molecule and Kubevirt integration can enhance our current
testing methods, enabling more realistic testing.

---------

Signed-off-by: Jose Angel Morena <[email protected]>
Co-authored-by: Jose Angel Morena <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Ajinkya Udgirkar <[email protected]>

docs/kubevirt.md

@@ -0,0 +1,110 @@
+# Using Kubevirt
+
+Below you can see a scenario that is using [Kubevirt VMs](https://kubevirt.io/user-guide/) as test hosts. For Ansible to connect with the SSH in the KubeVirt VMs, it will be made accessible through the Service NodePort.
+When you run `molecule test --scenario kubevirt` the `create`, `converge` and
+`destroy` steps will be run one after another.
+
+This example is using Ansible playbooks and it does not need any molecule
+plugins to run. You can fully control which test requirements you need to be
+installed.
+
+## Prerequisites
+
+The `create.yml` and `destroy.yml` Ansible playbooks require the Ansible collection `kubernetes.core`. For seamless communication with the Kubernetes API server, the collection uses the following environment variables:
+
+- `K8S_AUTH_API_KEY`: This is the token from the service account used to authenticate with the Kubernetes cluster.
+
+- `K8S_AUTH_HOST`: This points to the URL of the Kubernetes cluster's API.
+
+- `K8S_AUTH_VERIFY_SSL`: If set to `false`, this disables the verification of SSL/TLS certificates, which might pose a security risk. It's mainly used for testing environments, particularly when dealing with self-signed certificates.
+
+Additionally, for the playbooks to work, the Kubernetes service account needs specific roles and role bindings to operate in a particular namespace. This ensures the playbook has sufficient privileges to execute commands on the Kubernetes resources. These roles include getting, listing, watching, creating, deleting, and editing virtual machines and services.
+
+```yaml
+---
+apiVersion: v1
+kind: ServiceAccount
+metadata:
+  name: <Molecule Kubernetes Serviceaccount>
+  namespace: <Kubernetes VM Namespace>
+---
+apiVersion: rbac.authorization.k8s.io/v1
+kind: Role
+metadata:
+  namespace: <Kubernetes VM Namespace>
+  name: <Molecule Kubernetes Role>
+rules:
+  - apiGroups: ["kubevirt.io"]
+    resources: ["virtualmachines"]
+    verbs: ["get", "list", "watch", "create", "delete", "patch", "edit"]
+  - apiGroups: [""]
+    resources: ["services"]
+    verbs: ["get", "list", "watch", "create", "delete", "patch", "edit"]
+---
+apiVersion: rbac.authorization.k8s.io/v1
+kind: RoleBinding
+metadata:
+  name: <Molecule Kubernetes Rolebinding>
+  namespace: <Kubernetes VM Namespace>
+subjects:
+  - kind: ServiceAccount
+    name: <Molecule Kubernetes Serviceaccount>
+    namespace: <Kubernetes VM Namespace>
+roleRef:
+  kind: Role
+  name: <Molecule Kubernetes Role>
+  apiGroup: rbac.authorization.k8s.io
+```
+
+You will need to substitute the following placeholders:
+
+- `<Molecule Kubernetes Serviceaccount>`: This refers to the name of the Kubernetes Serviceaccount that the molecule test utilizes to create the KubeVirt VM.
+- `<Kubernetes VM Namespace>`: This denotes the name of the Kubernetes namespace where the VMs will be instantiated.
+- `<Molecule Kubernetes Role>`: This is the name of the Kubernetes role which encapsulates the necessary permissions for the molecule test to function.
+- `<Molecule Kubernetes Rolebinding>`: This represents the name of the Kubernetes rolebinding that associates the role `<Molecule Kubernetes Role>` with the serviceaccount `<Molecule Kubernetes Serviceaccount>`.
+
+## Considerations
+
+- This example is using ephemeral VMs, which enhance the speed of VM creation and cleanup. However, it is important to note that any data in the system will not be retained if the VM is rebooted.
+- You don't need to worry about setting up SSH keys. The `create.yml` Ansible playbook takes care of configuring a temporary SSH key.
+
+## Config playbook
+
+```yaml title="molecule.yml"
+{!../molecule/kubevirt/molecule.yml!}
+```
+
+Please, replace the following parameters:
+
+- `<Kubernetes VM Namespace>`: This should be replaced with the namespace in Kubernetes where you intend to create the KubeVirt VMs.
+- `<Kubernetes Node FQDN>`: Change this to the fully qualified domain name (FQDN) of the Kubernetes node that Ansible will attempt to SSH into via the Service NodePort.
+
+```yaml title="requirements.yml"
+{!../molecule/kubevirt/requirements.yml!}
+```
+
+## Create playbook
+
+```yaml title="create.yml"
+{!../molecule/kubevirt/create.yml!}
+```
+
+```yaml title="tasks/create_vm.yml"
+{!../molecule/kubevirt/tasks/create_vm.yml!}
+```
+
+```yaml title="tasks/create_vm_dictionary.yml"
+{!../molecule/kubevirt/tasks/create_vm_dictionary.yml!}
+```
+
+## Converge playbook
+
+```yaml title="converge.yml"
+{!../molecule/kubevirt/converge.yml!}
+```
+
+## Destroy playbook
+
+```yaml title="destroy.yml"
+{!../molecule/kubevirt/destroy.yml!}
+```

mkdocs.yml

@@ -63,6 +63,7 @@ nav:
   - Examples:
       - docker.md
       - podman.md
+      - kubevirt.md
   - examples.md
   - faq.md
   - contributing.md

molecule/kubevirt/converge.yml

@@ -0,0 +1,29 @@
+- name: Fail if molecule group is missing
+  hosts: localhost
+  tasks:
+    - name: Print some info
+      ansible.builtin.debug:
+        msg: "{{ groups }}"
+
+    - name: Assert group existence
+      ansible.builtin.assert:
+        that: "'molecule' in groups"
+        fail_msg: |
+          molecule group was not found inside inventory groups: {{ groups }}
+
+- name: Converge
+  hosts: molecule
+  # We disable gather facts because it would fail due to our container not
+  # having python installed. This will not prevent use from running 'raw'
+  # commands. Most molecule users are expected to use containers that already
+  # have python installed in order to avoid notable delays installing it.
+  gather_facts: false
+  tasks:
+    - name: Check uname
+      ansible.builtin.raw: uname -a
+      register: result
+      changed_when: false
+
+    - name: Print some info
+      ansible.builtin.assert:
+        that: result.stdout | regex_search("^Linux")

molecule/kubevirt/create.yml

@@ -0,0 +1,96 @@
+- name: Create
+  hosts: localhost
+  connection: local
+  gather_facts: false
+  vars:
+    temporary_ssh_key_size: 2048 # Variable for the size of the SSH key
+  tasks:
+    - name: Set default SSH key path # Sets the path of the SSH key
+      set_fact:
+        tempoary_ssh_key_path: "{{ molecule_ephemeral_directory }}/identity_file"
+
+    - name: Generate SSH key pair # Generates a new SSH key pair
+      community.crypto.openssh_keypair:
+        path: "{{ tempoary_ssh_key_path }}"
+        size: "{{ temporary_ssh_key_size }}"
+      register: temporary_ssh_keypair # Stores the output of this task in a variable
+
+    - name: Set SSH public key # Sets the SSH public key from the key pair
+      set_fact:
+        temporary_ssh_public_key: "{{ temporary_ssh_keypair.public_key }}"
+
+    - name: Create VM in KubeVirt # Calls another file to create the VM in KubeVirt
+      include_tasks: tasks/create_vm.yml
+      loop: "{{ molecule_yml.platforms }}" # Loops over all platforms defined in molecule_yml
+      loop_control:
+        loop_var: vm # Sets the variable for the current item in the loop
+
+    - name: Create Nodeport service if ssh_type is set to NodePort # Conditional block, executes if vm.ssh_service.type is NodePort
+      block:
+        - name: Create ssh NodePort Kubernetes Services # Creates a new NodePort service in Kubernetes
+          kubernetes.core.k8s:
+            state: present
+            definition:
+              apiVersion: v1
+              kind: Service
+              metadata:
+                name: "{{ vm.name }}"
+                namespace: "{{ vm.namespace }}"
+              spec:
+                ports:
+                  - port: 22
+                    protocol: TCP
+                    targetPort: 22
+                selector:
+                  kubevirt.io/domain: "{{ vm.name }}"
+                type: NodePort
+          loop: "{{ molecule_yml.platforms }}" # Loops over all platforms defined in molecule_yml
+          loop_control:
+            loop_var: vm # Sets the variable for the current item in the loop
+
+        - name: Retrieve Service Info # Retrieves information about the service
+          kubernetes.core.k8s_info:
+            api_version: v1
+            kind: Service
+            name: "{{ vm.name }}"
+            namespace: "{{ vm.namespace }}"
+          loop: "{{ molecule_yml.platforms }}" # Loops over all platforms defined in molecule_yml
+          loop_control:
+            loop_var: vm # Sets the variable for the current item in the loop
+          register: node_port_services # Stores the output of this task in a variable
+      when: "vm.ssh_service.type == 'NodePort'" # The block is executed when this condition is met
+
+    - name: Create VM dictionary # Calls another file to create a dictionary with information about the VM
+      include_tasks: tasks/create_vm_dictionary.yml
+      loop: "{{ molecule_yml.platforms }}" # Loops over all platforms defined in molecule_yml
+      loop_control:
+        loop_var: vm # Sets the variable for the current item in the loop
+
+    - name: Create ansible inventory from dictionary # Creates an Ansible inventory file from the dictionary
+      vars:
+        molecule_inventory:
+          all:
+            children:
+              molecule:
+                hosts: "{{ molecule_systems }}"
+      ansible.builtin.copy:
+        content: "{{ molecule_inventory | to_nice_yaml }}"
+        dest: "{{ molecule_ephemeral_directory }}/inventory/molecule_inventory.yml"
+        mode: 0600 # Sets the permissions of the file to -rw-------
+
+    - name: Refresh inventory # Refreshes the inventory
+      ansible.builtin.meta: refresh_inventory
+
+    - name: Assert molecule group exists # Checks if the 'molecule' group exists in the inventory
+      ansible.builtin.assert:
+        that: "'molecule' in groups"
+        fail_msg: "Molecule group was not found in inventory groups: {{ groups }}"
+      run_once: true # Ensures this task is only run once, not on every host in 'hosts'
+
+- name: Validate that inventory was refreshed # New playbook to validate the inventory
+  hosts: molecule # Runs on hosts in the 'molecule' group
+  gather_facts: false # Disables fact gathering
+  tasks:
+    - name: Wait for the host to be reachable # Waits for the host to become reachable
+      ansible.builtin.wait_for_connection:
+        timeout: 120 # Waits for up to 120 seconds

molecule/kubevirt/destroy.yml

@@ -0,0 +1,25 @@
+---
+- name: Destroy
+  hosts: localhost
+  connection: local
+  gather_facts: false
+  tasks:
+    - name: Delete VM Instance in KubeVirt
+      kubernetes.core.k8s:
+        state: absent
+        kind: VirtualMachine
+        name: "{{ vm.name }}"
+        namespace: "{{ vm.namespace }}"
+      loop: "{{ molecule_yml.platforms }}"
+      loop_control:
+        loop_var: vm
+
+    - name: Delete VM Instance in KubeVirt
+      kubernetes.core.k8s:
+        state: absent
+        kind: Service
+        name: "{{ vm.name }}"
+        namespace: "{{ vm.namespace }}"
+      loop: "{{ molecule_yml.platforms }}"
+      loop_control:
+        loop_var: vm

molecule/kubevirt/molecule.yml

@@ -0,0 +1,45 @@
+---
+dependency:
+  name: galaxy
+  options:
+    requirements-file: requirements.yml
+    role-file: requirements.yml
+platforms:
+  - name: rhel9
+    image: registry.redhat.io/rhel9/rhel-guest-image
+    namespace: <Kubernetes VM Namespace>
+    ssh_service:
+      type: NodePort
+      nodeport_host: <Kubernetes Node FQDN>
+    ansible_user: cloud-user
+    memory: 1Gi
+  - name: rhel8
+    image: registry.redhat.io/rhel8/rhel-guest-image
+    namespace: <Kubernetes VM Namespace>
+    ssh_service:
+      type: NodePort
+      nodeport_host: <Kubernetes Node FQDN>
+    ansible_user: cloud-user
+    memory: 1Gi
+provisioner:
+  name: ansible
+  config_options:
+    defaults:
+      interpreter_python: auto_silent
+      callback_whitelist: profile_tasks, timer, yaml
+    ssh_connection:
+      pipelining: false
+  log: true
+verifier:
+  name: ansible
+scenario:
+  test_sequence:
+    - dependency
+    - destroy
+    - syntax
+    - create
+    - converge
+    - idempotence
+    - side_effect
+    - verify
+    - destroy

molecule/kubevirt/requirements.yml

@@ -0,0 +1,4 @@
+---
+collections:
+  - name: kubernetes.core
+  - name: community.crypto

molecule/kubevirt/tasks/create_vm.yml

@@ -0,0 +1,59 @@
+---
+- name: Create VM in KubeVirt
+  kubernetes.core.k8s: # Uses the k8s module from the kubernetes.core Ansible collection
+    state: present # Ensures the VM exists. If it doesn't, it will be created.
+    definition:
+      apiVersion: kubevirt.io/v1 # KubeVirt's API version
+      kind: VirtualMachine # The type of Kubernetes resource to create
+      metadata:
+        labels:
+          kubevirt.io/domain: "{{ vm.name }}" # Labels for the VM
+        name: "{{ vm.name }}" # Name of the VM
+        namespace: "{{ vm.namespace }}" # Namespace where the VM will be created
+      spec:
+        running: true # Starts the VM after creation
+        template:
+          metadata:
+            labels:
+              kubevirt.io/domain: "{{ vm.name }}" # Labels for the VM's template
+          spec:
+            domain:
+              devices:
+                disks:
+                  - disk:
+                      bus: virtio # Type of disk bus
+                    name: containerdisk # Name of the container disk
+                  - disk:
+                      bus: virtio # Type of disk bus
+                    name: cloudinitdisk # Name of the cloud-init disk
+                  - name: emptydisk # Name of the empty disk
+                    disk:
+                      bus: virtio # Type of disk bus
+              resources:
+                requests:
+                  memory: "{{ vm.memory | default('1Gi') }}" # Amount of memory requested for the VM
+            volumes:
+              - name: emptydisk
+                emptyDisk:
+                  capacity: "{{ vm.capacity | default('2Gi') }}" # Capacity of the empty ephemeral disk
+              - containerDisk:
+                  image: "{{ vm.image }}" # The image used for the container disk
+                name: containerdisk
+              - cloudInitNoCloud: # Cloud-init configuration
+                  userData: | # User-data script
+                    #cloud-config
+                    preserve_hostname: true
+                    hostname: "{{ vm.name }}"  # Sets the hostname
+                    fqdn: "{{ vm.name }}"      # Fully Qualified Domain Name
+                    prefer_fqdn_over_hostname: true
+                    users:
+                      - default
+                      - name: {{ vm.ansible_user }}
+                        lock_passwd: true   # Locks the password
+                        ssh_authorized_keys:
+                          - "{{ temporary_ssh_public_key }}"  # SSH public key
+                    runcmd:
+                      - [ sh, -c, "hostnamectl set-hostname {{ vm.name }}" ]  # Sets the hostname
+                      - [ sudo, yum, install, -y, qemu-guest-agent ]  # Installs qemu-guest-agent
+                      - [ sudo, systemctl, start, qemu-guest-agent ]  # Starts qemu-guest-agent
+                name: cloudinitdisk