[confcom] updating READMEs and -h command, allowing for regex env vars

src/confcom/HISTORY.rst

@@ -2,6 +2,11 @@
 
 Release History
 ===============
+0.2.11
+* bug fix for clean room scenario where non-existent docker client connection attempted to be closed
+* adding ability for ARM Template workflows to use regex for environment variables
+* fixing linux permissions for dmverity-vhd tool
+
 0.2.10
 * dmverity-vhd tool fixes
 * changing startup checks to errors rather than warnings

src/confcom/README.md

@@ -4,19 +4,11 @@
   - [Repository](#repository)
   - [Prerequisites](#prerequisites)
   - [Installation Instructions (End User)](#installation-instructions-end-user)
-  - [Generating a confidential execution enforcement (cce) policy](#generating-a-confidential-execution-enforcement-cce-policy)
-  - [Setup and Instructions for Developers](#setup-and-instructions-for-developers)
-    - [Setup Development Environment](#setup-development-environment)
-    - [Build Extension Binary(Wheel) and Run Extension Tests](#build-extension-binarywheel-and-run-extension-tests)
-    - [Miscellaneous](#miscellaneous)
-  - [Azure Container Registration authentication](#azure-container-registration-authentication)
-    - [Authentication with service principals](#authentication-with-service-principals)
-    - [Authenticate with Azure managed identity](#authenticate-with-azure-managed-identity)
   - [Trademarks](#trademarks)
 
 ## Repository
 
-- <https://github.com/Azure/ACC-CLI/tree/main/az_extensions/confcom>
+- <https://github.com/Azure/azure-cli-extensions/tree/main/src/confcom>
 
 ## Prerequisites
 
@@ -37,23 +29,6 @@
 
   - Windows: [Docker Desktop](https://www.docker.com/products/docker-desktop) and [WSL2](https://docs.microsoft.com/en-us/windows/wsl/install)
 
-## Docker Standalone Instructions (End User)
-
-### TODO: change this image when it goes to a public registry
-
-1. Download the docker container: `fishersnpregistry.azurecr.io/confcom-cli:clean-room`
-2. Run:
-
-  ```bash
-    docker run -v "$(pwd):/temp" -v /var/run/docker.sock:/var/run/docker.sock fishersnpregistry.azurecr.io/confcom-cli:clean-room az confcom acipolicygen -a temp/template.json
-  ```
-
-Notes:
-
-- The first `-v` flag can be changed to go wherever in the local machine that has the input files for generating policies. For example, the ARM Template that is going to be used.
-- The second `-v` is for mounting the Docker socket into the container, so Docker must be running on the host machine in order to generate policies from images that are contained within the Docker daemon. This includes images that need to be pulled from a remote registry.
-- The path to the input file in the `az confcom acipolicygen` snippet must line up with where the local folder is getting mounted in the first `-v` flag. For example, above we are mounting to `/temp` in the container so the CLI command will be `az confcom acipolicygen -a /temp/template.json` because `template.json` is in the current local directory.
-
 ## Installation Instructions (End User)
 
 1. Install Azure CLI through following ways:
@@ -67,9 +42,11 @@ Notes:
 
    2. Option 2:(Linux Only) [Install through Linux Package Tools](https://docs.microsoft.com/en-us/cli/azure/install-azure-cli-linux?pivots=apt).
 
-## Generating a confidential execution enforcement (cce) policy
+2. Install the `confcom` extension:
 
-Please see [ACIConfidentialSecurityPolicySpec](https://microsoft-my.sharepoint.com/:w:/p/sewong/EV7PkPR5kWJMnmqm9TtWt0QBhmpYg1HqKwknw07DleugKQ?e=zLQZOl)
+   ```bash
+   az extension add -n confcom
+   ```
 
 ## Trademarks
 

src/confcom/azext_confcom/_help.py

@@ -82,10 +82,12 @@
           short-summary: 'When enabled, the generated security policy is printed to the command line instead of injected into the input ARM Template'
 
     examples:
-        - name: Input a policy.json file to create a base64 encoded Confidential Container Security Policy
-          text: az confcom acipolicygen --input "./policy.json"
-        - name: Input a policy.json file to create a human-readable Confidential Container Security Policy
-          text: az confcom acipolicygen --input "./policy.json" --outraw-pretty-print
-        - name: Input a policy.json file to save a Confidential Container Security Policy to a file
-          text: az confcom acipolicygen --input "./policy.json" -s "./output-file.txt"
+        - name: Input an ARM Template file to inject a base64 encoded Confidential Container Security Policy into the ARM Template
+          text: az confcom acipolicygen --template-file "./template.json"
+        - name: Input an ARM Template file to create a human-readable Confidential Container Security Policy
+          text: az confcom acipolicygen --template-file "./template.json" --outraw-pretty-print
+        - name: Input an ARM Template file to save a Confidential Container Security Policy to a file
+          text: az confcom acipolicygen --template-file "./template.json" -s "./output-file.txt"
+        - name: Input an ARM Template file and use a tar file as the image source instead of the Docker daemon
+          text: az confcom acipolicygen --template-file "./template.json" --tar "./image.tar"
 """

src/confcom/azext_confcom/container.py

@@ -308,7 +308,7 @@ def __init__(
         allow_elevated: bool,
         id_val: str,
         extraEnvironmentRules: Dict,
-        allowStdioAccess: bool = False,
+        allowStdioAccess: bool = True,
         execProcesses: List = None,
         signals: List = None,
     ) -> None:

src/confcom/azext_confcom/data/internal_config.json

@@ -1,5 +1,5 @@
 {
-    "version": "0.2.10",
+    "version": "0.2.11",
     "hcsshim_config": {
         "maxVersion": "1.0.0",
         "minVersion": "0.0.1"

src/confcom/azext_confcom/rootfs_proxy.py

@@ -6,6 +6,7 @@
 import subprocess
 from typing import List
 import os
+import stat
 from pathlib import Path
 import platform
 from azext_confcom.errors import eprint
@@ -40,6 +41,10 @@ def __init__(self):
 
         if not os.path.exists(self.policy_bin):
             raise RuntimeError("The extension binary file cannot be located.")
+        if not os.access(self.policy_bin, os.X_OK):
+            # add executable permissions for the current user if they don't exist
+            st = os.stat(self.policy_bin)
+            os.chmod(self.policy_bin, st.st_mode | stat.S_IXUSR)
 
     def get_policy_image_layers(
         self, image: str, tag: str, tar_location: str = ""

src/confcom/azext_confcom/security_policy.py

@@ -139,8 +139,6 @@ def _get_rootfs_proxy(self) -> SecurityPolicyProxy:
     def _close_docker_client(self) -> None:
         if self._docker_client:
             self._get_docker_client().close()
-        else:
-            docker.from_env().close()
 
     def close(self) -> None:
         self._close_docker_client()
@@ -375,7 +373,7 @@ def _policy_serialization(self, use_json, pretty_print=False) -> str:
 
         if not is_sidecars:
             # add in the default containers that have their hashes pre-computed
-            policy += config.DEFAULT_CONTAINERS
+            policy += copy.deepcopy(config.DEFAULT_CONTAINERS)
             if self._disable_stdio:
                 for container in policy:
                     container[config.POLICY_FIELD_CONTAINERS_ALLOW_STDIO_ACCESS] = False
@@ -706,6 +704,7 @@ def load_policy_from_image_name(
             config.POLICY_FIELD_CONTAINERS_ELEMENTS_REGO_FRAGMENTS: config.DEFAULT_REGO_FRAGMENTS,
         },
         debug_mode=debug_mode,
+        disable_stdio=disable_stdio,
     )
 
 

src/confcom/azext_confcom/template_util.py

@@ -138,21 +138,33 @@ def process_env_vars_from_template(image_properties: dict) -> List[Dict[str, str
     )
 
     if template_env_vars:
-        env_vars = [
-            {
-                config.ACI_FIELD_CONTAINERS_ENVS_NAME: case_insensitive_dict_get(
-                    x, "name"
-                ),
-                config.ACI_FIELD_CONTAINERS_ENVS_VALUE: case_insensitive_dict_get(
-                    x, "value"
-                ) or
-                case_insensitive_dict_get(
-                    x, "secureValue"
-                ),
-                config.ACI_FIELD_CONTAINERS_ENVS_STRATEGY: "string",
-            }
-            for x in template_env_vars
-        ]
+        for env_var in template_env_vars:
+            name = case_insensitive_dict_get(env_var, "name")
+            value = case_insensitive_dict_get(env_var, "value") or case_insensitive_dict_get(env_var, "secureValue")
+
+            if not name:
+                eprint(
+                    f"Environment variable with value: {value} is missing a name"
+                )
+
+            if value:
+                if config.ACI_FIELD_TEMPLATE_PARAMETERS in value:
+                    response = input(f'Create a wildcard policy for the environment variable {name} (y/n): ')
+                    if response.lower() == 'y':
+                        env_vars.append({
+                            config.ACI_FIELD_CONTAINERS_ENVS_NAME: name,
+                            config.ACI_FIELD_CONTAINERS_ENVS_VALUE: ".*",
+                            config.ACI_FIELD_CONTAINERS_ENVS_STRATEGY: "re2",
+                        })
+                else:
+                    env_vars.append({
+                        config.ACI_FIELD_CONTAINERS_ENVS_NAME: name,
+                        config.ACI_FIELD_CONTAINERS_ENVS_VALUE: value,
+                        config.ACI_FIELD_CONTAINERS_ENVS_STRATEGY: "string",
+                    })
+            else:
+                eprint(f'Environment variable {name} does not have a value. Please check the template file.')
+
     return env_vars
 
 
@@ -329,7 +341,7 @@ def change_key_names(dictionary) -> Dict:
     return dictionary
 
 
-def find_value_in_params_and_vars(params: dict, vars_dict: dict, search: str) -> str:
+def find_value_in_params_and_vars(params: dict, vars_dict: dict, search: str, ignore_undefined_parameters=False) -> str:
     """Utility function: either returns the input search value,
     or replaces it with the defined value in either params or vars of the ARM template"""
     # this pattern might need to be updated for more naming options in the future
@@ -352,7 +364,7 @@ def find_value_in_params_and_vars(params: dict, vars_dict: dict, search: str) ->
 
         if not param_value:
             eprint(
-                f"""Field ["{param_name}"] not found in ["{config.ACI_FIELD_TEMPLATE_PARAMETERS}"]
+                f"""Field "{param_name}" not found in ["{config.ACI_FIELD_TEMPLATE_PARAMETERS}"]
                  or ["{config.ACI_FIELD_TEMPLATE_VARIABLES}"]"""
             )
         # fallback to default value
@@ -362,16 +374,16 @@ def find_value_in_params_and_vars(params: dict, vars_dict: dict, search: str) ->
     else:
         match = case_insensitive_dict_get(vars_dict, param_name)
 
-    if not match:
+    if not match and not ignore_undefined_parameters:
         eprint(
-            f"""Field ["{param_name}"] not found in ["{config.ACI_FIELD_TEMPLATE_PARAMETERS}"]
+            f"""Field "{param_name}"'s value not found in ["{config.ACI_FIELD_TEMPLATE_PARAMETERS}"]
              or ["{config.ACI_FIELD_TEMPLATE_VARIABLES}"]"""
         )
 
-    return match
+    return match or search
 
 
-def parse_template(params: dict, vars_dict: dict, template) -> Any:
+def parse_template(params: dict, vars_dict: dict, template, ignore_undefined_parameters=False) -> Any:
     """Utility function: replace all instances of variable and parameter references in an ARM template
     current limitations:
         - object values for parameters and variables
@@ -382,12 +394,17 @@ def parse_template(params: dict, vars_dict: dict, template) -> Any:
     if isinstance(template, dict):
         for key, value in template.items():
             if isinstance(value, str):
-                template[key] = find_value_in_params_and_vars(params, vars_dict, value)
+                # we want to ignore undefined parameters for only env var values, not names
+                template[key] = find_value_in_params_and_vars(params, vars_dict, value,
+                                                              ignore_undefined_parameters=ignore_undefined_parameters
+                                                              and key.lower() in ("value", "securevalue"))
             elif isinstance(value, dict):
                 parse_template(params, vars_dict, value)
             elif isinstance(value, list):
                 for i, _ in enumerate(value):
-                    template[key][i] = parse_template(params, vars_dict, value[i])
+                    template[key][i] = parse_template(params, vars_dict, value[i],
+                                                      ignore_undefined_parameters=key
+                                                      == config.ACI_FIELD_CONTAINERS_ENVS)
     return template
 
 

src/confcom/azext_confcom/tests/latest/README.md

@@ -37,6 +37,9 @@ test_update_infrastructure_svn | python:3.6.14-slim-buster | Change the minimum
 test_multiple_policies | python:3.6.14-slim-buster & rust:1.52.1 | See if two unique policies are generated from a single ARM Template container multiple container groups. Also have an extra resource that is untouched. Also has a secureValue for an environment variable.
 test_arm_template_with_init_container | python:3.6.14-slim-buster & rust:1.52.1 | See if having an initContainer is picked up and added to the list of valid containers
 test_arm_template_without_stdio_access | rust:1.52.1 | See if disabling container stdio access gets passed down to individual containers
+test_arm_template_policy_regex | python:3.6.14-slim-buster | Make sure the regex generated from the ARM Template workflow matches that of the policy.json workflow
+test_wildcard_env_var | python:3.6.14-slim-buster | Check that an "allow all" regex is created when a value for env var is not provided via a parameter value
+test_wildcard_env_var_invalid | N/A | Make sure the process errors out if a value is not given for an env var or an undefined parameter is used for the name of an env var
 
 ## policy.json [test file](test_confcom_scenario.py)