Skip to content
This repository was archived by the owner on May 13, 2025. It is now read-only.

Commit d847792

Browse files
deeksha345Miraj50Rishabh Rajbragi92Yue Yu
authored
K8s extension/release 1.3.8 (Azure#5798)
* Kubernetes Data Protection Extension CLI (#173) * First draft for Data Protection K8s backup extension (Pending internal review) * Removing tracing * Minor changes to improve azdev style * Internal PR review feedback Co-authored-by: Rishabh Raj <[email protected]> * {AKS - ARC} fix: Update DCR creation to Clusters resource group instead of workspace (#175) * fix: Update DCR creation to Clusters resource group instead of workspace * . * . * casing check * Add self-signed cert to fix PR gate for azureml extension * adding the api version to the operation definition in the client factory * bump k8s-extension version to 1.3.6 * adding tests for all 4 extension types calls * adding to test config file * updating the api version for extension types to be the correct version expected by the service * add test case for flux extension (#184) * bump k8s-extension version to 1.3.6 * bump k8s-extension version to 1.3.6 * adding upstream test for extension types * updating history.rst * [Dapr] Prompt user for existing Dapr installation during extension create (#188) * Add more validations and user prompt for existing installation scenario Signed-off-by: Shubham Sharma <[email protected]> * Add Dapr test' Signed-off-by: Shubham Sharma <[email protected]> * Handle stateful set Signed-off-by: Shubham Sharma <[email protected]> * Update default handling Signed-off-by: Shubham Sharma <[email protected]> * Fix HA handling Signed-off-by: Shubham Sharma <[email protected]> * Add placement service todo Signed-off-by: Shubham Sharma <[email protected]> * Add non-interactive mode Signed-off-by: Shubham Sharma <[email protected]> * Fix lint Signed-off-by: Shubham Sharma <[email protected]> * Update tests Signed-off-by: Shubham Sharma <[email protected]> * Reset configuration for StatefulSet during k8s upgrade Signed-off-by: Shubham Sharma <[email protected]> * Fix lint Signed-off-by: Shubham Sharma <[email protected]> * Retrigger tests Signed-off-by: Shubham Sharma <[email protected]> * Add changes to manage ha and placement params Signed-off-by: Shubham Sharma <[email protected]> * Update message Signed-off-by: Shubham Sharma <[email protected]> * nits Signed-off-by: Shubham Sharma <[email protected]> Signed-off-by: Shubham Sharma <[email protected]> * bump k8s-extension version to 1.3.7 * [Dapr] Disable applying CRDs during a downgrade (#193) * Add logging Signed-off-by: Shubham Sharma <[email protected]> * Lint Signed-off-by: Shubham Sharma <[email protected]> * Update log Signed-off-by: Shubham Sharma <[email protected]> * Revert applyCrds when not downgrading Signed-off-by: Shubham Sharma <[email protected]> * Update logic for removing hooks.applyCrds Signed-off-by: Shubham Sharma <[email protected]> * Revert logic Signed-off-by: Shubham Sharma <[email protected]> * Handle explicit hooks configuration Signed-off-by: Shubham Sharma <[email protected]> * Update comment Signed-off-by: Shubham Sharma <[email protected]> * re-trigger pipeline Signed-off-by: Shubham Sharma <[email protected]> Signed-off-by: Shubham Sharma <[email protected]> * ContainerInsights extension - Add dataCollectionSettings configuration settings (#200) * data collection settings * add support for dataCollectionSettings * fix indention * avoid duplicate use of json loads * remove whitespaces * fix pr feedback * Upgrade Python version from 3.6 to 3.7 (#203) * Upgrade Python version from 3.6 to 3.10 Upgrade to 3.10 for the job that runs Wheel, PyLint, Flake, etc., since 3.6 is not supported anymore by hosted-agent-software. * Upgrade to Python 3.10 from 3.6 Upgrade to 3.10 as 3.6 is not supported * Switch PyLink to 1.9.4 Switch PyLink to 1.9.4 from 1.9.5, as 1.9.5 is not supported with Python 3.10 * Use Python 3.7 for Static Analysis Use 3.7, as 3.10 does not support certain properties used by astpeephole.py that is used by Static Analysis tools * Try unpinned version of PyLint PyLint 1.9.5 doesn't work with Python 3.7. So, trying to see if it automatically pulls the latest compatible version. * Run pylint as a separate command * Update pylintrc (#204) * Update pylintrc * Update k8s-custom-pipelines.yml * Disable PyLint (#205) Disable PyLint for now, as the new version has breaking changes and requires lot more fixes * Disable PyLint on CI scripts * Fixes for script errors * Upgrade Static Analysis Python version Upgrade the Python version for Static Analysis to 3.10, from 3.7, now that PyLint is disabled * Try 3.9, as 3.10 has breaking changes for Flake8 * Remove version pinning for flake8 Try Python 3.10, without pinning flake8 to a version * Update k8s-custom-pipelines.yml * Use Python 3.8.1 & flake8 6.0.0 * Use Python 3.8 instead of 3.8.1 * Update k8s-custom-pipelines.yml * Update .flake8 Update to reflect breaking change in flake8 6.0 * Update source_code_static_analysis.py Scope static analysis tools to only k8s-extension module's source in our branch. * Update k8s-custom-pipelines.yml * Update k8s-custom-pipelines.yml * Update k8s-custom-pipelines.yml * Update pool name in StaticAnalysis To mirror what is in main of azure-cli-extensions * Update k8s-custom-pipelines.yml * Fix indentation * Update k8s-custom-pipelines.yml * Update k8s-custom-pipelines.yml * Revert changes * Revert changes * Revert changes to source_code_static_analysis.py * Update source_code_static_analysis.py * Revert changes * Use Ubuntu 20.4 for BuiltTestPublish stage * Switch to ubuntu-20.04 from latest Co-authored-by: Rishik Hombal <[email protected]> * [Dapr] Do not apply CRD hook when version is unchanged or auto-upgrade is being disabled (#201) * Update logic Signed-off-by: Shubham Sharma <[email protected]> * re-trigger pipeline Signed-off-by: Shubham Sharma <[email protected]> * re-trigger pipeline Signed-off-by: Shubham Sharma <[email protected]> Signed-off-by: Shubham Sharma <[email protected]> Co-authored-by: NarayanThiru <[email protected]> * add dummy key for amalogs as well * bump k8s-extension version to 1.3.8 * Adding GA api version 2022-11-01 exposing isSystemExtension and support for plan info * Seperate args for plan name, product and publisher * updating cassete file * updating HISTORY.rst * k8s-extension release 1.3.8 --------- Signed-off-by: Shubham Sharma <[email protected]> Co-authored-by: Rishabh Raj <[email protected]> Co-authored-by: Rishabh Raj <[email protected]> Co-authored-by: bragi92 <[email protected]> Co-authored-by: Yue Yu <[email protected]> Co-authored-by: Deeksha Sharma <[email protected]> Co-authored-by: Bavneet Singh <[email protected]> Co-authored-by: Shubham Sharma <[email protected]> Co-authored-by: Bavneet Singh <[email protected]> Co-authored-by: Ganga Mahesh Siddem <[email protected]> Co-authored-by: NarayanThiru <[email protected]> Co-authored-by: Rishik Hombal <[email protected]> Co-authored-by: Amol Agrawal <[email protected]> Co-authored-by: Amol Agrawal <[email protected]> Co-authored-by: Arif Lakhani <[email protected]> Co-authored-by: Arif-lakhani <[email protected]>
1 parent 055c2cf commit d847792

File tree

82 files changed

+22673
-94
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

82 files changed

+22673
-94
lines changed

src/k8s-extension/HISTORY.rst

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,13 @@
33
Release History
44
===============
55

6+
1.3.8
7+
++++++++++++++++++
8+
* Fixes to address the bug with msi auth mode for azuremonitor-containers extension version >= 3.0.0
9+
* microsoft.dapr: disable apply-CRDs hook if auto-upgrade is disabled
10+
* microsoft.azuremonitor.containers: ContainerInsights Extension add dataCollectionSettings to configuration settings
11+
* k8s-extension Adding GA api version 2022-11-01 exposing isSystemExtension and support
12+
613
1.3.7
714
++++++++++++++++++
815
* microsoft.dapr: prompt user for existing dapr installation during extension create

src/k8s-extension/README.rst

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -31,6 +31,10 @@ az k8s-extension create \
3131
--version versionNumber \
3232
--auto-upgrade-minor-version autoUpgrade \
3333
--configuration-settings exampleSetting=exampleValue \
34+
--plan-name examplePlanName \
35+
--plan-publisher examplePublisher \
36+
--plan-product exampleOfferId \
37+
3438
```
3539

3640
##### Get a KubernetesExtension

src/k8s-extension/azext_k8s_extension/_format.py

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -15,12 +15,21 @@ def k8s_extension_show_table_format(result):
1515

1616

1717
def __get_table_row(result):
18+
plan_name, plan_publisher, plan_product = '', '', ''
19+
if result['plan']:
20+
plan_name = result['plan']['name']
21+
plan_publisher = result['plan']['publisher']
22+
plan_product = result['plan']['product']
1823
return OrderedDict([
1924
('name', result['name']),
2025
('extensionType', result.get('extensionType', '')),
2126
('version', result.get('version', '')),
2227
('provisioningState', result.get('provisioningState', '')),
2328
('lastModifiedAt', result.get('systemData', {}).get('lastModifiedAt', '')),
29+
('plan_name', plan_name),
30+
{'plan_publisher', plan_publisher},
31+
('plan_product', plan_product),
32+
('isSystemExtension', result.get('isSystemExtension', '')),
2433
])
2534

2635

src/k8s-extension/azext_k8s_extension/_help.py

Lines changed: 10 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -15,13 +15,19 @@
1515

1616
helps[f'{consts.EXTENSION_NAME} create'] = f"""
1717
type: command
18-
short-summary: Create a Kubernetes Extension.
18+
short-summary: Create a Kubernetes Cluster Extension, including purchasing an extension Offer from Azure Marketplace (AKS only). Please refer to the example at the end to see how to create an extension or purchase an extension offer.
1919
examples:
2020
- name: Create a Kubernetes Extension
2121
text: |-
2222
az {consts.EXTENSION_NAME} create --resource-group my-resource-group \
2323
--cluster-name mycluster --cluster-type connectedClusters --name myextension \
2424
--extension-type microsoft.openservicemesh --scope cluster --release-train stable
25+
- name: Create a Kubernetes Marketplace Extension
26+
text: |-
27+
az {consts.EXTENSION_NAME} create --resource-group my-resource-group \
28+
--cluster-name mycluster --cluster-type managedClusters --name myextension \
29+
--extension-type Contoso.AzureVoteKubernetesAppTest --scope cluster --release-train stable \
30+
--plan-name testplan --plan-product kubernetest_apps_demo_offer --plan-publisher test_test_mix3pptest0011614206850774
2531
"""
2632

2733
helps[f'{consts.EXTENSION_NAME} list'] = f"""
@@ -67,9 +73,9 @@
6773
--cluster-name mycluster --cluster-type connectedClusters \
6874
--name myextension --auto-upgrade true/false --version extension-version \
6975
--release-train stable --configuration-settings settings-key=settings-value \
70-
--configuration-protected-settings protected-settings-key=protected-value \
71-
--configuration-settings-file=config-settings-file \
72-
--configuration-protected-settings-file=protected-settings-file
76+
--config-protected-settings protected-settings-key=protected-value \
77+
--config-settings-file=config-settings-file \
78+
--config-protected-file=protected-settings-file
7379
"""
7480

7581
helps[f'{consts.EXTENSION_NAME} extension-types'] = """

src/k8s-extension/azext_k8s_extension/_params.py

Lines changed: 17 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -12,7 +12,7 @@
1212

1313
from .action import (
1414
AddConfigurationSettings,
15-
AddConfigurationProtectedSettings
15+
AddConfigurationProtectedSettings,
1616
)
1717

1818

@@ -52,30 +52,41 @@ def load_arguments(self, _):
5252
help='Specify the release train for the extension type.')
5353
c.argument('configuration_settings',
5454
arg_group="Configuration",
55-
options_list=['--configuration-settings', '--config-settings', '--config'],
55+
options_list=['--configuration-settings', '--config'],
5656
action=AddConfigurationSettings,
5757
nargs='+',
5858
help='Configuration Settings as key=value pair. Repeat parameter for each setting')
5959
c.argument('configuration_protected_settings',
6060
arg_group="Configuration",
61-
options_list=['--configuration-protected-settings', '--config-protected-settings', '--config-protected'],
61+
options_list=['--config-protected-settings', '--config-protected'],
6262
action=AddConfigurationProtectedSettings,
6363
nargs='+',
6464
help='Configuration Protected Settings as key=value pair. Repeat parameter for each setting')
6565
c.argument('configuration_settings_file',
6666
arg_group="Configuration",
67-
options_list=['--configuration-settings-file', '--config-settings-file', '--config-file'],
67+
options_list=['--config-settings-file', '--config-file'],
6868
help='JSON file path for configuration-settings')
6969
c.argument('configuration_protected_settings_file',
7070
arg_group="Configuration",
71-
options_list=['--configuration-protected-settings-file', '--config-protected-settings-file', '--config-protected-file'],
71+
options_list=['--config-protected-file', '--protected-settings-file'],
7272
help='JSON file path for configuration-protected-settings')
7373
c.argument('release_namespace',
7474
help='Specify the namespace to install the extension release.')
7575
c.argument('target_namespace',
7676
help='Specify the target namespace to install to for the extension instance. This'
7777
' parameter is required if extension scope is set to \'namespace\'')
78-
78+
c.argument('plan_name',
79+
arg_group="Marketplace",
80+
options_list=['--plan-name'],
81+
help='The plan name is referring to the Plan ID of the extension that is being taken from Marketplace portal under Usage Information + Support')
82+
c.argument('plan_product',
83+
arg_group="Marketplace",
84+
options_list=['--plan-product'],
85+
help='The plan product is referring to the Product ID of the extension that is being taken from Marketplace portal under Usage Information + Support. An example of this is the name of the ISV offering used.')
86+
c.argument('plan_publisher',
87+
arg_group="Marketplace",
88+
options_list=['--plan-publisher'],
89+
help='The plan publisher is referring to the Publisher ID of the extension that is being taken from Marketplace portal under Usage Information + Support')
7990
with self.argument_context(f"{consts.EXTENSION_NAME} update") as c:
8091
c.argument('yes',
8192
options_list=['--yes', '-y'],

src/k8s-extension/azext_k8s_extension/consts.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -21,7 +21,7 @@
2121
PROVISIONED_CLUSTER_TYPE = "provisionedclusters"
2222

2323
CONNECTED_CLUSTER_API_VERSION = "2021-10-01"
24-
MANAGED_CLUSTER_API_VERSION = "2021-10-01"
24+
MANAGED_CLUSTER_API_VERSION = "2022-11-01"
2525
APPLIANCE_API_VERSION = "2021-10-31-preview"
2626
HYBRIDCONTAINERSERVICE_API_VERSION = "2022-05-01-preview"
2727

src/k8s-extension/azext_k8s_extension/custom.py

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -109,6 +109,9 @@ def create_k8s_extension(
109109
configuration_settings_file=None,
110110
configuration_protected_settings_file=None,
111111
no_wait=False,
112+
plan_name=None,
113+
plan_publisher=None,
114+
plan_product=None
112115
):
113116
"""Create a new Extension Instance."""
114117

@@ -182,6 +185,9 @@ def create_k8s_extension(
182185
config_protected_settings,
183186
configuration_settings_file,
184187
configuration_protected_settings_file,
188+
plan_name,
189+
plan_publisher,
190+
plan_product
185191
)
186192

187193
# Common validations

src/k8s-extension/azext_k8s_extension/partner_extensions/AzureDefender.py

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -21,7 +21,8 @@ class AzureDefender(DefaultExtension):
2121
def Create(self, cmd, client, resource_group_name, cluster_name, name, cluster_type, cluster_rp,
2222
extension_type, scope, auto_upgrade_minor_version, release_train, version, target_namespace,
2323
release_namespace, configuration_settings, configuration_protected_settings,
24-
configuration_settings_file, configuration_protected_settings_file):
24+
configuration_settings_file, configuration_protected_settings_file, plan_name,
25+
plan_publisher, plan_product):
2526

2627
"""ExtensionType 'microsoft.azuredefender.kubernetes' specific validations & defaults for Create
2728
Must create and return a valid 'Extension' object.

src/k8s-extension/azext_k8s_extension/partner_extensions/AzureMLKubernetes.py

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -114,7 +114,8 @@ def __init__(self):
114114
def Create(self, cmd, client, resource_group_name, cluster_name, name, cluster_type, cluster_rp,
115115
extension_type, scope, auto_upgrade_minor_version, release_train, version, target_namespace,
116116
release_namespace, configuration_settings, configuration_protected_settings,
117-
configuration_settings_file, configuration_protected_settings_file):
117+
configuration_settings_file, configuration_protected_settings_file, plan_name,
118+
plan_publisher, plan_product):
118119

119120
logger.warning("Troubleshooting: {}".format(self.TSG_LINK))
120121

src/k8s-extension/azext_k8s_extension/partner_extensions/ContainerInsights.py

Lines changed: 28 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,7 @@
77

88
import datetime
99
import json
10+
import re
1011

1112
from ..utils import get_cluster_rp_api_version
1213
from .. import consts
@@ -36,7 +37,8 @@ class ContainerInsights(DefaultExtension):
3637
def Create(self, cmd, client, resource_group_name, cluster_name, name, cluster_type, cluster_rp,
3738
extension_type, scope, auto_upgrade_minor_version, release_train, version, target_namespace,
3839
release_namespace, configuration_settings, configuration_protected_settings,
39-
configuration_settings_file, configuration_protected_settings_file):
40+
configuration_settings_file, configuration_protected_settings_file,
41+
plan_name, plan_publisher, plan_product):
4042
"""ExtensionType 'microsoft.azuremonitor.containers' specific validations & defaults for Create
4143
Must create and return a valid 'Extension' object.
4244
@@ -453,6 +455,7 @@ def _get_container_insights_settings(cmd, cluster_resource_group_name, cluster_r
453455
subscription_id = get_subscription_id(cmd.cli_ctx)
454456
workspace_resource_id = ''
455457
useAADAuth = False
458+
extensionSettings = {}
456459

457460
if configuration_settings is not None:
458461
if 'loganalyticsworkspaceresourceid' in configuration_settings:
@@ -473,6 +476,26 @@ def _get_container_insights_settings(cmd, cluster_resource_group_name, cluster_r
473476
logger.info("provided useAADAuth flag is : %s", useAADAuthSetting)
474477
if (isinstance(useAADAuthSetting, str) and str(useAADAuthSetting).lower() == "true") or (isinstance(useAADAuthSetting, bool) and useAADAuthSetting):
475478
useAADAuth = True
479+
if useAADAuth and ('dataCollectionSettings' in configuration_settings):
480+
dataCollectionSettingsString = configuration_settings["dataCollectionSettings"]
481+
logger.info("provided dataCollectionSettings is : %s", dataCollectionSettingsString)
482+
dataCollectionSettings = json.loads(dataCollectionSettingsString)
483+
if 'interval' in dataCollectionSettings.keys():
484+
intervalValue = dataCollectionSettings["interval"]
485+
if (bool(re.match(r'^[0-9]+[m]$', intervalValue))) is False:
486+
raise InvalidArgumentValueError('interval format must be in <number>m')
487+
intervalValue = int(intervalValue.rstrip("m"))
488+
if intervalValue <= 0 or intervalValue > 30:
489+
raise InvalidArgumentValueError('interval value MUST be in the range from 1m to 30m')
490+
if 'namespaceFilteringMode' in dataCollectionSettings.keys():
491+
namespaceFilteringModeValue = dataCollectionSettings["namespaceFilteringMode"].lower()
492+
if namespaceFilteringModeValue not in ["off", "exclude", "include"]:
493+
raise InvalidArgumentValueError('namespaceFilteringMode value MUST be either Off or Exclude or Include')
494+
if 'namespaces' in dataCollectionSettings.keys():
495+
namspaces = dataCollectionSettings["namespaces"]
496+
if isinstance(namspaces, list) is False:
497+
raise InvalidArgumentValueError('namespaces must be an array type')
498+
extensionSettings["dataCollectionSettings"] = dataCollectionSettings
476499

477500
workspace_resource_id = workspace_resource_id.strip()
478501

@@ -502,7 +525,7 @@ def _get_container_insights_settings(cmd, cluster_resource_group_name, cluster_r
502525
if is_ci_extension_type:
503526
if useAADAuth:
504527
logger.info("creating data collection rule and association")
505-
_ensure_container_insights_dcr_for_monitoring(cmd, subscription_id, cluster_resource_group_name, cluster_rp, cluster_type, cluster_name, workspace_resource_id)
528+
_ensure_container_insights_dcr_for_monitoring(cmd, subscription_id, cluster_resource_group_name, cluster_rp, cluster_type, cluster_name, workspace_resource_id, extensionSettings)
506529
elif not _is_container_insights_solution_exists(cmd, workspace_resource_id):
507530
logger.info("Creating ContainerInsights solution resource, since it doesn't exist and it is using legacy authentication")
508531
_ensure_container_insights_for_monitoring(cmd, workspace_resource_id).result()
@@ -520,6 +543,7 @@ def _get_container_insights_settings(cmd, cluster_resource_group_name, cluster_r
520543

521544
# workspace key not used in case of AAD MSI auth
522545
configuration_protected_settings['omsagent.secret.key'] = "<not_used>"
546+
configuration_protected_settings['amalogs.secret.key'] = "<not_used>"
523547
if not useAADAuth:
524548
shared_keys = log_analytics_client.shared_keys.get_shared_keys(
525549
workspace_rg_name, workspace_name)
@@ -570,7 +594,7 @@ def get_existing_container_insights_extension_dcr_tags(cmd, dcr_url):
570594
return tags
571595

572596

573-
def _ensure_container_insights_dcr_for_monitoring(cmd, subscription_id, cluster_resource_group_name, cluster_rp, cluster_type, cluster_name, workspace_resource_id):
597+
def _ensure_container_insights_dcr_for_monitoring(cmd, subscription_id, cluster_resource_group_name, cluster_rp, cluster_type, cluster_name, workspace_resource_id, extensionSettings):
574598
from azure.core.exceptions import HttpResponseError
575599

576600
cluster_region = ''
@@ -665,6 +689,7 @@ def _ensure_container_insights_dcr_for_monitoring(cmd, subscription_id, cluster_
665689
"Microsoft-ContainerInsights-Group-Default"
666690
],
667691
"extensionName": "ContainerInsights",
692+
"extensionSettings": extensionSettings
668693
}
669694
]
670695
},

0 commit comments

Comments
 (0)