Skip to content
This repository was archived by the owner on May 13, 2025. It is now read-only.

Commit e298f57

Browse files
jonathan-innisyuyue9284Yue Yuliakaz
authored
[k8s-configuration] Update sourceControlConfiguration SDK Models to Track2 (Azure#3676)
* Create pull.yml * Update pull.yml * Update azure-pipelines.yml * Initial commit of k8s-extension * Update CODEOWNERS * Update azure-pipelines.yml * Create pull.yml * Update pull.yml * Update pull.yml * Update pipelines file * Update k8s-configuration name * Update test script params * Update pipeline file * Remove codeowners * Update pipelines file * Update CODEOWNERS * Update private preview pipelines * Remove open service mesh from public release * Update pipeline files * Update custom pipelines files * Add publish step to k8s-configuration * Update pipeline to publish extension * Update public extension pipeline * Change condition variable * Update pipeline naming * Add version to public preview/private preview * Update pipelines * Add different testing based on private branch * Add annotations to extension model * Update k8s-custom-pipelines.yml * Update SDKs with Updated Swagger Spec for 2020-07-01-preview (#13) * Update sdks with updated swagger spec * Update version and history rst * Reorder release history timeline * Fix ExtensionInstanceForCreate for import * remove py2 bdist support * Add custom table formatting * Remove unnecessary files * Fix style issues * Fix branch based on comments * Update identity piece manually * Don't handle defaults at the CLI level * Remove defaults from CLI client * Check null target namespace with namespace scope * Update style * Add cassandra operator and location to model * Stage Public Version of k8s-extension 0.2.0 for official release (#15) * Create pull.yml * Update pull.yml * Update azure-pipelines.yml * Initial commit of k8s-extension * Update pipelines file * Update CODEOWNERS * Update private preview pipelines * Remove open service mesh from public release * Update pipeline files * Update public extension pipeline * Change condition variable * Add version to public preview/private preview * Update pipelines * Add different testing based on private branch * Add annotations to extension model * Update k8s-custom-pipelines.yml * Update SDKs with Updated Swagger Spec for 2020-07-01-preview (#13) * Update sdks with updated swagger spec * Update version and history rst * Reorder release history timeline * Fix ExtensionInstanceForCreate for import * remove py2 bdist support * Add custom table formatting * Remove unnecessary files * Fix style issues * Fix branch based on comments * Update identity piece manually * Don't handle defaults at the CLI level * Remove defaults from CLI client * Check null target namespace with namespace scope * Update style * Add cassandra operator and location to model Co-authored-by: [email protected] <Action - Fork Sync> * Remove custom pipelines file * Update extension description, remove private const * Update pipeline file * Disable check ref docs * Disable refs docs * Update to include better create warning logs and remove update context (#20) * Update to include better create warning logs and remove update context * Remove help text for update * Fix spelling error * Update message * Fix k8s-extension conflict with private version * Fix style errors * Fix filename * add customization for microsoft.azureml.kubernetes (#23) * add customization for microsoft.azureml.kubernetes * Update release history Co-authored-by: Yue Yu <[email protected]> Co-authored-by: jonathan-innis <[email protected]> * Add E2E Testing from Separate branch into internal code (#26) * Add internal e2e testing * Change to testing folder * Inference CLI validation for Scoring FE (#24) * cli validation starter * added the call to the fe validation function * nodeport validation not required * test fix Co-authored-by: Jonathan Innis <[email protected]> * legal warning added (#27) * Remove deprecated method logger.warn * Update k8s-custom-pipelines.yml for Azure Pipelines * Update k8s-custom-pipelines.yml for Azure Pipelines * Add Azure Defender to E2E testing (#28) * Add azure defender testing to e2e * Remove the debug flag * Add configuration testing * Fix pipeline failures * Make test script more intuitive * Remove parameter from testing * Add some debug * Fix wrong location for k8s config whl * Fix pip install upgrade issue * Fix pip install upgrade issue * Add Check for Provider Registration and Refactor (#19) * Add check for provider registration and refactor * Fix bug in checking registration * Add license header to utils * Update private key check and error messaging * Update based on refactoring * Fix failing tests * Add provider registration check * Create a test for uppercase url, address comments * Add blank line to fix style check * Testing increase to ubuntu-latest * Update k8s-configuration Models to Track2 (#63) * Update models to track2 * Increase k8s-configuration version number * Update kind version * Change error to warning because of DSA failure * Remove unneeded files Co-authored-by: [email protected] <Action - Fork Sync> Co-authored-by: yuyue9284 <[email protected]> Co-authored-by: Yue Yu <[email protected]> Co-authored-by: Lia Kazakova <[email protected]>
1 parent 6d11a74 commit e298f57

27 files changed

+2033
-1201
lines changed

src/k8s-configuration/HISTORY.rst

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,14 @@
33
Release History
44
===============
55

6+
1.1.0
7+
++++++++++++++++++
8+
* Update sourceControlConfiguration resource models to Track2
9+
10+
1.0.1
11+
++++++++++++++++++
12+
* Add provider registration check
13+
614
1.0.0
715
++++++++++++++++++
816
* Support api-version 2021-03-01

src/k8s-configuration/azext_k8s_configuration/_client_factory.py

Lines changed: 7 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -3,13 +3,18 @@
33
# Licensed under the MIT License. See License.txt in the project root for license information.
44
# --------------------------------------------------------------------------------------------
55

6+
from azure.cli.core.commands.client_factory import get_mgmt_service_client
67

7-
def cf_k8s_configuration(cli_ctx, *_):
88

9-
from azure.cli.core.commands.client_factory import get_mgmt_service_client
9+
def cf_k8s_configuration(cli_ctx, *_):
1010
from azext_k8s_configuration.vendored_sdks import SourceControlConfigurationClient
1111
return get_mgmt_service_client(cli_ctx, SourceControlConfigurationClient)
1212

1313

1414
def cf_k8s_configuration_operation(cli_ctx, _):
1515
return cf_k8s_configuration(cli_ctx).source_control_configurations
16+
17+
18+
def _resource_providers_client(cli_ctx):
19+
from azure.mgmt.resource import ResourceManagementClient
20+
return get_mgmt_service_client(cli_ctx, ResourceManagementClient).providers
Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
# --------------------------------------------------------------------------------------------
2+
# Copyright (c) Microsoft Corporation. All rights reserved.
3+
# Licensed under the MIT License. See License.txt in the project root for license information.
4+
# --------------------------------------------------------------------------------------------
5+
6+
PROVIDER_NAMESPACE = 'Microsoft.KubernetesConfiguration'
7+
REGISTERED = "Registered"

src/k8s-configuration/azext_k8s_configuration/_params.py

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,7 @@
1313
)
1414

1515
from azure.cli.core.commands.validators import get_default_location_from_resource_group
16-
from ._validators import validate_configuration_type, validate_operator_namespace, validate_operator_instance_name
16+
from ._validators import _validate_configuration_type, _validate_operator_namespace, _validate_operator_instance_name
1717

1818

1919
def load_arguments(self, _):
@@ -38,7 +38,7 @@ def load_arguments(self, _):
3838
arg_type=get_enum_type(['namespace', 'cluster']),
3939
help='''Specify scope of the operator to be 'namespace' or 'cluster' ''')
4040
c.argument('configuration_type',
41-
validator=validate_configuration_type,
41+
validator=_validate_configuration_type,
4242
arg_type=get_enum_type(['sourceControlConfiguration']),
4343
help='Type of the configuration')
4444
c.argument('enable_helm_operator',
@@ -60,11 +60,11 @@ def load_arguments(self, _):
6060
c.argument('operator_instance_name',
6161
arg_group="Operator",
6262
help='Instance name of the Operator',
63-
validator=validate_operator_instance_name)
63+
validator=_validate_operator_instance_name)
6464
c.argument('operator_namespace',
6565
arg_group="Operator",
6666
help='Namespace in which to install the Operator',
67-
validator=validate_operator_namespace)
67+
validator=_validate_operator_namespace)
6868
c.argument('operator_type',
6969
arg_group="Operator",
7070
help='''Type of the operator. Valid value is 'flux' ''')
Lines changed: 61 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,61 @@
1+
# --------------------------------------------------------------------------------------------
2+
# Copyright (c) Microsoft Corporation. All rights reserved.
3+
# Licensed under the MIT License. See License.txt in the project root for license information.
4+
# --------------------------------------------------------------------------------------------
5+
6+
import base64
7+
from azure.cli.core.azclierror import MutuallyExclusiveArgumentError, InvalidArgumentValueError
8+
9+
10+
def _get_cluster_type(cluster_type):
11+
if cluster_type.lower() == 'connectedclusters':
12+
return 'Microsoft.Kubernetes'
13+
# Since cluster_type is an enum of only two values, if not connectedClusters, it will be managedClusters.
14+
return 'Microsoft.ContainerService'
15+
16+
17+
def _fix_compliance_state(config):
18+
# If we get Compliant/NonCompliant as compliance_sate, change them before returning
19+
if config.compliance_status.compliance_state.lower() == 'noncompliant':
20+
config.compliance_status.compliance_state = 'Failed'
21+
elif config.compliance_status.compliance_state.lower() == 'compliant':
22+
config.compliance_status.compliance_state = 'Installed'
23+
24+
return config
25+
26+
27+
def _get_data_from_key_or_file(key, filepath):
28+
if key != '' and filepath != '':
29+
raise MutuallyExclusiveArgumentError(
30+
'Error! Both textual key and key filepath cannot be provided',
31+
'Try providing the file parameter without providing the plaintext parameter')
32+
data = ''
33+
if filepath != '':
34+
data = _read_key_file(filepath)
35+
elif key != '':
36+
data = key
37+
return data
38+
39+
40+
def _read_key_file(path):
41+
try:
42+
with open(path, "r") as myfile: # user passed in filename
43+
data_list = myfile.readlines() # keeps newline characters intact
44+
data_list_len = len(data_list)
45+
if (data_list_len) <= 0:
46+
raise Exception("File provided does not contain any data")
47+
raw_data = ''.join(data_list)
48+
return _to_base64(raw_data)
49+
except Exception as ex:
50+
raise InvalidArgumentValueError(
51+
'Error! Unable to read key file specified with: {0}'.format(ex),
52+
'Verify that the filepath specified exists and contains valid utf-8 data') from ex
53+
54+
55+
def _from_base64(base64_str):
56+
return base64.b64decode(base64_str)
57+
58+
59+
def _to_base64(raw_data):
60+
bytes_data = raw_data.encode('utf-8')
61+
return base64.b64encode(bytes_data).decode('utf-8')

src/k8s-configuration/azext_k8s_configuration/_validators.py

Lines changed: 106 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -4,34 +4,48 @@
44
# --------------------------------------------------------------------------------------------
55

66
import re
7-
from azure.cli.core.azclierror import InvalidArgumentValueError
7+
import io
8+
from azure.cli.core.azclierror import InvalidArgumentValueError, MutuallyExclusiveArgumentError
9+
10+
from knack.log import get_logger
11+
from azext_k8s_configuration._client_factory import _resource_providers_client
12+
from azext_k8s_configuration._utils import _from_base64
13+
import azext_k8s_configuration._consts as consts
14+
from urllib.parse import urlparse
15+
from paramiko.hostkeys import HostKeyEntry
16+
from paramiko.ed25519key import Ed25519Key
17+
from paramiko.ssh_exception import SSHException
18+
from Crypto.PublicKey import RSA, ECC, DSA
19+
20+
21+
logger = get_logger(__name__)
822

923

1024
# Parameter-Level Validation
11-
def validate_configuration_type(configuration_type):
25+
def _validate_configuration_type(configuration_type):
1226
if configuration_type.lower() != 'sourcecontrolconfiguration':
1327
raise InvalidArgumentValueError(
1428
'Invalid configuration-type',
1529
'Try specifying the valid value "sourceControlConfiguration"')
1630

1731

18-
def validate_operator_namespace(namespace):
32+
def _validate_operator_namespace(namespace):
1933
if namespace.operator_namespace:
20-
__validate_k8s_name(namespace.operator_namespace, "--operator-namespace", 23)
34+
_validate_k8s_name(namespace.operator_namespace, "--operator-namespace", 23)
2135

2236

23-
def validate_operator_instance_name(namespace):
37+
def _validate_operator_instance_name(namespace):
2438
if namespace.operator_instance_name:
25-
__validate_k8s_name(namespace.operator_instance_name, "--operator-instance-name", 23)
39+
_validate_k8s_name(namespace.operator_instance_name, "--operator-instance-name", 23)
2640

2741

2842
# Create Parameter Validation
29-
def validate_configuration_name(configuration_name):
30-
__validate_k8s_name(configuration_name, "--name", 63)
43+
def _validate_configuration_name(configuration_name):
44+
_validate_k8s_name(configuration_name, "--name", 63)
3145

3246

3347
# Helper
34-
def __validate_k8s_name(param_value, param_name, max_len):
48+
def _validate_k8s_name(param_value, param_name, max_len):
3549
if len(param_value) > max_len:
3650
raise InvalidArgumentValueError(
3751
'Error! Invalid {0}'.format(param_name),
@@ -44,3 +58,86 @@ def __validate_k8s_name(param_value, param_name, max_len):
4458
raise InvalidArgumentValueError(
4559
'Error! Invalid {0}'.format(param_name),
4660
'Parameter {0} can only contain lowercase alphanumeric characters and hyphens'.format(param_name))
61+
62+
63+
def _validate_url_with_params(repository_url, ssh_private_key_set, known_hosts_contents_set, https_auth_set):
64+
scheme = urlparse(repository_url).scheme
65+
66+
if scheme.lower() in ('http', 'https'):
67+
if ssh_private_key_set:
68+
raise MutuallyExclusiveArgumentError(
69+
'Error! An --ssh-private-key cannot be used with an http(s) url',
70+
'Verify the url provided is a valid ssh url and not an http(s) url')
71+
if known_hosts_contents_set:
72+
raise MutuallyExclusiveArgumentError(
73+
'Error! --ssh-known-hosts cannot be used with an http(s) url',
74+
'Verify the url provided is a valid ssh url and not an http(s) url')
75+
if not https_auth_set and scheme == 'https':
76+
logger.warning('Warning! https url is being used without https auth params, ensure the repository '
77+
'url provided is not a private repo')
78+
else:
79+
if https_auth_set:
80+
raise MutuallyExclusiveArgumentError(
81+
'Error! https auth (--https-user and --https-key) cannot be used with a non-http(s) url',
82+
'Verify the url provided is a valid http(s) url and not an ssh url')
83+
84+
85+
def _validate_known_hosts(knownhost_data):
86+
try:
87+
knownhost_str = _from_base64(knownhost_data).decode('utf-8')
88+
except Exception as ex:
89+
raise InvalidArgumentValueError(
90+
'Error! ssh known_hosts is not a valid utf-8 base64 encoded string',
91+
'Verify that the string provided safely decodes into a valid utf-8 format') from ex
92+
lines = knownhost_str.split('\n')
93+
for line in lines:
94+
line = line.strip(' ')
95+
line_len = len(line)
96+
if (line_len == 0) or (line[0] == "#"):
97+
continue
98+
try:
99+
host_key = HostKeyEntry.from_line(line)
100+
if not host_key:
101+
raise Exception('not enough fields found in known_hosts line')
102+
except Exception as ex:
103+
raise InvalidArgumentValueError(
104+
'Error! ssh known_hosts provided in wrong format',
105+
'Verify that all lines in the known_hosts contents are provided in a valid sshd(8) format') from ex
106+
107+
108+
def _validate_private_key(ssh_private_key_data):
109+
try:
110+
RSA.import_key(_from_base64(ssh_private_key_data))
111+
return
112+
except ValueError:
113+
try:
114+
ECC.import_key(_from_base64(ssh_private_key_data))
115+
return
116+
except ValueError:
117+
try:
118+
DSA.import_key(_from_base64(ssh_private_key_data))
119+
return
120+
except ValueError:
121+
try:
122+
key_obj = io.StringIO(_from_base64(ssh_private_key_data).decode('utf-8'))
123+
Ed25519Key(file_obj=key_obj)
124+
return
125+
except SSHException:
126+
raise InvalidArgumentValueError(
127+
'Error! --ssh-private-key provided in invalid format',
128+
'Verify the key provided is a valid PEM-formatted key of type RSA, ECC, DSA, or Ed25519')
129+
130+
131+
# pylint: disable=broad-except
132+
def _validate_cc_registration(cmd):
133+
try:
134+
rp_client = _resource_providers_client(cmd.cli_ctx)
135+
registration_state = rp_client.get(consts.PROVIDER_NAMESPACE).registration_state
136+
137+
if registration_state.lower() != consts.REGISTERED.lower():
138+
logger.warning("'Source Control Configuration' cannot be used because '%s' provider has not been "
139+
"registered. More details for registering this provider can be found here - "
140+
"https://aka.ms/RegisterKubernetesConfigurationProvider", consts.PROVIDER_NAMESPACE)
141+
except Exception:
142+
logger.warning("Unable to fetch registration state of '%s' provider. "
143+
"Failed to enable 'source control configuration' feature...", consts.PROVIDER_NAMESPACE)

0 commit comments

Comments
 (0)