Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

az connectedk8s connect has been failed #2077

Closed
Dongbumlee opened this issue Jul 28, 2020 · 10 comments
Closed

az connectedk8s connect has been failed #2077

Dongbumlee opened this issue Jul 28, 2020 · 10 comments
Assignees
@akashkeshari @RakeshMohanMSFT
Labels
Connected Kubernetes Service Attention This issue is responsible by Azure service team.

Comments

@Dongbumlee
Copy link

This is autogenerated. Please review and update as needed.

Describe the bug

Command Name
az connectedk8s connect Extension Name: connectedk8s. Version: 0.2.1.

Errors:

cli.azure.cli.core.util : The command failed with an unexpected error. Here is the traceback:

The command failed with an unexpected error. Here is the traceback:

cli.azure.cli.core.util : Expecting value: line 1 column 1 (char 0)
Traceback (most recent call last):
  File "C:\Users\VSSADM~1\AppData\Local\Temp\pip-install-_kvuguic\knack\knack\cli.py", line 215, in invoke
  File "D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure\cli\core\commands\__init__.py", line 654, in execute
  File "D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure\cli\core\commands\__init__.py", line 718, in _run_jobs_serially
  File "D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure\cli\core\commands\__init__.py", line 711, in _run_job
  File "C:\Users\VSSADM~1\AppData\Local\Temp\pip-install-_kvuguic\six\six.py", line 703, in reraise
  File "D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure\cli\core\commands\__init__.py", line 688, in _run_job
  File "D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure\cli\core\commands\__init__.py", line 325, in __call__
  File "D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure\cli\core\__init__.py", line 545, in default_command_handler
  File "C:\Users\donlee\.azure\cliextensions\connectedk8s\azext_connectedk8s\custom.py", line 209, in create_connectedk8s
  File "C:\Users\donlee\.azure\cliextensions\connectedk8s\azext_connectedk8s\custom.py", line 408, in get_helm_registry
    telemetry.set_exception(exception=str(response.json()), fault_type=Get_HelmRegistery_Path_Fault_Type,
  File "C:\Users\VSSADM~1\AppData\Local\Temp\pip-install-_kvuguic\requests\requests\models.py", line 897, in json
  File "C:\Program Files (x86)\Microsoft SDKs\Azure\CLI2\lib\json\__init__.py", line 354, in loads
    return _default_decoder.decode(s)
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "C:\Program Files (x86)\Microsoft SDKs\Azure\CLI2\lib\json\decoder.py", line 357, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
Expecting value: line 1 column 1 (char 0)
Traceback (most recent call last):
  File "C:\Users\VSSADM~1\AppData\Local\Temp\pip-install-_kvuguic\knack\knack\cli.py", line 215, in invoke
  File "D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure\cli\core\commands\__init__.py", line 654, in execute
  File "D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure\cli\core\commands\__init__.py", line 718, in _run_jobs_serially
  File "D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure\cli\core\commands\__init__.py", line 711, in _run_job
  File "C:\Users\VSSADM~1\AppData\Local\Temp\pip-install-_kvuguic\six\six.py", line 703, in reraise
  File "D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure\cli\core\commands\__init__.py", line 688, in _run_job
  File "D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure\cli\core\commands\__init__.py", line 325, in __call__
  File "D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure\cli\core\__init__.py", line 545, in default_command_handler
  File "C:\Users\donlee\.azure\cliextensions\connectedk8s\azext_connectedk8s\custom.py", line 209, in create_connectedk8s
    registry_path = os.getenv('HELMREGISTRY') if os.getenv('HELMREGISTRY') else get_helm_registry(profile, location)
  File "C:\Users\donlee\.azure\cliextensions\connectedk8s\azext_connectedk8s\custom.py", line 408, in get_helm_registry
    telemetry.set_exception(exception=str(response.json()), fault_type=Get_HelmRegistery_Path_Fault_Type,
  File "C:\Users\VSSADM~1\AppData\Local\Temp\pip-install-_kvuguic\requests\requests\models.py", line 897, in json
  File "C:\Program Files (x86)\Microsoft SDKs\Azure\CLI2\lib\json\__init__.py", line 354, in loads
    return _default_decoder.decode(s)
  File "C:\Program Files (x86)\Microsoft SDKs\Azure\CLI2\lib\json\decoder.py", line 339, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "C:\Program Files (x86)\Microsoft SDKs\Azure\CLI2\lib\json\decoder.py", line 357, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

To Reproduce:

Steps to reproduce the behavior. Note that argument values have been redacted, as they may contain sensitive information.

  • Put any pre-requisite steps here...
  • az connectedk8s connect --name {} --resource-group {} -l {} --debug

Expected Behavior

Environment Summary

Windows-10-10.0.19041-SP0
Python 3.6.6
Installer: MSI

azure-cli 2.7.0 *

Extensions:
connectedk8s 0.2.1
k8sconfiguration 0.1.8

Additional Context
@ghost ghost added needs-triage This is a new issue that needs to be triaged to the appropriate team. question The issue doesn't require a change to the product in order to be resolved. Most issues start as that labels Jul 28, 2020
@Dongbumlee
Copy link
Author

After setting HELM registry URL by manually, it works.

SET HELMREGISTRY=azurearcfork8s.azurecr.io/batch1/stable/azure-arc-k8sagents:0.2.4

@ghost ghost removed the needs-triage This is a new issue that needs to be triaged to the appropriate team. label Jul 28, 2020
@arrownj
Copy link
Contributor

arrownj commented Jul 28, 2020

Hi @akashkeshari , could you please help with this issue ? Thanks

@akashkeshari
Copy link
Contributor

Hi @Dongbumlee

Can you run the following command and share the output so that we can look into the issue:
'az rest -m post -u https://eastus.dp.kubernetesconfiguration.azure.com/azure-arc-k8sagents/GetLatestHelmPackagePath?api-version=2019-11-01-preview --resource https://management.core.windows.net/ --debug'

To unblock you, run the following commands:

  1. set HELMREGISTRY=azurearcfork8s.azurecr.io/batch1/stable/azure-arc-k8sagents:0.2.4
  2. az connectedk8s connect --name {} --resource-group {}
    Let me know if you face any issues following above steps.

Thanks,
Akash

@yonzhan yonzhan added the Service Attention This issue is responsible by Azure service team. label Jul 28, 2020
@ghost
Copy link

ghost commented Jul 28, 2020

Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @akashkeshari.

@yonzhan yonzhan removed the question The issue doesn't require a change to the product in order to be resolved. Most issues start as that label Jul 28, 2020
@Dongbumlee
Copy link
Author

Dongbumlee commented Jul 28, 2020 via email

@ooeygui
Copy link

ooeygui commented Aug 19, 2020 •
edited
Loading

I am hitting the same issue. The workaround does not work for me.

Environment Summary

Linux-4.15.0-112-generic-x86_64-with-debian-buster-sid
Python 3.6.10
Installer: DEB

azure-cli 2.10.1

Extensions:
k8sconfiguration 0.1.8
connectedk8s 0.2.3

The output:


az rest -m post -u https://eastus.dp.kubernetesconfiguration.azure.com/azure-arc-k8sagents/GetLatestHelmPackagePath?api-version=2019-11-01-preview --resource https://management.core.windows.net/ --debug
Command arguments: ['rest', '-m', 'post', '-u', 'https://eastus.dp.kubernetesconfiguration.azure.com/azure-arc-k8sagents/GetLatestHelmPackagePath?api-version=2019-11-01-preview', '--resource', 'https://management.core.windows.net/', '--debug']
Event: Cli.PreExecute []
Event: CommandParser.OnGlobalArgumentsCreate [<function CLILogging.on_global_arguments at 0x7f0d6ef65f28>, <function OutputProducer.on_global_arguments at 0x7f0d6ed3af28>, <function CLIQuery.on_global_arguments at 0x7f0d6eae5268>]
Event: CommandInvoker.OnPreCommandTableCreate []
Modules found from index for 'rest': ['azure.cli.command_modules.util']
Loading command modules:
Name                  Load Time    Groups  Commands
util                      0.148         1         2
Total (1)                 0.148         1         2
These extensions are not installed and will be skipped: ['azext_ai_examples', 'azext_ai_did_you_mean_this']
Loading extensions:
Name                  Load Time    Groups  Commands  Directory
Total (0)                 0.000         0         0
Loaded 1 groups, 2 commands.
Found a match in the command table.
Raw command  : rest
Command table: rest
Event: CommandInvoker.OnPreCommandTableTruncate [<function AzCliLogging.init_command_file_logging at 0x7f0d6e47e048>]
az_command_data_logger : command args: rest -m {} -u {} --resource {} --debug
metadata file logging enabled - writing logs to '/home/ubuntu/.azure/commands'.
Event: CommandInvoker.OnPreArgumentLoad [<function register_global_subscription_argument.<locals>.add_subscription_parameter at 0x7f0d6e4a2378>]
Event: CommandInvoker.OnPostArgumentLoad []
Event: CommandInvoker.OnPostCommandTableCreate [<function register_ids_argument.<locals>.add_ids_arguments at 0x7f0d6e458950>, <function register_cache_arguments.<locals>.add_cache_arguments at 0x7f0d6e458a60>]
Event: CommandInvoker.OnCommandTableLoaded []
Event: CommandInvoker.OnPreParseArgs []
Event: CommandInvoker.OnPostParseArgs [<function OutputProducer.handle_output_argument at 0x7f0d6ed43048>, <function CLIQuery.handle_query_parameter at 0x7f0d6eae52f0>, <function register_ids_argument.<locals>.parse_ids_arguments at 0x7f0d6e4589d8>]
Retrieving token for resource https://management.core.windows.net/
attempting to read file /home/ubuntu/.azure/accessTokens.json as utf-8-sig
adal-python : 4b98fc59-296b-409f-b8de-ed0ace3a07e1 - Authority:Performing instance discovery: ...
adal-python : 4b98fc59-296b-409f-b8de-ed0ace3a07e1 - Authority:Performing static instance discovery
adal-python : 4b98fc59-296b-409f-b8de-ed0ace3a07e1 - Authority:Authority validated via static instance discovery
adal-python : 4b98fc59-296b-409f-b8de-ed0ace3a07e1 - TokenRequest:Getting token from cache with refresh if necessary.
adal-python : 4b98fc59-296b-409f-b8de-ed0ace3a07e1 - CacheDriver:finding with query keys: {'_clientId': '...', 'userId': '...'}
adal-python : 4b98fc59-296b-409f-b8de-ed0ace3a07e1 - CacheDriver:Looking for potential cache entries: {'_clientId': '...', 'userId': '...'}
adal-python : 4b98fc59-296b-409f-b8de-ed0ace3a07e1 - CacheDriver:Found 4 potential entries.
adal-python : 4b98fc59-296b-409f-b8de-ed0ace3a07e1 - CacheDriver:Resource specific token found.
adal-python : 4b98fc59-296b-409f-b8de-ed0ace3a07e1 - CacheDriver:Returning token from cache lookup, AccessTokenId: b'xxx', RefreshTokenId: b'xxx'
Request URL: 'https://eastus.dp.kubernetesconfiguration.azure.com/azure-arc-k8sagents/GetLatestHelmPackagePath?api-version=2019-11-01-preview'
Request method: 'POST'
Request headers:
    'User-Agent': 'AZURECLI/2.10.1 (DEB)'
    'Accept-Encoding': 'gzip, deflate'
    'Accept': '*/*'
    'Connection': 'keep-alive'
    'x-ms-client-request-id': 'xxx'
    'CommandName': 'rest'
    'ParameterSetName': '-m -u --resource --debug'
    'Authorization': 'Bearer xxx...'
    'Content-Length': '0'
Request body:
None
urllib3.connectionpool : Starting new HTTPS connection (1): eastus.dp.kubernetesconfiguration.azure.com:443
urllib3.connectionpool : https://eastus.dp.kubernetesconfiguration.azure.com:443 "POST /azure-arc-k8sagents/GetLatestHelmPackagePath?api-version=2019-11-01-preview HTTP/1.1" 200 86
Response status: 200
Response headers:
    'Server': 'openresty/1.15.8.2'
    'Date': 'Wed, 19 Aug 2020 21:14:50 GMT'
    'Content-Type': 'application/json; charset=utf-8'
    'Content-Length': '86'
    'Connection': 'close'
    'Strict-Transport-Security': 'max-age=15724800; includeSubDomains, max-age=2592000'
    'api-supported-versions': '2019-11-01-Preview'
    'x-content-type-options': 'nosniff'
Response content:
{"repositoryPath":"azurearcfork8s.azurecr.io/batch1/stable/azure-arc-k8sagents:0.2.9"}
Event: CommandInvoker.OnTransformResult [<function _resource_group_transform at 0x7f0d6e4512f0>, <function _x509_from_base64_to_hex_transform at 0x7f0d6e451378>]
Event: CommandInvoker.OnFilterResult []
{
  "repositoryPath": "azurearcfork8s.azurecr.io/batch1/stable/azure-arc-k8sagents:0.2.9"
}
Event: Cli.PostExecute [<function AzCliLogging.deinit_cmd_metadata_logging at 0x7f0d6e47e268>]
az_command_data_logger : exit code: 0
Command ran in 1.310 seconds (init: 0.172, invoke: 1.137)
telemetry.save : Save telemetry record of length 2626 in cache

Then set the environment based on that path:

export HELMREPOSITORY=azurearcfork8s.azurecr.io/batch1/stable/azure-arc-k8sagents:0.2.9

The command failed with an unexpected error. Here is the traceback:

Expecting value: line 1 column 1 (char 0)
Traceback (most recent call last):
  File "/opt/az/lib/python3.6/site-packages/knack/cli.py", line 215, in invoke
    cmd_result = self.invocation.execute(args)
  File "/opt/az/lib/python3.6/site-packages/azure/cli/core/commands/__init__.py", line 654, in execute
    raise ex
  File "/opt/az/lib/python3.6/site-packages/azure/cli/core/commands/__init__.py", line 718, in _run_jobs_serially
    results.append(self._run_job(expanded_arg, cmd_copy))
  File "/opt/az/lib/python3.6/site-packages/azure/cli/core/commands/__init__.py", line 711, in _run_job
    six.reraise(*sys.exc_info())
  File "/opt/az/lib/python3.6/site-packages/six.py", line 703, in reraise
    raise value
  File "/opt/az/lib/python3.6/site-packages/azure/cli/core/commands/__init__.py", line 688, in _run_job
    result = cmd_copy(params)
  File "/opt/az/lib/python3.6/site-packages/azure/cli/core/commands/__init__.py", line 325, in __call__
    return self.handler(*args, **kwargs)
  File "/opt/az/lib/python3.6/site-packages/azure/cli/core/__init__.py", line 779, in default_command_handler
    return op(**command_args)
  File "/home/ubuntu/.azure/cliextensions/connectedk8s/azext_connectedk8s/custom.py", line 109, in create_connectedk8s
    release_namespace = get_release_namespace(kube_config, kube_context)
  File "/home/ubuntu/.azure/cliextensions/connectedk8s/azext_connectedk8s/custom.py", line 540, in get_release_namespace
    output_helm_release = json.loads(output_helm_release)
  File "/opt/az/lib/python3.6/json/__init__.py", line 354, in loads
    return _default_decoder.decode(s)
  File "/opt/az/lib/python3.6/json/decoder.py", line 339, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/opt/az/lib/python3.6/json/decoder.py", line 357, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

Debugging the script, it looks like:

def get_release_namespace(kube_config, kube_context):

it looks like it is attempting to enumerate the helm repo which returns an empty array, which the json parser fails on.

$ helm list -a --all-namespaces --output json
[]

@akashkeshari
Copy link
Contributor

@ooeygui ,

Can you please share the version of helm installed on your machine?

@ooeygui
Copy link

ooeygui commented Aug 23, 2020

@akashkeshari Thanks for the reply - I'm using helm3 installed using microk8s enable helm3 with Ubuntu 18.04 on amd64. Furthermore, I set an alias for helm3 & kubectl

sudo snap alias microk8s.helm3 helm3
sudo snap alias microk8s.kubectl kubectl

helm3 version
version.BuildInfo{Version:"v3.0.2", GitCommit:"19e47ee3283ae98139d98460de796c1be1e3975f", GitTreeState:"clean", GoVersion:"go1.13.5"}

@akashkeshari
Copy link
Contributor

@Dongbumlee ,

This issue has been fixed. Please update the connectedk8s extension to version 0.2.4 and try again:
Update using this cmd: 'az extension update -n connectedk8s'

Let us know if you still face this issue.

@ooeygui ,

I think we already had a discussion over email. You may also switch to the new version of connectedk8s (0.2.4).

@ooeygui
Copy link

ooeygui commented Aug 27, 2020

@akashkeshari Thank you - this specific issue no longer manifests for me with your fixes.

@RakeshMohanMSFT RakeshMohanMSFT self-assigned this Nov 11, 2022
@RakeshMohanMSFT RakeshMohanMSFT mentioned this issue Nov 11, 2022
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@akashkeshari akashkeshari

@RakeshMohanMSFT RakeshMohanMSFT

Labels
Connected Kubernetes Service Attention This issue is responsible by Azure service team.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
6 participants
@arrownj @ooeygui @Dongbumlee @akashkeshari @RakeshMohanMSFT @yonzhan