Unauthorized publish/add using keyring with GCP Artifact Registry

[jasonphillipcarpenter]

• I am on the latest Poetry version.
• I have searched the issues of this repo and believe that this is not a duplicate.
• If an exception occurs when executing a command, I executed it again in debug mode (-vvv option).

• OS version and name: macOS Big Sur version 11.4 on both M1 and Intel.
• Poetry version: 1.1.8
• Link of a Gist with the contents of your pyproject.toml file: https://gist.github.com/boweevil/26400a05a76e5372eceff49f3fbabe9e

Issue

When trying to publish a package to or add a package from a private GCP Artifact Registry using keyring, I get a 401 unauthorized error. I have followed this guide as closely as possible in an attempt to adapt it to the poetry use case. I am able to publish using twine and pull using pip. In poetry's documentation it talks about supporting keyring. However, it seems this may only be in regards to storing and retrieving username and password. I am prompted for a username only when publishing. I have hit enter at this prompt as I'm trying to use the Google stored credentials in the keyring.

Poetry publish

poetry publish -r example-gcp-registry -vvv

Username:
Publishing example_package (0.1.0) to example-gcp-registry
 - Uploading example_package-0.1.0.tar.gz 100%

  Stack trace:

  7  ~/Library/Application Support/pypoetry/venv/lib/python3.9/site-packages/clikit/console_application.py:131 in run
      129│             parsed_args = resolved_command.args
      130│
    → 131│             status_code = command.handle(parsed_args, io)
      132│         except KeyboardInterrupt:
      133│             status_code = 1

  6  ~/Library/Application Support/pypoetry/venv/lib/python3.9/site-packages/clikit/api/command/command.py:120 in handle
      118│     def handle(self, args, io):  # type: (Args, IO) -> int
      119│         try:
    → 120│             status_code = self._do_handle(args, io)
      121│         except KeyboardInterrupt:
      122│             if io.is_debug():

  5  ~/Library/Application Support/pypoetry/venv/lib/python3.9/site-packages/clikit/api/command/command.py:171 in _do_handle
      169│         handler_method = self._config.handler_method
      170│
    → 171│         return getattr(handler, handler_method)(args, io, self)
      172│
      173│     def __repr__(self):  # type: () -> str

  4  ~/Library/Application Support/pypoetry/venv/lib/python3.9/site-packages/cleo/commands/command.py:92 in wrap_handle
       90│         self._command = command
       91│
    →  92│         return self.handle()
       93│
       94│     def handle(self):  # type: () -> Optional[int]

  3  ~/Library/Application Support/pypoetry/venv/lib/python3.9/site-packages/poetry/console/commands/publish.py:77 in handle
      75│         )
      76│
    → 77│         publisher.publish(
      78│             self.option("repository"),
      79│             self.option("username"),

  2  ~/Library/Application Support/pypoetry/venv/lib/python3.9/site-packages/poetry/publishing/publisher.py:93 in publish
      91│         )
      92│
    → 93│         self._uploader.upload(
      94│             url,
      95│             cert=cert or get_cert(self._poetry.config, repository_name),

  1  ~/Library/Application Support/pypoetry/venv/lib/python3.9/site-packages/poetry/publishing/uploader.py:119 in upload
      117│
      118│         try:
    → 119│             self._upload(session, url, dry_run)
      120│         finally:
      121│             session.close()

  UploadError

  HTTP Error 401: Unauthorized

  at ~/Library/Application Support/pypoetry/venv/lib/python3.9/site-packages/poetry/publishing/uploader.py:216 in _upload
      212│                     self._register(session, url)
      213│                 except HTTPError as e:
      214│                     raise UploadError(e)
      215│
    → 216│             raise UploadError(e)
      217│
      218│     def _do_upload(
      219│         self, session, url, dry_run=False
      220│     ):  # type: (requests.Session, str, Optional[bool]) -> None

Poetry add

poetry add --source example-gcp-registry example_package -vvv
Using virtualenv: /Users/user/Library/Caches/pypoetry/virtualenvs/example_package-LmoIvwng-py3.9

  Stack trace:

  10  ~/Library/Application Support/pypoetry/venv/lib/python3.9/site-packages/clikit/console_application.py:131 in run
       129│             parsed_args = resolved_command.args
       130│
     → 131│             status_code = command.handle(parsed_args, io)
       132│         except KeyboardInterrupt:
       133│             status_code = 1

   9  ~/Library/Application Support/pypoetry/venv/lib/python3.9/site-packages/clikit/api/command/command.py:120 in handle
       118│     def handle(self, args, io):  # type: (Args, IO) -> int
       119│         try:
     → 120│             status_code = self._do_handle(args, io)
       121│         except KeyboardInterrupt:
       122│             if io.is_debug():

   8  ~/Library/Application Support/pypoetry/venv/lib/python3.9/site-packages/clikit/api/command/command.py:171 in _do_handle
       169│         handler_method = self._config.handler_method
       170│
     → 171│         return getattr(handler, handler_method)(args, io, self)
       172│
       173│     def __repr__(self):  # type: () -> str

   7  ~/Library/Application Support/pypoetry/venv/lib/python3.9/site-packages/cleo/commands/command.py:92 in wrap_handle
        90│         self._command = command
        91│
     →  92│         return self.handle()
        93│
        94│     def handle(self):  # type: () -> Optional[int]

   6  ~/Library/Application Support/pypoetry/venv/lib/python3.9/site-packages/poetry/console/commands/add.py:106 in handle
       104│             return 0
       105│
     → 106│         requirements = self._determine_requirements(
       107│             packages,
       108│             allow_prereleases=self.option("allow-prereleases"),

   5  ~/Library/Application Support/pypoetry/venv/lib/python3.9/site-packages/poetry/console/commands/init.py:328 in _determine_requirements
       326│             elif "version" not in requirement:
       327│                 # determine the best version automatically
     → 328│                 name, version = self._find_best_version_for_package(
       329│                     requirement["name"],
       330│                     allow_prereleases=allow_prereleases,

   4  ~/Library/Application Support/pypoetry/venv/lib/python3.9/site-packages/poetry/console/commands/init.py:361 in _find_best_version_for_package
       359│
       360│         selector = VersionSelector(self._get_pool())
     → 361│         package = selector.find_best_candidate(
       362│             name, required_version, allow_prereleases=allow_prereleases, source=source
       363│         )

   3  ~/Library/Application Support/pypoetry/venv/lib/python3.9/site-packages/poetry/version/version_selector.py:32 in find_best_candidate
       30│             },
       31│         )
     → 32│         candidates = self._pool.find_packages(dependency)
       33│         only_prereleases = all([c.version.is_prerelease() for c in candidates])
       34│

   2  ~/Library/Application Support/pypoetry/venv/lib/python3.9/site-packages/poetry/repositories/pool.py:166 in find_packages
       164│
       165│         if repository is not None and not self._ignore_repository_names:
     → 166│             return self.repository(repository).find_packages(dependency)
       167│
       168│         packages = []

   1  ~/Library/Application Support/pypoetry/venv/lib/python3.9/site-packages/poetry/repositories/legacy_repository.py:264 in find_packages
       262│             versions = self._cache.store("matches").get(key)
       263│         else:
     → 264│             page = self._get("/{}/".format(dependency.name.replace(".", "-")))
       265│             if page is None:
       266│                 return []

  RepositoryError

  401 Client Error: Unauthorized for url: https://us-east1-python.pkg.dev/example_project/example-gcp-registry/simple/example_package/

  at ~/Library/Application Support/pypoetry/venv/lib/python3.9/site-packages/poetry/repositories/legacy_repository.py:393 in _get
      389│             if response.status_code == 404:
      390│                 return
      391│             response.raise_for_status()
      392│         except requests.HTTPError as e:
    → 393│             raise RepositoryError(e)
      394│
      395│         if response.status_code in (401, 403):
      396│             self._log(
      397│                 "Authorization error accessing {url}".format(url=response.url),

[pymak3r]

I've been able to get this to work (most of the time) by setting the keyring service poetry looks for when using a 'legacy repository'.

First, you have to add the repository to config:
poetry config repositories.<name of repository>

then you can set the keyring service for that repository:
gcloud auth print-access-token |keyring set poetry-repository-<name of repository> oauth2accesstoken

Note: <name of repository> has to match.

Really hoping we have an update for this soon.

[jasonphillipcarpenter]

Unfortunately, I get the same 401 errors for publish and add using the method you shared, @jaycarroll. Thank you for sharing though. Perhaps I have something additional misconfigured.

[caseydialpad]

@boweevil

Keyring authentication is handled with the following PR #4086 (slated for release with version 1.2.0).

As for the 401 error to pull, you would use the GCP URL url = "https://us-east1-python.pkg.dev/example_project/example_gcp_registry/simple" and to push you would use url = "https://us-east1-python.pkg.dev/example_project/example_gcp_registry/".

Let me know if that helps. I am unaware a technique beyond defining two separate repositories for supporting publish and pull on different URLs to the same repository. If you have a solution, I am all ears.

[jasonphillipcarpenter]

It looks like this is solved in version 1.2.0a2. So, I agree that 1.2.0 will resolve the issue. Thank you for the follow up!

[neersighted]

Closed as #4086 is merged.

[github-actions]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.