Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Authentication from poetry.toml doesn't work on 1.1.x #3216

Closed
3 tasks done
jairhenrique opened this issue Oct 15, 2020 · 41 comments
Closed
3 tasks done

Authentication from poetry.toml doesn't work on 1.1.x #3216

jairhenrique opened this issue Oct 15, 2020 · 41 comments
Labels
kind/bug Something isn't working as expected status/triage This issue needs to be triaged

Comments

@jairhenrique
Copy link
Contributor

  • I am on the latest Poetry version.

  • I have searched the issues of this repo and believe that this is not a duplicate.

  • If an exception occurs when executing a command, I executed it again in debug mode (-vvv option).

  • OS version and name: MacOS 10.15.7

  • Poetry version: 1.1.3

Issue

After upgrade from 1.0.10 to 1.1.3 the authentication for private repositories doesn't work.

I have an file poetry.toml with this content:

[repositories]
[repositories.MyPrivateFeed]
url = "https://my-private-repo/pypi/simple/"

[http-basic]
[http-basic.MyPrivateFeed]
username = "user"
password = "pass"

And on pyproject.toml

  [[tool.poetry.source]]
  name = "MyPrivateFeed"
  url = "https://my-private-repo/pypi/simple/"
  secondary = true

When I try to update/lock/install I get an error:

update

poetry update

Updating dependencies
Resolving dependencies... (0.4s)

  RepositoryError

  401 Client Error: Unauthorized for url: https://my-private-repo/pypi/simple/mypy/

  at ~/.pyenv/versions/3.8.5/envs/my-project/lib/python3.8/site-packages/poetry/repositories/legacy_repository.py:390 in _get
      386│             if response.status_code == 404:
      387│                 return
      388│             response.raise_for_status()
      389│         except requests.HTTPError as e:
    → 390│             raise RepositoryError(e)
      391│
      392│         if response.status_code in (401, 403):
      393│             self._log(
      394│                 "Authorization error accessing {url}".format(url=url), level="warn"

lock

poetry lock

Updating dependencies
Resolving dependencies... (0.4s)

  RepositoryError

  401 Client Error: Unauthorized for url: https://my-private-repo/pypi/simple/mypy/

  at ~/.pyenv/versions/3.8.5/envs/my-project/lib/python3.8/site-packages/poetry/repositories/legacy_repository.py:390 in _get
      386│             if response.status_code == 404:
      387│                 return
      388│             response.raise_for_status()
      389│         except requests.HTTPError as e:
    → 390│             raise RepositoryError(e)
      391│
      392│         if response.status_code in (401, 403):
      393│             self._log(
      394│                 "Authorization error accessing {url}".format(url=url), level="warn"
@jairhenrique jairhenrique added kind/bug Something isn't working as expected status/triage This issue needs to be triaged labels Oct 15, 2020
@zcherries
Copy link

I'm also experiencing this same issue

@abn
Copy link
Member

abn commented Oct 16, 2020

Not relevant to the issue, but note that the repositories.MyPrivateFeed configuration is for publishing, so /simple/ url will not work for that, you are better off removing that if you do not publishing it.

@jairhenrique for sanity,

  1. Does curl https://user:pass@my-private-repo/pypi/simple/mypy/ work as expected?
  2. Can you attempt this inside a container or a clean environment without any environment variables or existing configuration.
  3. Are there anyother repositories with the same netloc configured with a different credential?

Also, just realised, you store credentials in your poetry.toml file? I do not think this is loaded for credentials. Try,

poetry config http-basic.MyPrivateFeed user pass

@jairhenrique
Copy link
Contributor Author

Not relevant to the issue, but note that the repositories.MyPrivateFeed configuration is for publishing, so /simple/ url will not work for that, you are better off removing that if you do not publishing it.

@jairhenrique for sanity,

1. Does `curl https://user:pass@my-private-repo/pypi/simple/mypy/` work as expected?

2. Can you attempt this inside a container or a clean environment without any environment variables or existing configuration.

3. Are there anyother repositories with the same netloc configured with a different credential?

Also, just realised, you store credentials in your poetry.toml file? I do not think this is loaded for credentials. Try,

poetry config http-basic.MyPrivateFeed user pass

@abn

  1. Yes, calling curl it works.
  2. My env is clean, I don't set any poetry environment variable.
  3. No.

I just ran poetry config http-basic.MyPrivateFeed user pass but it no make effect.
This command should create an environment variable?

@abn
Copy link
Member

abn commented Oct 16, 2020

2. My env is clean, I don't set any poetry environment variable.

Environment variables, pre-existing configuration files (~/.config/pypoetry) etc. can have an impact.

This command should create an environment variable?

https://python-poetry.org/docs/repositories/#configuring-credentials

This will either store the credentials in your system keyring or to a file at ~/.config/pypoetry/auth.toml. Can you try removing the poetry.toml file and using just the global config please/

@jairhenrique
Copy link
Contributor Author

  1. My env is clean, I don't set any poetry environment variable.

Environment variables, pre-existing configuration files (~/.config/pypoetry) etc. can have an impact.

This command should create an environment variable?

https://python-poetry.org/docs/repositories/#configuring-credentials

This will either store the credentials in your system keyring or to a file at ~/.config/pypoetry/auth.toml. Can you try removing the poetry.toml file and using just the global config please/

@abn

First test

  • Delete poetry.toml from source folder.
  • Delete key from keyring.
  • Run poetry config http-basic.MyFeed user pass
  • poetry lock -> error:
RepositoryError

  401 Client Error: Unauthorized for url: https://my-feed/pypi/simple/mypy/

Second test

  • Delete poetry.toml from source folder.
  • pip uninstall keyring
  • Run poetry config http-basic.MyFeed user pass with output:
Using a plaintext file to store and retrieve credentials
Using a plaintext file to store and retrieve credentials
  • ls ~/.config/pypoetry with output:
ls: /Users/jairhenrique/.config/pypoetry: No such file or directory
  • poetry lock -> error:
RepositoryError

  401 Client Error: Unauthorized for url: https://my-feed/pypi/simple/mypy/

@FinchPowers
Copy link

A temporary solution that works for me is adding the private repository via

poetry config repositories.<name> <url>

@ayalganem55
Copy link

I had the same issue, it works for me when I do the following steps:

  1. In the file pyproject.toml remove the word simple from the url in [[tool.poetry.source]]
  2. make sure that you run poetry config repositories <REPOSITORY_NAME> <URL> where url is the same one that you set in [[tool.poetry.source]] (without the word simple)

@mrksr
Copy link

mrksr commented Dec 3, 2020

Another data point: I could not get @ayalganem55's steps to work. But what seemed to do the trick was to set default = true for the private source. In our case, the private source is an Artifactory which includes a pypi mirror. Preliminary testing suggests that neither setting poetry config repositories nor removing simple from the URL are required as long as default = true is set. I'm using poetry version 1.1.4.

In pyproject.toml:

[[tool.poetry.source]]
name = "mysource"
url = "https://artifactory.com/artifactory/api/pypi/pypi/simple"
default = true

Local configuration:
poetry config http-basic.mysource <user> <password>

@jairhenrique
Copy link
Contributor Author

@abn I execute poetry config --unset repositories.OtherRepositor and it works.

The process to build config merge this config with my poetry.toml but this only this first config is used on install process.

@DustinMoriarty
Copy link
Contributor

@jairhenrique : I see this is closed. However, I am unclear on the conclusion. I am seeing this on poetry 1.1.6. I don't see any linked PR's or anything in the comments to indicate resolution. Can you help point me in the right direction?

@DustinMoriarty
Copy link
Contributor

By the way, in my case the repo should not even be requiring authorization. I can reach it with curl just fine without any auth. I need auth to publish to it but not to pull. The poetry config really shouldn't even be used in this case.

@marcosborges
Copy link

marcosborges commented Apr 23, 2021

@abn

I'm using Azure's private artifact servers. I managed to publish without any problem, but when I use the poetry install or poetry add commands, I can't restore or add any package from my private repository.

I am really enjoying using poetry and this has been the only barrier I have encountered so far.

note: I tried to configure the poetry config as suggested here #3344

I also ran the .py script that does the authentication and already sets up the poetry, but nothing worked.

The message I always get is:

poetry install

Because teste-mb27 depends on teste-mb26 (0.0.1.dev0) which doesn't match any versions, version solving failed.

poetry add

Could not find a matching version of package teste-mb26
image
image
image
image
image
image
image

can you help me? am I setting up something wrong?

image
using the pip I was successful, but I would like to solve everything just with poetry

@marcosborges
Copy link

Guys I managed to solve the problem and now it is working 100%.

My first mistake was declaring the repository on the first line of my .toml file. The execution started to work as soon as I put it down [tool.poetry]
image

The second step was to add the script to login to azure in my scritps.py and declare it on pyproject.toml
image
image

now with a simple poetry run azure_login everything works.
image

@DustinMoriarty
Copy link
Contributor

Well, this issue is closed, but I found my issue which I think may actually be different. I had http-basic setup for the repository for the purpose of posting to it. For our artifactory pypi repository authorization is not needed to get from the repository, only to publish. Therefore, it did not even occur to me that poetry may be using the basic auth credentials for get operations. However, I changed my password so auth failed even for get. I guess that kind of makes sense because poetry doesn't know how the security is configured on your repo so if you provide a user name and password it just uses it for all operations. Your user name and password should be valid. It just took me longer to troubleshoot because the particular operation I was performing did not actually require authorization so I forgot to check if the credentials themselves are valid. I guess @marcosborges we both solved our respective problems. However, maybe this will help the next person.

@cglacet
Copy link

cglacet commented May 28, 2021

I have a similar issue, I'm using the following configuration in project.toml:

[[tool.poetry.source]]
name = "privateRepo"
url = "https://__token__:[email protected]/api/v4/projects/1234567/packages/pypi/simple"

Everything works fine on my machine, but on gitlab I get the following error:

Command ['/root/.cache/pypoetry/virtualenvs/xxxxx-py3.9/bin/pip', 'install', '--no-deps', '--index-url',
      'https://__token__:AXXXXXXA@__token__:[email protected]/api/v4/projects/1234567/packages/pypi/simple',
      'package-name==0.2.2'] 
      errored with the following return code 1, and output: 
  Looking in indexes: https://__token__:[email protected]/api/v4/projects/1234567/packages/pypi/simple
  WARNING: 401 Error, Credentials not correct for https://__token__:[email protected]/api/v4/projects/1234567/packages/pypi/simple/package-name/

I noticed that the url includes __token__:X twice, so maybe that's the problem?

@xkortex
Copy link

xkortex commented Sep 2, 2021

Running into this issue, this time in a docker build. Poetry version 1.1.7. I'm using pypiserver to host my own private index an https://pypi.$COMPANY.tld (behind an AWS load balancer), and poetry has zero problems adding/locking. Also note, I have it configured to list without auth, but update and download are authentication-locked.

add/lock works totally fine in my outside environment. update: I can add and lock on my host, but I cannot poetry install it seems, so it's more than just docker. The server is serving on HTTP:8080, and then I'm using an AWS load balancer to terminate TLS and give me the https, so maybe that might be part of it.

Inside the container, I get the error

401 Client Error: Unauthorized for url: http://pypi.$COMPANY.tld:80/pypi/packages/$COMPANY_core-0.7.1-py3-none-any.whl#md5=1f4172b9955ef0fb696452f60d4da33e

Notice that is http://pypi.$COMPANY.tld, even though I double checked and made sure I'm using https://pypi.$COMPANY.tld everywhere.

pyproject.toml contains

[[tool.poetry.source]]
name = "$COMPANY"
url = "https://pypi.$COMPANY.tld"
default = true # I've tried both default and secondary, neither work

[tool.poetry.dependencies]
$COMPANY_core = "^0.7"
python = "^3.7"

The Dockerfile looks something like:


WORKDIR /app
COPY ./poetry.lock ./pyproject.toml ./
COPY secrets/auth.toml /root/.config/pypoetry/
COPY secrets/config.toml /root/.config/pypoetry/

RUN \
    poetry config --list -vvv &&\
    cat /root/.config/pypoetry/config.toml &&\
    cat /root/.config/pypoetry/auth.toml &&\
    poetry install --no-root --no-dev --no-interaction --no-ansi &&\
    :

Outside, I have no problem doing

curl https://USER:PASS@pypi.$COMPANY.tld/pypi/packages/$COMPANY_core-0.7.1-py3-none-any.whl#md5=1f4172b9955ef0fb696452f60d4da33e -o $COMPANY_core-0.7.1-py3-none-any.whl
pip install $COMPANY_core-0.7.1-py3-none-any.whl # works fine
md5sum $COMPANY_core-0.7.1-py3-none-any.whl # hash checks out
1f4172b9955ef0fb696452f60d4da33e  $COMPANY_core-0.7.1-py3-none-any.wh

To set up my local env, I ran:

poetry config repositories.$COMPANY https://pypi.$COMPANY.tld
poetry config http-basic.$COMPANY USER PASS

which generates config:

[repositories]
[repositories.$COMPANY]
url = "https://pypi.$COMPANY.tld

I'm doing the same exact thing in the container, no dice. I even tried copying in the plaintext password into auth.toml, and deleted the entry in my keyring, to force it to use plaintext. When I dump the config/auth.toml inside the container, it looks exactly correct:

#11 1.691 [repositories]
#11 1.691 [repositories.$COMPANY]
#11 1.691 url = "https://pypi.$COMPANY.tld"
#11 1.692 [http-basic]
#11 1.692 [http-basic.$COMPANY]
#11 1.692 username = "USER"
#11 1.692 password = "PASS"

In fact I even tried copying the exact files I'm using locally, into the docker container. Still failing.

However, it looks like I can run this sequence:

WORKDIR /app
COPY ./pyproject.toml ./
COPY secrets/auth.toml /root/.config/pypoetry/
COPY secrets/config.toml /root/.config/pypoetry/

RUN \
    poetry config --list -vvv &&\
    cat /root/.config/pypoetry/config.toml &&\
    cat /root/.config/pypoetry/auth.toml &&\
    poetry lock --no-update -vvv &&\
    :

This works great until I go to actually install the local package with RUN poetry install --no-interaction. So it looks like the mechanism which is requesting based on the contents of the lockfile is different than when running poetry lock. This might in fact be a new bug.

Notice that in the poetry lock operation it's using https correctly:

#14 136.6 $COMPANY: Response URL https://pypi.org/simple/jaraco-functools/ differs from request URL https://pypi.$COMPANY.tld/jaraco-functools/
#14 136.6 $COMPANY 33 packages found for jaraco.functools *
#14 136.9 $COMPANY Response URL https://pypi.org/simple/jaraco-functools/ differs from request URL https://pypi.$COMPANY.tld/jaraco-functools/
#14 136.9 $COMPANY: Downloading wheel: jaraco.functools-3.3.0-py3-none-any.whl
#14 137.1    1: fact: jaraco.functools (3.3.0) depends on more-itertools (*)
#14 137.1    1: selecting jaraco.functools (3.3.0)
#14 137.3 $COMPANY: Response URL https://pypi.org/simple/more-itertools/ differs from request URL https://pypi.$COMPANY.tld/more-itertools/
#14 137.4 $COMPANY: Downloading wheel: more_itertools-8.8.0-py3-none-any.whl

"Full" trace btw (I don't get a full traceback, even with -vvv):



  HTTPError

  401 Client Error: Unauthorized for url: http://pypi.$COMPANY.tld:80/pypi/packages/$COMPANY_core-0.7.1-py3-none-any.whl#md5=1f4172b9955ef0fb696452f60d4da33e

  at ~/.poetry/lib/poetry/_vendor/py3.9/requests/models.py:943 in raise_for_status
      939│         elif 500 <= self.status_code < 600:
      940│             http_error_msg = u'%s Server Error: %s for url: %s' % (self.status_code, reason, self.url)
      941│
      942│         if http_error_msg:
    → 943│             raise HTTPError(http_error_msg, response=self)
      944│
      945│     def close(self):
      946│
      947│         called the underlying ``raw`` object must not be accessed again.

@jgentil
Copy link

jgentil commented Oct 29, 2021

In exceutor.py _download_archive function, it uses self._authenticator which is NOT the authenticator that should be used! The LegacyRepository instance's authenticator should be used instead, as that has the credentials. So the link chooser creates a list of links without credentials, because that's what's returned in the HTML from the private pypi/azure devops server, and then it uses the wrong authenticator.

Or, alternatively, a Link object should have it's own Authenticator instance. Or, the find_links_for_package method should re-write links to copy the authentication information back into the link.

@jgentil
Copy link

jgentil commented Nov 1, 2021

This seems to fix the problem with embedded URLs not getting the correct credentials:

    def find_links_for_package(self, package: Package) -> List[Link]:
        page = self._get("/{}/".format(package.name.replace(".", "-")))
        if page is None:
            return []

        links = list(page.links_for_version(package.version))
        for link in links:
            if link.netloc in self._authenticator._credentials:
                url = list(urllib.parse.urlsplit(link.url))
                username, password = self._authenticator._credentials[link.netloc]
                url[1] = f'{username}:{password}@{link.netloc}'  # [1] is netloc
                link.url = urllib.parse.urlunsplit(url)

        return links

@alchemistake
Copy link

Why this issue is closed? It seems like the fix has not been pushed

@SimonVerhoek
Copy link

After trying out everything stated here multiple times without avail and banging my head against the wall over the course of a few days, I finally seem to have a situation working where a poetry project can import a self-made private package published with poetry.

While looking at the auth.toml, I noticed that no password was set. It looked like this:

[http-basic.my-package]
username = "<personal_access_token>"

Apparently, the password is set in your OS's native keychain by the library called keyring by default. I did the following:

  • switch off keyring like this
  • rerun poetry config http-basic.my-package <personal_access_token> <personal_access_token_password>. This now gave the following messages in return:
No suitable keyring backends were found
Using a plaintext file to store and retrieve credentials

Poetry's auth.toml now looked like this:

[http-basic.my-package]
username = "<personal_access_token>"
password = "<personal_access_token_password>"

...and after running poetry update once more, the library installed flawlessly.

This worked with poetry==1.1.11, installed by brew on MacOS.
(part of) my pyproject.toml:

[[tool.poetry.source]]
name = "my-package"
url = "https://gitlab.com/api/v4/projects/<project_no>/packages/pypi/simple/"
secondary = true

[tool.poetry.dependencies]
my-package = { version = "^<your.package.version>", source = "my-package" }

@tropxy
Copy link

tropxy commented Dec 16, 2021

I also dont understand why this ticket was closed. I had exactly the same problem... In fact, because of this Keyring issue, when installing the public packages, poetry even starts looking first for my private repo, so I started specifying in each dependency, the repo source:

[tool.poetry.dependencies]
python = "^3.9"
asyncio-mqtt = "^0.11.0"
environs = "^9.3.5"
mqtt-api = { version = "^0.5.0", source = "pypi-private"}

[tool.poetry.dev-dependencies]
pytest =  { version = "^6.2.5", source = "pypi"}
pytest-lazy-fixture = { version ="^0.6.3", source = "pypi"}
...

And just after this process, the library tells me the problem is really with the authentication with the private server.

After following the steps provided by @SimonVerhoek everything worked as expected.
Also, if one specifies directly in the toml the credentials like this:

[[tool.poetry.source]]
name = "pypi-private"
url = "https://username@password:pypi-private.com/simple/"
default = false
secondary = true

it also works, but this is not ideal as no one wants the credentials to be plain text in github.

Another fun fact: I didnt experience this problem neither in MacOS or Docker container. Was really only when I tried in a Linux virtual machine :/

@AlviseSembenico
Copy link

AlviseSembenico commented Dec 22, 2021

I am experiencing the same issue with GCP artifact registry, even after trying all the aforementioned alternative, I've also tried version 1.2.0a2 with the same result.

@oofnikj
Copy link

oofnikj commented Feb 8, 2022

Thank you @SimonVerhoek , your steps helped me solve this problem when trying to download packages from AWS CodeArtifact. We are building container images and using Docker secrets to mount the auth.conf from the host temporarily into the build environment, which is incompatible with using the OS keyring.

I opted to configure the Python keyring with environment variables in our build scripts like this:

export PYTHON_KEYRING_BACKEND=keyring.backends.fail.Keyring
poetry config http-basic.codeartifact aws "$(
  aws codeartifact get-authorization-token --domain <domain> \
  --query 'authorizationToken' --output text
  )"

@robin92 robin92 mentioned this issue Feb 15, 2022
2 tasks
@Lilja
Copy link

Lilja commented Feb 18, 2022

I am also experiencing the same issue.

$ poetry config http-basic.fury "<test>" NOPASS

$ cat "/Users/lilja/Library/Application Support/pypoetry/auth.toml"
[http-basic]
[http-basic.fury]
username = "<test>"
$ poetry config http-basic.fury -vvv
Loading configuration file /Users/lilja/Library/Application Support/pypoetry/config.toml
Loading configuration file /Users/lilja/Library/Application Support/pypoetry/auth.toml

  Stack trace:

  4  ~/.poetry/lib/poetry/_vendor/py3.8/clikit/console_application.py:131 in run
      129│             parsed_args = resolved_command.args
      130│
    → 131│             status_code = command.handle(parsed_args, io)
      132│         except KeyboardInterrupt:
      133│             status_code = 1

  3  ~/.poetry/lib/poetry/_vendor/py3.8/clikit/api/command/command.py:120 in handle
      118│     def handle(self, args, io):  # type: (Args, IO) -> int
      119│         try:
    → 120│             status_code = self._do_handle(args, io)
      121│         except KeyboardInterrupt:
      122│             if io.is_debug():

  2  ~/.poetry/lib/poetry/_vendor/py3.8/clikit/api/command/command.py:171 in _do_handle
      169│         handler_method = self._config.handler_method
      170│
    → 171│         return getattr(handler, handler_method)(args, io, self)
      172│
      173│     def __repr__(self):  # type: () -> str

  1  ~/.poetry/lib/poetry/_vendor/py3.8/cleo/commands/command.py:92 in wrap_handle
       90│         self._command = command
       91│
    →  92│         return self.handle()
       93│
       94│     def handle(self):  # type: () -> Optional[int]

  ValueError

  There is no http-basic.fury setting.

  at ~/.poetry/lib/poetry/console/commands/config.py:128 in handle
      124│                 self.line(str(value))
      125│             else:
      126│                 values = self.unique_config_values
      127│                 if setting_key not in values:
    → 128│                     raise ValueError("There is no {} setting.".format(setting_key))
      129│
      130│                 value = config.get(setting_key)
      131│
      132│                 if not isinstance(value, basestring):

I don't know if this is the proper fix, but I manually edited poetry/console/commands/config.py:
image

Commenting out the raise ValueError fixed it for me.

@The-Fonz
Copy link

The-Fonz commented Feb 21, 2022

We have this same issue with a private gitlab package registry. When fully specifying the URL including token, it works:

[[tool.poetry.source]]
name = "<repo-name>"
url = "https://__token__:<TOKEN>@gitlab.com/api/v4/projects/<project_id>/packages/pypi/simple"

However, we don't want to hardcode credentials of course, but specify them using poetry config http-basic.<repo-name> __token__ <pass>. The function where it errors (legacy_repositories.py, since renamed to legacy_repository.py) does not seem to pass any credentials at all.

Edit: what I wrote here before was not true, I could not find a way to fix it using both the default=true and removing /simple methods, it just silently fails instead of raising an auth error.

There seem to be some fixes, I tried to upgrade poetry using poetry self update --preview to 1.2.0a2 but that did not make a difference (did still show 1.1.12 using poetry --version).

@rschalch
Copy link

rschalch commented May 20, 2022

I had this issue recently with AWS CodeArtifact and what solved my problem was using these environment variables:
POETRY_HTTP_BASIC_ARTIFACTORY_<AWS_CODEARTIFACT_REPO_NAME>_USERNAME
POETRY_HTTP_BASIC_ARTIFACTORY_<AWS_CODEARTIFACT_REPO_NAME>_PASSWORD
The last one having the same value of CODEARTIFACT_TOKEN
Um abraço @jairhenrique !

@abn
Copy link
Member

abn commented May 20, 2022

With recent changes the use cases with azure (keyring auth), gitlab (path based auth) etc should be significantly improved. There was also a bug in how the the POETRY_REPOSITORIES_* environment variables were being loaded as well. Can folks on this thread test it out and provide any issues you are still facing?

See these issues for more information: #5563 #5518 #4086

Install Poetry from VCS

pipx install --force --suffix=@git 'poetry @ git+https://github.com/python-poetry/poetry.git'

@kaybee99
Copy link

I've got the same issues with a private repo as mentioned above.

Upgraded to version 1.2.0b2.dev0 with @abn's suggestion, but I'm faced with the same issue albeit with a slightly different output

user@BTM-C02XX62HTD5 application % poetry@git add Beaker -vvv
...
Creating new session for nexus.tools.company.co.uk
[urllib3.connectionpool] Starting new HTTPS connection (1): nexus.tools.company.co.uk:443
[urllib3.connectionpool] https://nexus.tools.company.co.uk:443 "GET /repository/pypi-group/simple/beaker/ HTTP/1.1" 401 0
Source (pypi-nexus-proxy): Authorization error accessing https://nexus.tools.company.co.uk/repository/pypi-group/simple/beaker/
...

@abn
Copy link
Member

abn commented May 26, 2022

@kaybee99 how was auth configured?

@kaybee99
Copy link

kaybee99 commented May 26, 2022

Username added to auth.toml with poetry@git config http-basic.pypi-nexus-proxy <username>. My feeling is there's something wrong with the way it's retrieving the password from keychain

@The-Fonz suggestion does work however I'm keen to avoid commiting credentials wherever possible.

@abn
Copy link
Member

abn commented May 26, 2022

@kaybee99 you will need poetry@git config http-basic.pypi-nexus-proxy <username> <password> in order to add to poetry's keychain lookup.

@kaybee99
Copy link

Doesn't work either unfortunately. Same error

@abn
Copy link
Member

abn commented May 27, 2022

@kaybee99 unfortunately I can't do much here as I can't reproduce this with my setup.

You can find me on discord and we could maybe debug it together. Alternatively, you can consider adding some additional logs in authenticator and password manager to see what's going on.

@kaybee99
Copy link

Thanks, I appreciate it. I've already spent way too much time trying to get this working so I'm going to stick with the credentials in URL workaround for now as it's only affecting my environment and works fine in our pipeline. Just got to remember to not push those changes!

@neersighted
Copy link
Member

Closing for now as there are significant improvements in auth, and everything here is working as expected for me on 1.2.x. If you're having trouble passing credentials, I'd suggest first opening a Discussion or joining Discord to get some troubleshooting help. If you've done extensive troubleshooting and are sure there's a Poetry bug, please open a new issue.

@cglacet
Copy link

cglacet commented Feb 22, 2023

@neersighted I think it is still very unstable/opaque, I get auth errors every now and then (and I'm not even talking about CI/CD pipelines). There are multiple ways doing this which make it unclear for us how to search the root cause of the issue.

In my opinion, if a local token is hard coded in poetry.toml everything should just work, or if it doesn't, at least poetry should intercept the authentication error and display a message that explains what credentials it used (keyring? cache? It seems like nobody really knows the answer here). If the user hands poetry the name, url, auth token and it doesn't work, to me that lookks a lot like a bug (no matter what the local machine state is in).

So maybe people maintaining poetry have no idea what is going on and its perfectly fine, but closing the issue wont help solving it. Maybe its a network error, maybe its not a bug in poetry but somewhere else, but since we have no clue I think its still an issue.

For example today I upgraded from py3.10 to 3.11 and got the HTTP 401 error. I tried many random things, and found one that fixed the issue (but I have no idea why), I removed the token from the repo URL (which in the past was probably a fix I made to fix the exact same issue):

[[tool.poetry.source]]
name = "myrepo"
+url = "https://gitlab.com/api/v4/projects/xxxxxxxx/packages/pypi/simple"
-url = "https://__token__:[email protected]/api/v4/projects/xxxxxxxx/packages/pypi/simple"
secondary = true

I'm pretty sure I'll come back to this issue in 6 months only to find out that I made that removal and that it now causes a problem somewhere else (pipeline for example). I would love to know a solution that is stable over time.

@bensalem22
Copy link

I am facing authentication issues with a private repo that doesn't require authentication. any guidance on how to solve this?
401 Client Error: Unauthorized for url: XXX

@onmywayout
Copy link

After spending multiple hours trying the different solutions, for me the solution was changing the password to something that didn't have special characters and thus didn't need to be percent encoded. The current password i was using worked perfectly with other Pypi installation using pip, but didn't work here neither in the escaped or the not escaped version https://pip.pypa.io/en/stable/topics/authentication/#percent-encoding-special-characters

@mikeharbrdata
Copy link

Upgrade from poetry 1.1.13 to 1.5.1 seemed to resolve for me

@afarah1
Copy link

afarah1 commented Nov 9, 2023

I had this issue on a Mac, poetry having been installed with homebrew. tl;dr what worked was:

  1. export PYTHON_KEYRING_BACKEND=keyring.backends.fail.Keyring
  2. Create your auth.toml at /Users/<YOUR USER>/Library/Application Support/pypoetry

I had tried using auth.toml in the same folder as my pyproject.toml as well as adding the following entry directly in pyproject.toml, to no avail:

[http-basic.codeartifact]
username = "aws"
password = "<my jwt>"

I also tried setting the credentials by exporting CODEARTIFACT_TOKEN=<my jwt>, POETRY_HTTP_BASIC_ARTIFACT_PASSWORD=<my jwt> and POETRY_HTTP_BASIC_ARTIFACT_USERNAME=aws, also to no avail. I also tried export PYTHON_KEYRING_BACKEND=keyring.backends.fail.Keyring, to no avail apparently.

Following some comments above I ran with -vvv to see what poetry was doing:

Loading configuration file /Users/<me>/Library/Application Support/pypoetry/config.toml
...
Source (codeartifact): 0 packages found for <my lib from a private repo>
Falling back to installed packages to discover metadata for <my lib from a private repo>
Found 0 compatible packages for <my lib from a private repo>
Multiple source configurations found for <my private repo> - pypi-upstream, codeartifact
No suitable keyring backend found
No suitable keyring backends were found
Keyring is not available, credentials will be stored and retrieved from configuration files as plaintext.
Creating new session for <my private repo> 
Multiple source configurations found for <my private repo>  - pypi-upstream, codeartifact
[urllib3.connectionpool] Starting new HTTPS connection (1): <my private repo>
[urllib3.connectionpool] <my private repo> "GET /pypi/pypi-upstream/simple/whitenoise/ HTTP/1.1" 401 98

Ah!! So it's trying to read a config file at /Users/<me>/Library/Application Support/pypoetry/config.toml and completely ignoring the auth.toml form local dir and all env vars. Nice! That config file is empty, btw. So I just created an auth.toml at that same location and it now works!

Copy link

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Feb 29, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
kind/bug Something isn't working as expected status/triage This issue needs to be triaged
Projects
None yet
Development

No branches or pull requests