Cannot upload package to test.pypi.org #3396
Labels
network
Issues related to our CDN, users having problems connecting to PyPI
Comments
bmvandoren
added
the
network
|
I'm still having this issue and would very much appreciate guidance, as I still cannot upload updates to my package. |
|
@bmvandoren I think we're missing some key details about your set up here - specifically some versions of anaconda, twine, requests, urllib3, certifi, ca-certificates. We can see that you're able to connect via the |
miketheman
added
the
status: awaiting response
|
Thanks for getting back to me, @miketheman. Happy to provide whatever details would be helpful. conda version : 23.10.0 ca-certificates 2023.08.22 hecd8cb5_0 twine==4.0.2 Many thanks. |
|
Also worth highlighting that the package is large (81.3 mb), but below the 100 mb limit (and I was able to upload earlier versions of virtually the same size). |
|
Thanks. I've also noticed that some of the openssl debug commands are referring to pypi.org, not test.pypi.org where you're seeing the issue - can you please update/run those as well and provide the output? |
|
Happily! I think I found the relevant commands, but let me know if there are any others to run. Thanks @miketheman. |
miketheman
removed
the
status: awaiting response
|
I will point out that something doesn't look right - since you've said you're using Python 3.8, but the first traceback is showing an anaconda Python 3.10 in the logged paths - are you certain you're using the same python env? |
|
Hmm. It looks like I have 3.8 installed in the (base) conda environment, but the development environment for this package has Python 3.10, as intended. I created the development environment using this command:
If I run The output above was from the This clearly indicates a python version of 3.8, but I am not sure if this would be expected to show 3.10 in this case. I hope this helps. |
|
I believe this means that anaconda is installed in a base python version of 3.8, but this is different from the python version of the development environment, which is 3.10. However, I could definitely be wrong. |
|
And to answer your question, yes, all of the commands I have run here have been run in the |
|
@bmvandoren The only other things I can currently think of is that your TestPyPI account isn't currently set up for 2FA (which is already required on TestPyPI, and coming to PyPI), which would then also need an API Token instead of username/password for twine. I'd have expected a different error message than the one we saw though. Does the exact same error persist as when you originally reported the issue, or has the traceback changed? |
|
You could also add |
|
I do not have 2FA set up - I will try that now and let you know. |
|
Unfortunately that doesn't appear to have fixed the issue. I set up 2FA on my test.pypi account, and twine appears to be able to authenticate using this method. But I still get the same warning and eventual error. I should also note that a colleague of mine who is also on this project gets the same error when he tries to upload the package using his test PyPI account on his computer. Here's the twine output after enabling 2FA: |
|
@bmvandoren Thanks for stepping through that, it's super helpful to see the timings. And you'll need to do the 2FA setup anyhow 👼 I think what we're seeing is that the client (twine) taking too long to upload and complete all the steps, possibly due to network congestion, or some other bottleneck. We can see from the output that attempts appear to cut off around the 1 minute mark (Production PyPI has different limits in place). A couple of ideas you might want to try:
|
|
Thanks @miketheman. According to speedtest.net my download speed is 200 Mbps and upload is 10 Mbps. I'm in a city right now and my cell phone is getting faster upload speeds, so I tried tethering. It cut the upload time in half. However, it's still failing, but in a different way. It's bizarre because the upload finishes and then it gives the error. |
|
Hi @miketheman - writing with an update. Most of the bulk of the package is from a few model files. When I exclude the model files and try uploading the package, it works. Is it possible to try increasing the file size limit on this package? Here is the output from the upload when I remove the (necessary) model files. The package is then only 430 kB: |
|
@bmvandoren Glad to hear there was some successful uploads - that confirms all the authentication and upload stack. The "503: Service Unavailable" is coming from our caching tier, in response to a 5xx response from the backend, but I haven't been able to find any specific logged exceptions that would show this issue. If you can run the test again with the faster network and provide UTC timestamps so we can narrow down when this is happening. |
|
@miketheman - I ran these uploads again and printed UTC timestamps. The timestamps are in HH:MM:SS format and correspond to the UTC date of 2023-12-21. I first tried uploading the ~80 mb package, which again failed as before. Then I removed the model files and uploaded the rest of the package, which again worked. Many thanks for your continued help. Uploading full package (fails)Uploading package without large model files (succeeds) |
|
Thanks @bmvandoren ! The good news - I was able to reproduce the error on testpypi. The bad news - I played with limits, no dice there. The issue is happening between our edge cache and the backend application, and the request isn't showing up on the backend oddly. Considering that TestPyPI is generally not used for production-grade packages, would it make more sense to use production PyPI, and thus sidestep the problem completely? |
|
Ah, I see. I thought that best practice was to first upload to TestPyPI before deploying to PyPI. But that assumed the two would be essentially identical. I able to push this release to PyPI, so that's good! But it would be nice to be able to use TestPyPI for testing. |
Glad to hear it!
Indeed. Leaving this issue open for now, since there's definitely something different about how we are proxying the uploads for testpypi that is causing this issue. |
raminqaf
added a commit
to bakdata/kpops
that referenced
this issue
Jan 26, 2024
 There is a way to increase the timeout: https://python-poetry.org/docs/faq/#my-requests-are-timing-out ### Edit The above issue is still not solved. Even after setting the env variable, the timeout does not increase. Related issue for TestPyPI: pypi/support#3396
|
fwiw we are seeing the same sort of problems but when uploading from github actions to public pypi: https://github.com/getsentry/publish/actions/runs/8437326194/job/23106871689#step:11:814 happy to provide whatever additional information that would help debug this -- it's running in a docker image (latest stable debian) with the current latest |
|
TestPyPI is being used for testing a redis upgrade and should be back soon. No SLA on that service. |
|
@ewdurbin my message is about public pypi but the same failure modes |
|
Then please open an issue that isn't related to test.pypi.org, @asottile-sentry! |
|
I was hoping to not lose the context that's already present in this thread since it is the same error message and same symptoms (large uploads) |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
My Platform
Terminal (zsh) on MacOS 13.6. Spectrum service provider, no firewalls or proxies.
Fastly Debug
DNS Resolution
Traceroutes / IPv4
Traceroutes / IPv6 (If available)
HTTPS Requests / IPv4
HTTPS Requests / IPv6 (If available)
TLS Debug / IPv4
TLS Debug / IPv6 (If available)
Code of Conduct
The text was updated successfully, but these errors were encountered: