Upgrade most Python dependencies in 3rdparty/python/requirements.txt

[sureshjoshi]

Tried to capture most of the updates here - testing to see what happens in CI

==                    Upgraded dependencies                     ==

  anyio                          3.7.1        -->   4.11.0
  debugpy                        1.8.13       -->   1.8.16
  fastapi                        0.78.0       -->   0.120.0
  fasteners                      0.16.3       -->   0.20
  freezegun                      1.2.1        -->   1.5.5
  ijson                          3.2.3        -->   3.4.0.post0
  libcst                         1.4.0        -->   1.8.5
  packaging                      24.2         -->   25.0
  pydantic                       1.10.24      -->   2.12.3
  pygithub                       2.4.0        -->   2.8.1
  python-dotenv                  1.1.1        -->   1.2.1
  python-gnupg                   0.4.9        -->   0.5.5
  python-lsp-jsonrpc             1.0.0        -->   1.1.2
  python-multipart               0.0.5        -->   0.0.20
  setproctitle                   1.3.2        -->   1.3.7
  setuptools                     63.4.3       -->   80.9.0
  starlette                      0.19.1       -->   0.48.0
  strawberry-graphql             0.114.0      -->   0.284.1
  types-pyyaml                   6.0.3        -->   6.0.12.20250915
  types-requests                 2.28.1       -->   2.32.4.20250913
  types-setuptools               62.6.1       -->   80.9.0.20250822
  types-toml                     0.10.8       -->   0.10.8.20240310
  urllib3                        1.26.20      -->   2.5.0
  uvicorn                        0.17.6       -->   0.38.0
                                                                  
==                      Added dependencies                      ==

  annotated-doc                  0.0.3
  annotated-types                0.7.0
  lia-web                        0.2.3
  pydantic-core                  2.41.4
  typing-inspection              0.4.2
  watchfiles                     1.1.1
                                                                  
==                     Removed dependencies                     ==

  asgiref                        3.10.0
  backports-cached-property      1.0.2
  deprecated                     1.2.18
  types-freezegun                1.1.6
  types-urllib3                  1.26.25.14
  watchgod                       0.8.2
  wrapt                          1.17.3

Reviewers: I didn't upgrade all Python deps. The ones I went through, I just reviewed changelogs (didn't review their code changes) and pulled these from dependabot's suggestions.

I left some off if they looked more invasive. Will circle back to those later on. Left pex alone too, as I'll do that in the python 3.14 upgrade branch (since pex supports free-threaded Python downloads).

Can anyone who knows more about botocore vs urllib speak to the comment I placed in the review please?

I also opened #22814 to maybe split off explorer as a separate plugin, or remove it - as it seems unfinished, and requires several deps - which don't end up in our main pants package - but they're still part of our CI and lockfiles. Everything in the main repo that we don't use (or is unfinished) burdens us slightly when it comes to updates, so I'd rather it be it's own thing (related to built-in goals though, so might not move soon).

[sureshjoshi]

Get this error after updating SetupTools to support <81 (so, I've already pinned it). This import will need to be removed in Python 3.14 anyways.

There is no pantsd metadata at /Users/sj/Developer/scratch/pants/.pants.d/pids/0713377c7f5b/pantsd.
08:10:12.76 [WARN] /Users/sj/Developer/scratch/pants/src/python/pants/init/options_initializer.py:14: UserWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html. The pkg_resources package is slated for removal as early as 2025-11-30. Refrain from using this package or pin to Setuptools<81.
  import pkg_resources

/Users/sj/Developer/scratch/pants/src/python/pants/init/options_initializer.py:14: UserWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html. The pkg_resources package is slated for removal as early as 2025-11-30. Refrain from using this package or pin to Setuptools<81.
  import pkg_resources
08:10:15.58 [INFO] Initializing scheduler...
08:10:15.92 [INFO] Scheduler initialized.

[sureshjoshi]

From the explorer backend

08:13:13.28 [WARN] /Users/sj/.cache/pants/pants_dev_deps/3db7049535f77bbd89235b7d4b164e4b4f0a581f.venv/lib/python3.11/site-packages/websockets/legacy/__init__.py:6: DeprecationWarning: websockets.legacy is deprecated; see https://websockets.readthedocs.io/en/stable/howto/upgrade.html for upgrade instructions
  warnings.warn(  # deprecated in 14.0 - 2024-11-09

08:13:13.39 [WARN] /Users/sj/.cache/pants/pants_dev_deps/3db7049535f77bbd89235b7d4b164e4b4f0a581f.venv/lib/python3.11/site-packages/uvicorn/protocols/websockets/websockets_impl.py:17: DeprecationWarning: websockets.server.WebSocketServerProtocol is deprecated
  from websockets.server import WebSocketServerProtocol

[sureshjoshi]

As far as I can see, we meet the conditions for upgrading this (and if we don't, we have to hold back other deps).

Seems like this is resolved enough to update, but I don't know if this is captured in the tests - I saw other packages unpinned though.

boto/botocore#3138

To follow up, this was resolved in today's release (botocore 1.34.63) with boto/botocore#3141. Users on Python 3.10+ should be free to control their urllib3 2.x requirements going forward.

This is weird for us, because if someone is using an old python with botocore, will this interfere with that? Should this allow a range of deps? The comment I just deleted suggests this could be relaxed when we upgrade Python, but having never used Botocore, I can't safely state what will happen here.

Importantly, there were some security updates between what we used, and now - so 🤷🏽

[sureshjoshi]

If any reviewers are more educated on this, please let me know what I should do here ^^

[cognifloyd]

I think the code comment here, and the issue comment you quoted, combine to mean:

once we upgrade to Python 3.10+ (which we've done since we're on Python 3.11), we can remove this version constraint entirely. That should allow pex+pip to select the best version when people add botocore to their list of backend packages.

Suggested change

urllib3==2.5.0

Or if mypy or something complains about a missing dep, maybe:

Suggested change

	urllib3==2.5.0
	urllib3

[cognifloyd]

This is the PR that added the constraint, so we should be able to, basically, just revert that PR:
#21402

And that links to this issue: #21164

Also see more background in: #18959

[sureshjoshi]

Yeah, I'm even better with that

[sureshjoshi]

Internally, this is very much used as a float by uvicorn (comparing against a float and time.time()) - but they've typed it as an int. Also, 0.25s vs 1s for a timeout is fine, as they default it to 30s

[sureshjoshi]

I'm holding these back until after 3.30 is released. These are all also used in Python 3.14, which I'm also keeping out of 3.30

[cburroughs]

3.30

2.30? If so, why wait?

[sureshjoshi]

3.30

2.30? If so, why wait?

Ran into some quirks while testing on one of my branches with these changes. Unsure if it's directly related to these, but there isn't much else on that branch right now. So, I think this might need some more time. Alternatively, if I can see the one I think was causing a hassle, I'll drop just that.

[sureshjoshi]

@cburroughs Okay, turns out the quirk was in a rust update - or stale cache. I nuked every cache on my machine and re-ran, and I don't get intermittent failures anymore. It may have lingered from some of the Python 3.14 updates, as that caused weirdness too.

I'm good to merge this