gh-116741: Upgrade libexpat to 2.6.2

[sethmlarson]

cc @hartwork

• Issue: Please upgrade bundled Expat to 2.6.2 (e.g. for the fix to CVE-2024-28757) #116741

[hartwork]

@sethmlarson looks good, my verification script likes it. Thank you! 👍

# Copyright (c) 2022-2024 Sebastian Pipping <[email protected]>
# Licensed under the Apache License version 2.0

FROM alpine
RUN apk add --update \
            diffutils \
            git \
            sed \
        && \
    git clone --depth 1 https://github.com/python/cpython cpython-main \
        && \
    ( cd cpython-main && git rev-parse HEAD ) \
        && \
    git clone --depth 1 --branch libexpat-2.6.2 https://github.com/sethmlarson/cpython cpython-pr \
        && \
    ( cd cpython-pr && git rev-parse HEAD ) \
        && \
    git config --global advice.detachedHead false \
        && \
    git clone --depth 1 --branch R_2_6_0 https://github.com/libexpat/libexpat libexpat_2_6_0 \
        && \
    git clone --depth 1 --branch R_2_6_2 https://github.com/libexpat/libexpat libexpat_2_6_2 \
        && \
    diff -r -u libexpat_2_6_0/expat/lib/ cpython-main/Modules/expat/ | tee 2-6-0.diff \
        && \
    diff -r -u libexpat_2_6_2/expat/lib/ cpython-pr/Modules/expat/ | tee 2-6-2.diff \
        && \
    sed -e '/^Only in /d' -e '/^\(+++\|---\) /d' -e '/^diff /d' -i 2-6-0.diff 2-6-2.diff \
        && \
    diff -u 2-6-0.diff 2-6-2.diff \
        && \
    echo 'Diff is good.'

[hartwork]

@sethmlarson ready to merge?

[hartwork]

@sethmlarson any news?

[sethmlarson]

@hartwork I can't merge PRs on my own, I'll find a core developer to merge.

[gpshead]

These changes appear to match libexpat/libexpat@R_2_6_0...R_2_6_2.

We should probably script updating this.

[miss-islington-app]

Thanks @sethmlarson for the PR, and @gpshead for merging it 🌮🎉.. I'm working now to backport this PR to: 3.8, 3.9, 3.10, 3.11, 3.12.
🐍🍒⛏🤖

[miss-islington-app]

Sorry, @sethmlarson and @gpshead, I could not cleanly backport this to 3.11 due to a conflict.
Please backport using cherry_picker on command line.

cherry_picker c9829eec0883a8991ea4d319d965e123a3cf6c20 3.11

[bedevere-app]

GH-118166 is a backport of this pull request to the 3.12 branch.

[miss-islington-app]

Sorry, @sethmlarson and @gpshead, I could not cleanly backport this to 3.10 due to a conflict.
Please backport using cherry_picker on command line.

cherry_picker c9829eec0883a8991ea4d319d965e123a3cf6c20 3.10

[miss-islington-app]

Sorry, @sethmlarson and @gpshead, I could not cleanly backport this to 3.9 due to a conflict.
Please backport using cherry_picker on command line.

cherry_picker c9829eec0883a8991ea4d319d965e123a3cf6c20 3.9

[miss-islington-app]

Sorry, @sethmlarson and @gpshead, I could not cleanly backport this to 3.8 due to a conflict.
Please backport using cherry_picker on command line.

cherry_picker c9829eec0883a8991ea4d319d965e123a3cf6c20 3.8

[bedevere-app]

GH-118185 is a backport of this pull request to the 3.11 branch.

[bedevere-app]

GH-118186 is a backport of this pull request to the 3.10 branch.

[bedevere-app]

GH-118187 is a backport of this pull request to the 3.9 branch.

[bedevere-app]

GH-118188 is a backport of this pull request to the 3.8 branch.

[sethmlarson]

Created the backports:

• [3.11] gh-116741: Upgrade libexpat to 2.6.2 (GH-117296) #118185
• [3.10] gh-116741: Upgrade libexpat to 2.6.2 (GH-117296) #118186
• [3.9] gh-116741: Upgrade libexpat to 2.6.2 (GH-117296) #118187
• [3.8] gh-116741: Upgrade libexpat to 2.6.2 (GH-117296) #118188

[hartwork]

@sethmlarson thanks a lot! 👍

[bedevere-bot]

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot AMD64 RHEL8 FIPS Only Blake2 Builtin Hash 3.11 has failed when building commit 8133285.

What do you need to do:

1. Don't panic.
2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
3. Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/962/builds/1513) and take a look at the build logs.
4. Check if the failure is related to this commit (8133285) or if it is a false positive.
5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/962/builds/1513

Failed tests:

• test.test_multiprocessing_spawn.test_processes
• test.test_multiprocessing_fork.test_processes
• test.test_multiprocessing_forkserver.test_processes

Failed subtests:

• test_empty_authkey - test.test_multiprocessing_forkserver.test_processes.WithProcessesTestListener.test_empty_authkey
• test_empty_authkey - test.test_multiprocessing_fork.test_processes.WithProcessesTestListener.test_empty_authkey
• test_empty_authkey - test.test_multiprocessing_spawn.test_processes.WithProcessesTestListener.test_empty_authkey

Summary of the results of the build (if available):

==

Click to see traceback logs

Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.11.cstratak-RHEL8-fips-x86_64.no-builtin-hashes-except-blake2/build/Lib/test/_test_multiprocessing.py", line 3483, in test_empty_authkey
    with listener.accept() as d:
         ^^^^^^^^^^^^^^^^^
  File "/home/buildbot/buildarea/3.11.cstratak-RHEL8-fips-x86_64.no-builtin-hashes-except-blake2/build/Lib/multiprocessing/connection.py", line 482, in accept
    deliver_challenge(c, self._authkey)
  File "/home/buildbot/buildarea/3.11.cstratak-RHEL8-fips-x86_64.no-builtin-hashes-except-blake2/build/Lib/multiprocessing/connection.py", line 756, in deliver_challenge
    digest = hmac.new(authkey, message, 'md5').digest()
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/buildbot/buildarea/3.11.cstratak-RHEL8-fips-x86_64.no-builtin-hashes-except-blake2/build/Lib/hmac.py", line 184, in new
    return HMAC(key, msg, digestmod)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/buildbot/buildarea/3.11.cstratak-RHEL8-fips-x86_64.no-builtin-hashes-except-blake2/build/Lib/hmac.py", line 60, in __init__
    self._init_hmac(key, msg, digestmod)
  File "/home/buildbot/buildarea/3.11.cstratak-RHEL8-fips-x86_64.no-builtin-hashes-except-blake2/build/Lib/hmac.py", line 67, in _init_hmac
    self._hmac = _hashopenssl.hmac_new(key, msg, digestmod=digestmod)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ValueError: no reason supplied


Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.11.cstratak-RHEL8-fips-x86_64.no-builtin-hashes-except-blake2/build/Lib/threading.py", line 1045, in _bootstrap_inner
    self.run()
  File "/home/buildbot/buildarea/3.11.cstratak-RHEL8-fips-x86_64.no-builtin-hashes-except-blake2/build/Lib/threading.py", line 982, in run
    self._target(*self._args, **self._kwargs)
  File "/home/buildbot/buildarea/3.11.cstratak-RHEL8-fips-x86_64.no-builtin-hashes-except-blake2/build/Lib/test/_test_multiprocessing.py", line 3474, in run
    client = self.connection.Client(addr, authkey=authkey)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/buildbot/buildarea/3.11.cstratak-RHEL8-fips-x86_64.no-builtin-hashes-except-blake2/build/Lib/multiprocessing/connection.py", line 525, in Client
    answer_challenge(c, authkey)
  File "/home/buildbot/buildarea/3.11.cstratak-RHEL8-fips-x86_64.no-builtin-hashes-except-blake2/build/Lib/multiprocessing/connection.py", line 772, in answer_challenge
    digest = hmac.new(authkey, message, 'md5').digest()
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/buildbot/buildarea/3.11.cstratak-RHEL8-fips-x86_64.no-builtin-hashes-except-blake2/build/Lib/hmac.py", line 184, in new
    return HMAC(key, msg, digestmod)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/buildbot/buildarea/3.11.cstratak-RHEL8-fips-x86_64.no-builtin-hashes-except-blake2/build/Lib/hmac.py", line 60, in __init__
    self._init_hmac(key, msg, digestmod)
  File "/home/buildbot/buildarea/3.11.cstratak-RHEL8-fips-x86_64.no-builtin-hashes-except-blake2/build/Lib/hmac.py", line 67, in _init_hmac
    self._hmac = _hashopenssl.hmac_new(key, msg, digestmod=digestmod)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ValueError: no reason supplied
ERROR