AttributeError: 'xml.etree.ElementTree.Element' object has no attribute 'getchildren' with Python 3.9.4

[point212]

Tried to resolve previous issue by updating python 3.8 -> 3.9
Got new problem.

3605#20 90.36 Problem: <class 'AttributeError: 'xml.etree.ElementTree.Element' object has no attribute 'getchildren'
3606#20 90.36 S3cmd:   2.1.0
3607#20 90.36 python:   3.9.4 (default, Apr  5 2021, 18:19:42) 
3608#20 90.36 [GCC 10.2.1 20210328]
3609#20 90.36 environment LANG=en_US.UTF-8
3610#20 90.36 
3611#20 90.36 Traceback (most recent call last):
3612#20 90.36   File "/usr/bin/s3cmd", line 3121, in <module>
3613#20 90.36     rc = main()
3614#20 90.36   File "/usr/bin/s3cmd", line 3030, in main
3615#20 90.36     rc = cmd_func(args)
3616#20 90.36   File "/usr/bin/s3cmd", line 1898, in cmd_sync
3617#20 90.36     return cmd_sync_local2remote(args)
3618#20 90.36   File "/usr/bin/s3cmd", line 1871, in cmd_sync_local2remote
3619#20 90.36     ret = _single_process(args[:-1])
3620#20 90.36   File "/usr/bin/s3cmd", line 1625, in _single_process
3621#20 90.36     return _child(destination_base, source_args)
3622#20 90.36   File "/usr/bin/s3cmd", line 1729, in _child
3623#20 90.36     remote_list, dst_exclude_list, remote_total_size = fetch_remote_list(destbase_with_source_list, recursive = True, require_attribs = True)
3624#20 90.36   File "/usr/lib/python3.9/site-packages/S3/FileLists.py", line 471, in fetch_remote_list
3625#20 90.36     objectlist, tmp_total_size = _get_filelist_remote(uri, recursive = True)
3626#20 90.36   File "/usr/lib/python3.9/site-packages/S3/FileLists.py", line 407, in _get_filelist_remote
3627#20 90.36     response = s3.bucket_list(remote_uri.bucket(), prefix = remote_uri.object(),
3628#20 90.36   File "/usr/lib/python3.9/site-packages/S3/S3.py", line 321, in bucket_list
3629#20 90.36     for truncated, dirs, objects in self.bucket_list_streaming(bucket, prefix, recursive, uri_params, limit):
3630#20 90.36   File "/usr/lib/python3.9/site-packages/S3/S3.py", line 356, in bucket_list_streaming
3631#20 90.36     current_list = _get_contents(response["data"])
3632#20 90.36   File "/usr/lib/python3.9/site-packages/S3/S3.py", line 339, in _get_contents
3633#20 90.36     return getListFromXml(data, "Contents")
3634#20 90.36   File "/usr/lib/python3.9/site-packages/S3/Utils.py", line 120, in getListFromXml
3635#20 90.36     return parseNodes(nodes)
3636#20 90.36   File "/usr/lib/python3.9/site-packages/S3/Utils.py", line 67, in parseNodes
3637#20 90.36     for child in node.getchildren():
3638#20 90.36 AttributeError: 'xml.etree.ElementTree.Element' object has no attribute 'getchildren'
3639#20 90.36 
3640#20 90.36 !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
3641#20 90.36     An unexpected error has occurred.
3642#20 90.36   Please try reproducing the error using
3643#20 90.36   the latest s3cmd code from the git master
3644#20 90.36   branch found at:
3645#20 90.36     https://github.com/s3tools/s3cmd
3646#20 90.36   and have a look at the known issues list:
3647#20 90.36     https://github.com/s3tools/s3cmd/wiki/Common-known-issues-and-their-solutions
3648#20 90.36   If the error persists, please report the
3649#20 90.36   above lines (removing any private
3650#20 90.36   info as necessary) to:
3651#20 90.36    [email protected]
3652#20 90.36 !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

According ocrmypdf/OCRmyPDF#583 method getchildren was deprecated since 3.4 and now it removed from python standard library.

Suppose there must be some changes in code to fix this problem.
But i have no competence in python at all.

[point212]

Oh. I have seen now it was fixed since 2.1.0
Can i ask you to made new minor release like 2.1.1 or so.
Need this fix to update Alpine linux s3cmd package

[salty-horse]

Duplicate of #1162.
Until there's a new release, you can base the package on the commit that fixes it, or patch it manually.

[taraspos]

Installing it from specific commit via pip didn't work for me:

~# pip install https://github.com/s3tools/s3cmd/archive/5834228d5bddde3819a3ae5506dd656b62dfb2d1.zip
Collecting https://github.com/s3tools/s3cmd/archive/5834228d5bddde3819a3ae5506dd656b62dfb2d1.zip
  Downloading https://github.com/s3tools/s3cmd/archive/5834228d5bddde3819a3ae5506dd656b62dfb2d1.zip (477 kB)
     |████████████████████████████████| 477 kB 300 kB/s 
Requirement already satisfied: python-dateutil in /usr/local/lib/python3.9/site-packages/python_dateutil-2.8.1-py3.9.egg (from s3cmd===2.1.0-) (2.8.1)
Requirement already satisfied: python-magic in /usr/local/lib/python3.9/site-packages/python_magic-0.4.22-py3.9.egg (from s3cmd===2.1.0-) (0.4.22)
Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.9/site-packages (from python-dateutil->s3cmd===2.1.0-) (1.15.0)
Building wheels for collected packages: s3cmd
  Building wheel for s3cmd (setup.py) ... done
  Created wheel for s3cmd: filename=s3cmd-2.1.0_-py2.py3-none-any.whl size=150977 sha256=150baa01a4f556728e6d8b4a3aac87f86d4d6eaf63230119b386d9a61f985aaa
  Stored in directory: /tmp/pip-ephem-wheel-cache-hbsgj3tg/wheels/7d/b4/64/917d47da08f0dc10c288c8704e965c151b31c0d82b7fd5d6c4
  WARNING: Built wheel for s3cmd is invalid: Metadata 1.2 mandates PEP 440 version, but '2.1.0-' is not
Failed to build s3cmd
Installing collected packages: s3cmd
    Running setup.py install for s3cmd ... done
  DEPRECATION: s3cmd was installed using the legacy 'setup.py install' method, because a wheel could not be built for it. A possible replacement is to fix the wheel build issue reported above. You can find discussion regarding this at https://github.com/pypa/pip/issues/8368.
Successfully installed s3cmd-2.1.0-

~# s3cmd 
sh: s3cmd: not found

But installing it manually via python setup.py install works fine:

ENV S3CMD_COMMIT=5834228d5bddde3819a3ae5506dd656b62dfb2d1
RUN wget https://github.com/s3tools/s3cmd/archive/$S3CMD_COMMIT.zip -O /tmp/s3cmd.zip \
    && cd /tmp/ \
    && unzip /tmp/s3cmd.zip \
    && cd /tmp/s3cmd-$S3CMD_COMMIT/ \
    && python setup.py install \
    && rm -rf /tmp/s3cmd*

[salty-horse]

@trane9991 which pip version are you using? Your commands work for me with python 3.9.0 and pip 20.0.2, without the warnings about the wheel.

Building wheels for collected packages: s3cmd
  Building wheel for s3cmd (setup.py) ... done
  Created wheel for s3cmd: filename=s3cmd-2.1.0_-py2.py3-none-any.whl size=150977 sha256=64ff51d2e13ec7f03a85aef2ce06a76ce9ab83e660d81531f011a4d8a9d7cbe2
  Stored in directory: /tmp/pip-ephem-wheel-cache-5oxmwe2w/wheels/7d/b4/64/917d47da08f0dc10c288c8704e965c151b31c0d82b7fd5d6c4
Successfully built s3cmd

[point212]

Surprised to see that creates the wheel package.
When i do python3 setup.py install last time, it's made /usr/lib/python3.9/site-packages/s3cmd-2.1.0_py3.9.egg instead
And why you noticed pip? Isn't it about setuptools only? (Sorry, i have not much experience in python ecosystem yet)

[taraspos]

@salty-horse, pip 21.0.1 here is the Dockerfile to reproduce it:

FROM python:3.9.2-alpine3.13

RUN pip -V && pip install https://github.com/s3tools/s3cmd/archive/5834228d5bddde3819a3ae5506dd656b62dfb2d1.zip

RUN s3cmd

[salty-horse]

It breaks in pip 20.3.4 due to this change:

pip wheel now verifies the built wheel contains valid metadata, and can be installed by a subsequent pip install. This can be disabled with --no-verify. (#9206)

Try:
pip wheel --no-verify https://github.com/s3tools/s3cmd/archive/5834228d5bddde3819a3ae5506dd656b62dfb2d1.zip

A new bug should be opened about the failed wheel verification.

[fviard]

Thanks to @salty-horse investigation, I have renamed the development version, and so your pip install should be working again I hope.

For the original issue, as reported, it is already fixed in MASTER and I will soon do a new release that will have it included.
Thank you all for your reports.

[salty-horse]

Sorry, I forgot to mention that after pip wheel --no-verify you still need to run pip install s3cmd-*.whl to install the built wheel.

[MurphyAdam]

Sorry, I forgot to mention that after pip wheel --no-verify you still need to run pip install s3cmd-*.whl to install the built wheel.

you are a live saver. Thank you.

[salty-horse]

@MurphyAdam this is no longer needed if you install from the latest commit:
pip install https://github.com/s3tools/s3cmd/archive/ee4b303d814d35230ee51d5b04ef51d302d35506.zip

[MurphyAdam]

Thank you once again. I initially installed from https://pypi.org/ which seem to still have not been updated with the last changes.