General upgrade of dependencies

[AlanCoding]

SUMMARY

General run of

./updater.sh upgrade

This requires pulling in yaml/pyyaml#808 for cython stuff, thank you @nitzmahone. One of our unrelated libraries put a >3 constraint for cython, so this affects pyyaml and other stuff.

Otherwise the libraries seem to upgrade without issue.

ISSUE TYPE

• Bug, Docs Fix or other nominal change

COMPONENT NAME

• API

[AlanCoding]

Failures seem to be on:

  File "/awx_devel/awx/main/models/ha.py", line 57, in save
    super(BaseModel, self).save(*args, **kwargs)
  File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/django/db/models/base.py", line 814, in save
    self.save_base(
  File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/django/db/models/base.py", line 892, in save_base
    post_save.send(
  File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/django/dispatch/dispatcher.py", line 176, in send
    return [
           ^
  File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/django/dispatch/dispatcher.py", line 177, in <listcomp>
    (receiver, receiver(signal=self, sender=sender, **named))
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/awx_devel/awx/main/signals.py", line 435, in activity_stream_create
    changes = model_to_dict(instance, model_serializer_mapping())
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/awx_devel/awx/main/utils/common.py", line 440, in model_to_dict
    allowed_fields = get_allowed_fields(obj, serializer_mapping)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/awx_devel/awx/main/utils/common.py", line 359, in get_allowed_fields
    allowed_fields = [x for x in serializer_actual.fields if not serializer_actual.fields[x].read_only] + ['id']
                                 ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/django/utils/functional.py", line 57, in __get__
    res = instance.__dict__[self.name] = self.func(instance)
                                         ^^^^^^^^^^^^^^^^^^^
  File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/rest_framework/serializers.py", line 372, in fields
    for key, value in self.get_fields().items():
                      ^^^^^^^^^^^^^^^^^
  File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/rest_framework/serializers.py", line 1113, in get_fields
    fields[field_name] = field_class(**field_kwargs)
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/rest_framework/fields.py", line 992, in __init__
    warnings.warn("min_value should be a Decimal instance.")
UserWarning: min_value should be a Decimal instance.

This is documented somewhere...

[AlanCoding]

Link

• min_value in DecimalField should be Decimal type. #15664
• Min value should be Decimal #15413 from @kdelee

[AlanCoding]

Now the remaining work to be done is clear.

Exception: Included licenses not consistent with requirements:
 license file annotated-types.txt does not correspond to an existing requirement; it should be removed.
 license file azure-common.txt does not correspond to an existing requirement; it should be removed.
 license file docutils.txt does not correspond to an existing requirement; it should be removed.
 license file inflect.txt does not correspond to an existing requirement; it should be removed.
 embedded source for jwcrypto is 1.5.4 instead of the required version 1.5.6
 embedded source for psycopg is 3.1.18 instead of the required version 3.2.3
 license file pydantic.txt does not correspond to an existing requirement; it should be removed.
 license file pydantic-core.txt does not correspond to an existing requirement; it should be removed.
 embedded source for uwsgi is 2.0.24 instead of the required version 2.0.28
 license for requirement aiohappyeyeballs is missing
 license for requirement backports-tarfile is missing
 license for requirement durationpy is missing
 license for requirement importlib-resources is missing
 license for requirement propcache is missing

[AlanCoding]

So I think @jessicamack is going to take over this from here

[AlanCoding]

From logs of failing checks we have:

  File "/awx_devel/awx/main/management/commands/run_dispatcher.py", line 71, in handle
    consumer.run()
  File "/awx_devel/awx/main/dispatch/worker/base.py", line 245, in run
    for e in conn.events(yield_timeouts=True):
  File "/awx_devel/awx/main/dispatch/__init__.py", line 90, in events
    for notification in notification_generator:
  File "/awx_devel/awx/main/dispatch/__init__.py", line 75, in current_notifies
    enc = psycopg._encodings.pgconn_encoding(conn.pgconn)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: module 'psycopg._encodings' has no attribute 'pgconn_encoding'. Did you mean: 'conn_encoding'?

And this seems to be a good find because psycopg/psycopg#834 said:

I am doing a little refactoring to get rid of functions such as pgconn_encoding() and error_message().

And the suggested change from that is to replace pgconn_encoding use with connection._encoding, which that being the "inner" psycopg connection.

[AlanCoding]

Trying a full-scale deployment of this, running into this on the superuser creation command.

  File "/bin/awx-manage", line 8, in <module>
    sys.exit(manage())
             ^^^^^^^^
  File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/awx/__init__.py", line 101, in manage
    execute_from_command_line(sys.argv)
  File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/django/core/management/__init__.py", line 442, in execute_from_command_line
    utility.execute()
  File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/django/core/management/__init__.py", line 416, in execute
    django.setup()
  File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/django/__init__.py", line 24, in setup
    apps.populate(settings.INSTALLED_APPS)
  File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/django/apps/registry.py", line 116, in populate
    app_config.import_models()
  File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/django/apps/config.py", line 269, in import_models
    self.models_module = import_module(models_module_name)
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.11/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1204, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1147, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 940, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/awx/conf/models.py", line 13, in <module>
    from awx.main.models.base import CreatedModifiedModel
  File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/awx/main/models/__init__.py", line 23, in <module>
    from awx.main.models.projects import Project, ProjectUpdate  # noqa
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/awx/main/models/projects.py", line 25, in <module>
    from awx.main.models.notifications import (
  File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/awx/main/models/notifications.py", line 33, in <module>
    from awx.main.notifications.irc_backend import IrcBackend
  File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/awx/main/notifications/irc_backend.py", line 8, in <module>
    import irc.client
  File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/irc/client.py", line 69, in <module>
    from jaraco.stream import buffer
ModuleNotFoundError: No module named 'jaraco.stream'

[AlanCoding]

Interesting. We get an import error from this missing dependency, but it seems to be there. Don't know why we wouldn't have it.

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

[codecov]

Codecov Report

Attention: Patch coverage is 0% with 1 line in your changes missing coverage. Please review.

Project coverage is 75.21%. Comparing base (9a5ed20) to head (e292e84).
Report is 91 commits behind head on devel.

✅ All tests successful. No failed tests found.

🚀 New features to boost your workflow:

• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.