Add ruff linter configuration and apply auto-fixes

changelogs/fragments/3000-ruff-linter-config.yml

@@ -0,0 +1,18 @@
+---
+minor_changes:
+  - ruff - added comprehensive ruff linter configuration with Ansible-specific rules and backwards-compatibility safeguards (https://github.com/ansible-collections/amazon.aws/pull/3000).
+
+trivial:
+  - module_utils - standardised backwards-compatibility re-exports to use PEP 484 pattern (``import X as X``) for improved linter recognition while maintaining full compatibility (https://github.com/ansible-collections/amazon.aws/pull/3000).
+  - module_utils.botocore - modernised type hints to PEP 604 syntax (``X | None`` instead of ``Optional[X]``, ``dict`` instead of ``Dict``) for improved readability and standards compliance (https://github.com/ansible-collections/amazon.aws/pull/3000).
+  - module_utils.elb_utils - changed to re-export from public ``elbv2`` interface instead of private ``_elbv2`` modules, establishing proper layering and reducing code duplication (https://github.com/ansible-collections/amazon.aws/pull/3000).
+  - module_utils.modules - modernised type hints to PEP 604 syntax for consistency with botocore module (https://github.com/ansible-collections/amazon.aws/pull/3000).
+  - module_utils.version - changed ``LooseVersion`` re-export from function wrapper to direct class decorator, preserving class behaviour for ``isinstance()`` checks and type checkers (https://github.com/ansible-collections/amazon.aws/pull/3000).
+  - extensions - added ``extensions/`` directory to linter and formatter configurations (tox, darker, isort) to ensure EDA event sources and audit plugins are properly analysed (https://github.com/ansible-collections/amazon.aws/pull/3000).
+  - tests - added unit tests for backwards-compatibility re-export modules (core, ec2, elb_utils, version) to ensure import stability and provide line coverage (https://github.com/ansible-collections/amazon.aws/pull/3000).
+
+deprecated_features:
+  - module_utils.core - importing from ``plugins.module_utils.core`` is deprecated, import directly from the appropriate submodule instead (e.g. ``from plugins.module_utils.botocore import is_boto3_error_code``) (https://github.com/ansible-collections/amazon.aws/pull/3000).
+  - module_utils.ec2 - importing backwards-compatibility re-exports from ``plugins.module_utils.ec2`` is deprecated, import directly from the appropriate submodule instead (https://github.com/ansible-collections/amazon.aws/pull/3000).
+  - module_utils.elb_utils - importing from ``plugins.module_utils.elb_utils`` is deprecated, import directly from ``plugins.module_utils.elbv2`` instead (https://github.com/ansible-collections/amazon.aws/pull/3000).
+  - module_utils.version - importing ``LooseVersion`` from ``plugins.module_utils.version`` is deprecated, import directly from ``ansible.module_utils.compat.version`` instead (https://github.com/ansible-collections/amazon.aws/pull/3000).

docs/docsite/rst/dev_guidelines.rst

@@ -217,6 +217,37 @@ or:
        if not HAS_BOTO3:
            module.fail_json(missing_required_lib('botocore and boto3'))
 
+Importing from module_utils
+----------------------------
+
+When importing utilities from ``module_utils``, always import directly from the appropriate
+submodule rather than using the backwards-compatibility re-exports in ``core.py`` or ``ec2.py``.
+
+**Deprecated pattern** (will be removed in a future major release):
+
+.. code-block:: python
+
+   from ansible_collections.amazon.aws.plugins.module_utils.core import is_boto3_error_code
+   from ansible_collections.amazon.aws.plugins.module_utils.ec2 import ansible_dict_to_boto3_filter_list
+
+**Correct pattern**:
+
+.. code-block:: python
+
+   from ansible_collections.amazon.aws.plugins.module_utils.botocore import is_boto3_error_code
+   from ansible_collections.amazon.aws.plugins.module_utils.ec2 import ansible_dict_to_boto3_filter_list
+
+The ``core.py`` and ``ec2.py`` modules contain backwards-compatibility re-exports for historical reasons,
+but these are deprecated and will be removed in a future major release. New code should import directly
+from the source module where each function or class is defined:
+
+* ARN utilities: ``ansible_collections.amazon.aws.plugins.module_utils.arn``
+* Botocore utilities: ``ansible_collections.amazon.aws.plugins.module_utils.botocore``
+* Exception classes: ``ansible_collections.amazon.aws.plugins.module_utils.exceptions``
+* IAM policy utilities: ``ansible_collections.amazon.aws.plugins.module_utils.policy``
+* Module classes: ``ansible_collections.amazon.aws.plugins.module_utils.modules``
+* Transformation utilities: ``ansible_collections.amazon.aws.plugins.module_utils.transformation``
+
 Supporting Module Defaults
 --------------------------
 

extensions/eda/plugins/event_source/aws_cloudtrail.py

@@ -1,7 +1,9 @@
 import asyncio
 import datetime
 import json
-from typing import TYPE_CHECKING, Any, cast
+from typing import TYPE_CHECKING
+from typing import Any
+from typing import cast
 
 from aiobotocore.session import get_session
 from botocore.client import BaseClient
@@ -173,13 +175,13 @@ def connection_args(args: dict[str, Any]) -> dict[str, Any]:
 
 
 if __name__ == "__main__":
-    """MockQueue if running directly."""
+    # MockQueue if running directly.
 
     class MockQueue(asyncio.Queue[Any]):
         """A fake queue."""
 
         async def put(self: "MockQueue", event: dict[str, Any]) -> None:
             """Print the event."""
-            print(event)  # noqa: T201
+            print(event)
 
     asyncio.run(main(MockQueue(), {}))

extensions/eda/plugins/event_source/aws_sqs_queue.py

@@ -391,6 +391,6 @@ class MockQueue(asyncio.Queue[Any]):
 
         async def put(self: "MockQueue", event: dict[str, Any]) -> None:
             """Print the event."""
-            print(event)  # noqa: T201
+            print(event)
 
     asyncio.run(main(MockQueue(), {"region": "us-east-1", "name": "eda"}))

plugins/connection/aws_ssm.py

@@ -373,11 +373,6 @@
 
 if typing.TYPE_CHECKING:
     from typing import Any
-    from typing import Dict
-    from typing import Iterator
-    from typing import List
-    from typing import Optional
-    from typing import Tuple
 
 from ansible.errors import AnsibleError
 from ansible.errors import AnsibleFileNotFound
@@ -694,7 +689,7 @@ def exec_communicate(self, cmd: str, in_data: bytes | None, mark_begin: str, mar
 
             line = filter_ansi(self.session_manager.stdout_readline(), self.is_windows)
             line_count += 1
-            self.verbosity_display(4, f"EXEC_COMMUNICATE: stdout line {line_count}: {repr(line)}")
+            self.verbosity_display(4, f"EXEC_COMMUNICATE: stdout line {line_count}: {line!r}")
 
             # Check for end marker before adding line to output
             if end_search.search(line):
@@ -771,9 +766,9 @@ def exec_command(self, cmd: str, in_data: bytes | None = None, sudoable: bool =
             # DEBUG: Show what we're returning to Ansible (level 6 for long-term debugging of encoding issues)
             stdout_bytes = to_bytes(stdout, errors="surrogate_or_strict")
             stderr_bytes = to_bytes(stderr, errors="surrogate_or_strict")
-            self.verbosity_display(6, f"EXEC: Returning stdout (str): {repr(stdout[:200])}")
+            self.verbosity_display(6, f"EXEC: Returning stdout (str): {stdout[:200]!r}")
             self.verbosity_display(6, f"EXEC: Returning stdout (bytes): {stdout_bytes[:200]}")
-            self.verbosity_display(6, f"EXEC: Returning stderr (str): {repr(stderr[:200])}")
+            self.verbosity_display(6, f"EXEC: Returning stderr (str): {stderr[:200]!r}")
             self.verbosity_display(6, f"EXEC: Returning stderr (bytes): {stderr_bytes[:200]}")
 
             return (returncode, stdout_bytes, stderr_bytes)

plugins/inventory/aws_ec2.py

@@ -339,6 +339,7 @@
 from typing import Any
 from typing import Dict
 from typing import List
+from typing import Optional
 from typing import Set
 
 from ansible.errors import AnsibleError
@@ -496,8 +497,8 @@ def _get_tag_hostname(preference: str, instance: Dict[str, Any]) -> Any:
 def _prepare_host_vars(
     original_host_vars: Dict[str, Any],
     availability_zone_ids: Dict[str, str],
-    hostvars_prefix: str = None,
-    hostvars_suffix: str = None,
+    hostvars_prefix: Optional[str] = None,
+    hostvars_suffix: Optional[str] = None,
     use_contrib_script_compatible_ec2_tag_keys: bool = False,
 ) -> Dict[str, Any]:
     """
@@ -756,9 +757,9 @@ def _get_hostname_with_jinja2_filter(self, instance, preference, return_single_h
         else:
             hostname = _get_boto_attr_chain(preference, instance)
         if is_template:
-            template_var = "{{'%s'|%s}}" % (hostname, jinja2_filter)
+            template_var = "{{'%s'|%s}}" % (hostname, jinja2_filter)  # noqa: UP031
             if isinstance(hostname, list):
-                template_var = "{{%s|%s}}" % (hostname, jinja2_filter)
+                template_var = "{{%s|%s}}" % (hostname, jinja2_filter)  # noqa: UP031
             if trust_as_template:
                 template_var = trust_as_template(template_var)
             hostname = self.templar.template(variable=template_var)
@@ -940,7 +941,7 @@ def _is_matching_route53_hostname(self, hostname: str) -> bool:
         route53_hostnames = self.get_option("route53_hostnames")
         result = True
         if route53_hostnames:
-            result = any((hostname.endswith(name) for name in route53_hostnames))
+            result = any(hostname.endswith(name) for name in route53_hostnames)
         return result
 
     def _get_instance_route53_hostnames(self, instance: Dict[str, Any]) -> List[str]:
@@ -1070,7 +1071,7 @@ def _add_hosts(
     def build_include_filters(self):
         result = self.get_option("include_filters")
         if self.get_option("filters"):
-            result = [self.get_option("filters")] + result
+            result = [self.get_option("filters"), *result]
         return result or [{}]
 
     def parse(self, inventory, loader, path, cache=True):

plugins/module_utils/_elbv2/api.py

@@ -28,7 +28,6 @@
     from typing import Dict
     from typing import List
     from typing import Optional
-    from typing import Union
 
     from ..botocore import ClientType
 

plugins/module_utils/_elbv2/listeners.py

@@ -24,7 +24,6 @@
 
     from ..modules import AnsibleAWSModule
 
-from ..elb_utils import convert_tg_name_to_arn
 from ..transformation import scrub_none_parameters
 from . import api as _api
 from . import rules as _rules
@@ -230,7 +229,10 @@ def _prepare_listeners(
             tg_name = action.pop("TargetGroupName", None)
             if tg_name:
                 if tg_name not in target_group_mapping:
-                    target_group_mapping[tg_name] = convert_tg_name_to_arn(connection, module, tg_name)
+                    try:
+                        target_group_mapping[tg_name] = _api.get_target_group_arn_by_name(connection, tg_name)
+                    except AnsibleELBv2Error as e:
+                        module.fail_json_aws(e)
                 action["TargetGroupArn"] = target_group_mapping[tg_name]
         updated_listeners.append(listener)
     return updated_listeners

plugins/module_utils/_elbv2/rules.py

@@ -24,10 +24,10 @@
 
     from ..modules import AnsibleAWSModule
 
-from ..elb_utils import convert_tg_name_to_arn
 from . import actions as _actions
 from . import api as _api
 from . import transformations as _transformations
+from .common import AnsibleELBv2Error
 
 
 def _normalize_condition_values(condition: Dict[str, Any]) -> Dict[str, Any]:
@@ -360,11 +360,14 @@ def __init__(
     ) -> None:
         self.connection = connection
         self.module = module
-        self.rules = self._ensure_rules_action_has_arn(listener_rules)
-        self.changed = False
+        try:
+            self.rules = self._ensure_rules_action_has_arn(listener_rules)
+            self.changed = False
 
-        self.listener_arn = listener_arn
-        self.current_rules = _api.describe_rules(self.connection, ListenerArn=listener_arn)
+            self.listener_arn = listener_arn
+            self.current_rules = _api.describe_rules(self.connection, ListenerArn=listener_arn)
+        except AnsibleELBv2Error as e:
+            self.module.fail_json_aws(e)
 
     def _ensure_rules_action_has_arn(self, rules: List[Dict[str, Any]]) -> List[Dict[str, Any]]:
         """
@@ -380,8 +383,8 @@ def _ensure_rules_action_has_arn(self, rules: List[Dict[str, Any]]) -> List[Dict
             fixed_actions = []
             for action in rule["Actions"]:
                 if "TargetGroupName" in action:
-                    action["TargetGroupArn"] = convert_tg_name_to_arn(
-                        self.connection, self.module, action["TargetGroupName"]
+                    action["TargetGroupArn"] = _api.get_target_group_arn_by_name(
+                        self.connection, action["TargetGroupName"]
                     )
                     del action["TargetGroupName"]
                 fixed_actions.append(action)

plugins/module_utils/_elbv2/transformations.py

@@ -9,7 +9,6 @@
 
 if typing.TYPE_CHECKING:
     from typing import Dict
-    from typing import List
 
     from ..transformation import AnsibleAWSResource
     from ..transformation import AnsibleAWSResourceList

plugins/module_utils/autoscaling.py

@@ -12,7 +12,6 @@
 import typing
 
 # Not intended for general re-use / re-import
-from ._autoscaling import common as _common
 from ._autoscaling import groups as _groups
 from ._autoscaling import instances as _instances
 from ._autoscaling import transformations as _transformations
@@ -30,10 +29,13 @@
     from .transformation import BotoResourceList
 
 # Intended for general use / re-import
-AnsibleAutoScalingError = _common.AnsibleAutoScalingError
-AutoScalingErrorHandler = _common.AutoScalingErrorHandler
-WAITER_MAP = _waiters.WAITER_MAP
-transform_autoscaling_group = _transformations.transform_autoscaling_group
+# pylint: disable=unused-import,useless-import-alias
+from ._autoscaling.common import AnsibleAutoScalingError as AnsibleAutoScalingError
+from ._autoscaling.common import AutoScalingErrorHandler as AutoScalingErrorHandler
+from ._autoscaling.transformations import transform_autoscaling_group as transform_autoscaling_group
+from ._autoscaling.waiters import WAITER_MAP as WAITER_MAP
+
+# pylint: enable=unused-import,useless-import-alias
 
 
 def get_autoscaling_groups(