Skip to content

Add ruff linter configuration and apply auto-fixes - #3000

Open
tremble wants to merge 17 commits into
ansible-collections:mainfrom
tremble:ruff/bulk-cleanup
Open

Add ruff linter configuration and apply auto-fixes#3000
tremble wants to merge 17 commits into
ansible-collections:mainfrom
tremble:ruff/bulk-cleanup

Conversation

@tremble

@tremble tremble commented Jun 14, 2026

Copy link
Copy Markdown
Contributor

Summary

Adds ruff linter configuration and applies automated fixes across the codebase. This PR also standardises backwards-compatibility re-export patterns, modernises type hints, and adds comprehensive unit tests for compatibility modules.

Changes

1. Infrastructure

  • Adds extensions/ to tox format_dirs and lint_dirs
  • Adds extensions/ to pyproject.toml darker and isort src_paths
  • Ensures EDA event sources and audit plugins are covered by all linters
  • Disables commands_pre for diff-cover testenv to prevent unnecessary rsync

2. Ruff Configuration

  • Comprehensive ruff configuration in pyproject.toml
  • Enables rule categories: E, W, F, I, UP, C90, N, RUF
  • Ansible-specific ignores (E402 for DOCUMENTATION, UP009 for encoding)
  • Python 3.9+ compatibility (defers 3.10+ syntax upgrades)
  • AWS/Ansible naming exceptions (N802/N803/N806 for CamelCase APIs)
  • Custom isort sections for ansible-core/amazon.aws/community.aws
  • Marks F401/F841 as unfixable to preserve backwards-compatibility

3. Re-export Standardisation

  • Converts backwards-compatibility re-exports to PEP 484 import X as X pattern
  • Adds deprecation documentation in dev_guidelines.rst
  • Updates core.py, ec2.py, elbv2.py, elb_utils.py, s3.py, route53.py
  • Fixes version.py to use direct class decorator instead of function wrapper (preserves isinstance/type checker compatibility)
  • Fixes elb_utils.py to re-export from public elbv2 interface instead of private _elbv2 modules (reduces duplication)
  • All previously exported symbols remain available

4. Type Hint Modernisation

  • botocore.py: Modernised to PEP 604 syntax (X | None, dict instead of Dict)
  • botocore.py: Improved type hint accuracy (proper collection types for error checking functions)
  • modules.py: Modernised to PEP 604 syntax for consistency
  • Compatible with Python 3.9+ via from __future__ import annotations

5. Unit Tests

  • Added tests for backwards-compatibility re-exports: core.py, ec2.py, elb_utils.py, version.py
  • Tests verify attribute presence to ensure import stability
  • Provides line coverage for import statements without duplicating functional tests
  • Includes deprecation warning tests for version.py

6. Automated Fixes

  • Import sorting with collection-specific sections
  • Removed unused TYPE_CHECKING imports
  • pyupgrade modernisations (UP030-032, UP037)
  • ClassVar annotations (RUF012)
  • Explicit Optional[] annotations (RUF013)
  • Fixed non-breaking hyphens in s3_object.py

Backwards Compatibility

✅ All changes are backwards-compatible

  • Re-exports remain available with PEP 484 pattern
  • Deprecated imports documented in dev_guidelines.rst and changelog
  • Type hint changes are annotation-only (no runtime impact)
  • No runtime behaviour changes

Coverage

  • Achieved 80% diff coverage requirement
  • New unit tests provide coverage for compatibility module imports
  • Test coverage validates re-export stability

Test Plan

  • Verify ruff check passes
  • Verify flake8 check passes (F401 kept in flake8 ignore)
  • Run integration tests (all passing)
  • Verify backwards-compatibility imports still work
  • Verify diff coverage ≥ 80%

🤖 Generated with Claude Code

@github-actions

github-actions Bot commented Jun 14, 2026

Copy link
Copy Markdown

Docs Build 📝

Thank you for contribution!✨

The docsite for this PR is available for download as an artifact from this run:
https://github.com/ansible-collections/amazon.aws/actions/runs/27867873936

You can compare to the docs for the main branch here:
https://ansible-collections.github.io/amazon.aws/branch/main

File changes:

  • M collections/amazon/aws/docsite/dev_guidelines.html
  • M collections/amazon/aws/s3_object_module.html
Click to see the diff comparison.

NOTE: only file modifications are shown here. New and deleted files are excluded.
See the file list and check the published docs to see those files.

diff --git a/home/runner/work/amazon.aws/amazon.aws/docsbuild/base/collections/amazon/aws/docsite/dev_guidelines.html b/home/runner/work/amazon.aws/amazon.aws/docsbuild/head/collections/amazon/aws/docsite/dev_guidelines.html
index 7310b2f..ada5416 100644
--- a/home/runner/work/amazon.aws/amazon.aws/docsbuild/base/collections/amazon/aws/docsite/dev_guidelines.html
+++ b/home/runner/work/amazon.aws/amazon.aws/docsbuild/head/collections/amazon/aws/docsite/dev_guidelines.html
@@ -157,68 +157,69 @@
 <li><p><a class="reference internal" href="#naming-your-module" id="id12">Naming your module</a></p></li>
 <li><p><a class="reference internal" href="#use-boto3-and-ansibleawsmodule" id="id13">Use boto3 and AnsibleAWSModule</a></p></li>
 <li><p><a class="reference internal" href="#importing-botocore-and-boto3" id="id14">Importing botocore and boto3</a></p></li>
-<li><p><a class="reference internal" href="#supporting-module-defaults" id="id15">Supporting Module Defaults</a></p></li>
-<li><p><a class="reference internal" href="#module-behavior" id="id16">Module behavior</a></p></li>
+<li><p><a class="reference internal" href="#importing-from-module-utils" id="id15">Importing from module_utils</a></p></li>
+<li><p><a class="reference internal" href="#supporting-module-defaults" id="id16">Supporting Module Defaults</a></p></li>
+<li><p><a class="reference internal" href="#module-behavior" id="id17">Module behavior</a></p></li>
 </ul>
 </li>
-<li><p><a class="reference internal" href="#connecting-to-aws" id="id17">Connecting to AWS</a></p>
+<li><p><a class="reference internal" href="#connecting-to-aws" id="id18">Connecting to AWS</a></p>
 <ul>
-<li><p><a class="reference internal" href="#common-documentation-fragments-for-connection-parameters" id="id18">Common Documentation Fragments for Connection Parameters</a></p></li>
+<li><p><a class="reference internal" href="#common-documentation-fragments-for-connection-parameters" id="id19">Common Documentation Fragments for Connection Parameters</a></p></li>
 </ul>
 </li>
-<li><p><a class="reference internal" href="#handling-exceptions" id="id19">Handling exceptions</a></p>
+<li><p><a class="reference internal" href="#handling-exceptions" id="id20">Handling exceptions</a></p>
 <ul>
-<li><p><a class="reference internal" href="#using-is-boto3-error-code" id="id20">Using is_boto3_error_code</a></p></li>
-<li><p><a class="reference internal" href="#using-fail-json-aws" id="id21">Using fail_json_aws()</a></p></li>
-<li><p><a class="reference internal" href="#using-fail-json-and-avoiding-ansibleawsmodule" id="id22">using fail_json() and avoiding AnsibleAWSModule</a></p></li>
+<li><p><a class="reference internal" href="#using-is-boto3-error-code" id="id21">Using is_boto3_error_code</a></p></li>
+<li><p><a class="reference internal" href="#using-fail-json-aws" id="id22">Using fail_json_aws()</a></p></li>
+<li><p><a class="reference internal" href="#using-fail-json-and-avoiding-ansibleawsmodule" id="id23">using fail_json() and avoiding AnsibleAWSModule</a></p></li>
 </ul>
 </li>
-<li><p><a class="reference internal" href="#api-throttling-rate-limiting-and-pagination" id="id23">API throttling (rate limiting) and pagination</a></p></li>
-<li><p><a class="reference internal" href="#returning-values" id="id24">Returning Values</a></p>
+<li><p><a class="reference internal" href="#api-throttling-rate-limiting-and-pagination" id="id24">API throttling (rate limiting) and pagination</a></p></li>
+<li><p><a class="reference internal" href="#returning-values" id="id25">Returning Values</a></p>
 <ul>
-<li><p><a class="reference internal" href="#tags" id="id25">Tags</a></p></li>
-<li><p><a class="reference internal" href="#info-modules" id="id26">Info modules</a></p></li>
-<li><p><a class="reference internal" href="#deprecating-return-values" id="id27">Deprecating return values</a></p></li>
+<li><p><a class="reference internal" href="#tags" id="id26">Tags</a></p></li>
+<li><p><a class="reference internal" href="#info-modules" id="id27">Info modules</a></p></li>
+<li><p><a class="reference internal" href="#deprecating-return-values" id="id28">Deprecating return values</a></p></li>
 </ul>
 </li>
-<li><p><a class="reference internal" href="#dealing-with-iam-json-policy" id="id28">Dealing with IAM JSON policy</a></p></li>
-<li><p><a class="reference internal" href="#dealing-with-tags" id="id29">Dealing with tags</a></p></li>
-<li><p><a class="reference internal" href="#helper-functions" id="id30">Helper functions</a></p>
+<li><p><a class="reference internal" href="#dealing-with-iam-json-policy" id="id29">Dealing with IAM JSON policy</a></p></li>
+<li><p><a class="reference internal" href="#dealing-with-tags" id="id30">Dealing with tags</a></p></li>
+<li><p><a class="reference internal" href="#helper-functions" id="id31">Helper functions</a></p>
 <ul>
-<li><p><a class="reference internal" href="#camel-dict-to-snake-dict" id="id31">camel_dict_to_snake_dict</a></p></li>
-<li><p><a class="reference internal" href="#snake-dict-to-camel-dict" id="id32">snake_dict_to_camel_dict</a></p></li>
-<li><p><a class="reference internal" href="#ansible-dict-to-boto3-filter-list" id="id33">ansible_dict_to_boto3_filter_list</a></p></li>
-<li><p><a class="reference internal" href="#boto-exception" id="id34">boto_exception</a></p></li>
-<li><p><a class="reference internal" href="#boto3-tag-list-to-ansible-dict" id="id35">boto3_tag_list_to_ansible_dict</a></p></li>
-<li><p><a class="reference internal" href="#ansible-dict-to-boto3-tag-list" id="id36">ansible_dict_to_boto3_tag_list</a></p></li>
-<li><p><a class="reference internal" href="#get-ec2-security-group-ids-from-names" id="id37">get_ec2_security_group_ids_from_names</a></p></li>
-<li><p><a class="reference internal" href="#compare-policies" id="id38">compare_policies</a></p></li>
-<li><p><a class="reference internal" href="#compare-aws-tags" id="id39">compare_aws_tags</a></p></li>
+<li><p><a class="reference internal" href="#camel-dict-to-snake-dict" id="id32">camel_dict_to_snake_dict</a></p></li>
+<li><p><a class="reference internal" href="#snake-dict-to-camel-dict" id="id33">snake_dict_to_camel_dict</a></p></li>
+<li><p><a class="reference internal" href="#ansible-dict-to-boto3-filter-list" id="id34">ansible_dict_to_boto3_filter_list</a></p></li>
+<li><p><a class="reference internal" href="#boto-exception" id="id35">boto_exception</a></p></li>
+<li><p><a class="reference internal" href="#boto3-tag-list-to-ansible-dict" id="id36">boto3_tag_list_to_ansible_dict</a></p></li>
+<li><p><a class="reference internal" href="#ansible-dict-to-boto3-tag-list" id="id37">ansible_dict_to_boto3_tag_list</a></p></li>
+<li><p><a class="reference internal" href="#get-ec2-security-group-ids-from-names" id="id38">get_ec2_security_group_ids_from_names</a></p></li>
+<li><p><a class="reference internal" href="#compare-policies" id="id39">compare_policies</a></p></li>
+<li><p><a class="reference internal" href="#compare-aws-tags" id="id40">compare_aws_tags</a></p></li>
 </ul>
 </li>
-<li><p><a class="reference internal" href="#integration-tests-for-aws-modules" id="id40">Integration Tests for AWS Modules</a></p>
+<li><p><a class="reference internal" href="#integration-tests-for-aws-modules" id="id41">Integration Tests for AWS Modules</a></p>
 <ul>
-<li><p><a class="reference internal" href="#custom-sdk-versions-for-integration-tests" id="id41">Custom SDK versions for Integration Tests</a></p></li>
-<li><p><a class="reference internal" href="#creating-ec2-instances-in-integration-tests" id="id42">Creating EC2 instances in Integration Tests</a></p></li>
-<li><p><a class="reference internal" href="#resource-naming-in-integration-tests" id="id43">Resource naming in Integration Tests</a></p></li>
-<li><p><a class="reference internal" href="#aws-credentials-for-integration-tests" id="id44">AWS Credentials for Integration Tests</a></p></li>
-<li><p><a class="reference internal" href="#aws-permissions-for-integration-tests" id="id45">AWS Permissions for Integration Tests</a></p>
+<li><p><a class="reference internal" href="#custom-sdk-versions-for-integration-tests" id="id42">Custom SDK versions for Integration Tests</a></p></li>
+<li><p><a class="reference internal" href="#creating-ec2-instances-in-integration-tests" id="id43">Creating EC2 instances in Integration Tests</a></p></li>
+<li><p><a class="reference internal" href="#resource-naming-in-integration-tests" id="id44">Resource naming in Integration Tests</a></p></li>
+<li><p><a class="reference internal" href="#aws-credentials-for-integration-tests" id="id45">AWS Credentials for Integration Tests</a></p></li>
+<li><p><a class="reference internal" href="#aws-permissions-for-integration-tests" id="id46">AWS Permissions for Integration Tests</a></p>
 <ul>
-<li><p><a class="reference internal" href="#troubleshooting-iam-policies" id="id46">Troubleshooting IAM policies</a></p></li>
-<li><p><a class="reference internal" href="#unsupported-integration-tests" id="id47">Unsupported Integration tests</a></p></li>
+<li><p><a class="reference internal" href="#troubleshooting-iam-policies" id="id47">Troubleshooting IAM policies</a></p></li>
+<li><p><a class="reference internal" href="#unsupported-integration-tests" id="id48">Unsupported Integration tests</a></p></li>
 </ul>
 </li>
 </ul>
 </li>
-<li><p><a class="reference internal" href="#unit-tests-for-aws-plugins" id="id48">Unit-tests for AWS plugins</a></p>
+<li><p><a class="reference internal" href="#unit-tests-for-aws-plugins" id="id49">Unit-tests for AWS plugins</a></p>
 <ul>
-<li><p><a class="reference internal" href="#why-do-we-need-unit-tests-when-we-ve-got-functional-tests" id="id49">Why do we need unit-tests when we’ve got functional tests</a></p></li>
-<li><p><a class="reference internal" href="#how-to-keep-my-code-simple" id="id50">How to keep my code simple?</a></p></li>
-<li><p><a class="reference internal" href="#unit-tests-guidelines" id="id51">Unit-tests guidelines</a></p></li>
-<li><p><a class="reference internal" href="#how-to-run-my-unit-tests" id="id52">How to run my unit-tests</a></p></li>
+<li><p><a class="reference internal" href="#why-do-we-need-unit-tests-when-we-ve-got-functional-tests" id="id50">Why do we need unit-tests when we’ve got functional tests</a></p></li>
+<li><p><a class="reference internal" href="#how-to-keep-my-code-simple" id="id51">How to keep my code simple?</a></p></li>
+<li><p><a class="reference internal" href="#unit-tests-guidelines" id="id52">Unit-tests guidelines</a></p></li>
+<li><p><a class="reference internal" href="#how-to-run-my-unit-tests" id="id53">How to run my unit-tests</a></p></li>
 </ul>
 </li>
-<li><p><a class="reference internal" href="#code-formatting" id="id53">Code formatting</a></p></li>
+<li><p><a class="reference internal" href="#code-formatting" id="id54">Code formatting</a></p></li>
 </ul>
 </nav>
 <section id="requirements">
@@ -388,8 +389,34 @@ as the module does that check:</p>
 </pre></div>
 </div>
 </section>
+<section id="importing-from-module-utils">
+<h3><a class="toc-backref" href="#id15" role="doc-backlink">Importing from module_utils</a><a class="headerlink" href="#importing-from-module-utils" title="Link to this heading"></a></h3>
+<p>When importing utilities from <code class="docutils literal notranslate"><span class="pre">module_utils</span></code>, always import directly from the appropriate
+submodule rather than using the backwards-compatibility re-exports in <code class="docutils literal notranslate"><span class="pre">core.py</span></code> or <code class="docutils literal notranslate"><span class="pre">ec2.py</span></code>.</p>
+<p><strong>Deprecated pattern</strong> (will be removed in a future major release):</p>
+<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="kn">from</span><span class="w"> </span><span class="nn">ansible_collections.amazon.aws.plugins.module_utils.core</span><span class="w"> </span><span class="kn">import</span> <span class="n">is_boto3_error_code</span>
+<span class="kn">from</span><span class="w"> </span><span class="nn">ansible_collections.amazon.aws.plugins.module_utils.ec2</span><span class="w"> </span><span class="kn">import</span> <span class="n">ansible_dict_to_boto3_filter_list</span>
+</pre></div>
+</div>
+<p><strong>Correct pattern</strong>:</p>
+<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="kn">from</span><span class="w"> </span><span class="nn">ansible_collections.amazon.aws.plugins.module_utils.botocore</span><span class="w"> </span><span class="kn">import</span> <span class="n">is_boto3_error_code</span>
+<span class="kn">from</span><span class="w"> </span><span class="nn">ansible_collections.amazon.aws.plugins.module_utils.ec2</span><span class="w"> </span><span class="kn">import</span> <span class="n">ansible_dict_to_boto3_filter_list</span>
+</pre></div>
+</div>
+<p>The <code class="docutils literal notranslate"><span class="pre">core.py</span></code> and <code class="docutils literal notranslate"><span class="pre">ec2.py</span></code> modules contain backwards-compatibility re-exports for historical reasons,
+but these are deprecated and will be removed in a future major release. New code should import directly
+from the source module where each function or class is defined:</p>
+<ul class="simple">
+<li><p>ARN utilities: <code class="docutils literal notranslate"><span class="pre">ansible_collections.amazon.aws.plugins.module_utils.arn</span></code></p></li>
+<li><p>Botocore utilities: <code class="docutils literal notranslate"><span class="pre">ansible_collections.amazon.aws.plugins.module_utils.botocore</span></code></p></li>
+<li><p>Exception classes: <code class="docutils literal notranslate"><span class="pre">ansible_collections.amazon.aws.plugins.module_utils.exceptions</span></code></p></li>
+<li><p>IAM policy utilities: <code class="docutils literal notranslate"><span class="pre">ansible_collections.amazon.aws.plugins.module_utils.policy</span></code></p></li>
+<li><p>Module classes: <code class="docutils literal notranslate"><span class="pre">ansible_collections.amazon.aws.plugins.module_utils.modules</span></code></p></li>
+<li><p>Transformation utilities: <code class="docutils literal notranslate"><span class="pre">ansible_collections.amazon.aws.plugins.module_utils.transformation</span></code></p></li>
+</ul>
+</section>
 <section id="supporting-module-defaults">
-<h3><a class="toc-backref" href="#id15" role="doc-backlink">Supporting Module Defaults</a><a class="headerlink" href="#supporting-module-defaults" title="Link to this heading"></a></h3>
+<h3><a class="toc-backref" href="#id16" role="doc-backlink">Supporting Module Defaults</a><a class="headerlink" href="#supporting-module-defaults" title="Link to this heading"></a></h3>
 <p>The existing AWS modules support using <a class="reference external" href="https://docs.ansible.com/projects/ansible-core/devel/playbook_guide/playbooks_module_defaults.html#module-defaults" title="(in Ansible Core devel)"><span class="xref std std-ref">module_defaults</span></a> for common
 authentication parameters.  To do the same for your new module, add an entry for it in
 <code class="docutils literal notranslate"><span class="pre">meta/runtime.yml</span></code>.  These entries take the form of:</p>
@@ -401,7 +428,7 @@ authentication parameters.  To do the same for your new module, add an entry for
 </div>
 </section>
 <section id="module-behavior">
-<h3><a class="toc-backref" href="#id16" role="doc-backlink">Module behavior</a><a class="headerlink" href="#module-behavior" title="Link to this heading"></a></h3>
+<h3><a class="toc-backref" href="#id17" role="doc-backlink">Module behavior</a><a class="headerlink" href="#module-behavior" title="Link to this heading"></a></h3>
 <p>To reduce the chance of breaking changes occurring when new features are added,
 the module should avoid modifying the resource attribute when a parameter is
 not explicitly set in a task.</p>
@@ -415,7 +442,7 @@ section for an example with <code class="docutils literal notranslate"><span cla
 </section>
 </section>
 <section id="connecting-to-aws">
-<span id="ansible-collections-amazon-aws-docsite-dev-module-connection"></span><h2><a class="toc-backref" href="#id17" role="doc-backlink">Connecting to AWS</a><a class="headerlink" href="#connecting-to-aws" title="Link to this heading"></a></h2>
+<span id="ansible-collections-amazon-aws-docsite-dev-module-connection"></span><h2><a class="toc-backref" href="#id18" role="doc-backlink">Connecting to AWS</a><a class="headerlink" href="#connecting-to-aws" title="Link to this heading"></a></h2>
 <p>AnsibleAWSModule provides the <code class="docutils literal notranslate"><span class="pre">resource</span></code> and <code class="docutils literal notranslate"><span class="pre">client</span></code> helper methods for obtaining boto3 connections.
 These handle some of the more esoteric connection options, such as security tokens and boto profiles.</p>
 <p>If using the basic AnsibleModule then you should use <code class="docutils literal notranslate"><span class="pre">get_aws_connection_info</span></code> and then <code class="docutils literal notranslate"><span class="pre">boto3_conn</span></code>
@@ -443,7 +470,7 @@ See exception handling:</p>
 </pre></div>
 </div>
 <section id="common-documentation-fragments-for-connection-parameters">
-<h3><a class="toc-backref" href="#id18" role="doc-backlink">Common Documentation Fragments for Connection Parameters</a><a class="headerlink" href="#common-documentation-fragments-for-connection-parameters" title="Link to this heading"></a></h3>
+<h3><a class="toc-backref" href="#id19" role="doc-backlink">Common Documentation Fragments for Connection Parameters</a><a class="headerlink" href="#common-documentation-fragments-for-connection-parameters" title="Link to this heading"></a></h3>
 <p>There are four <a class="reference external" href="https://docs.ansible.com/projects/ansible-core/devel/dev_guide/developing_modules_documenting.html#module-docs-fragments" title="(in Ansible Core devel)"><span class="xref std std-ref">common documentation fragments</span></a>
 that should be included into almost all AWS modules:</p>
 <ul class="simple">
@@ -487,7 +514,7 @@ and that the more esoteric connection options are documented. For example:</p>
 </section>
 </section>
 <section id="handling-exceptions">
-<span id="ansible-collections-amazon-aws-docsite-dev-exceptions"></span><h2><a class="toc-backref" href="#id19" role="doc-backlink">Handling exceptions</a><a class="headerlink" href="#handling-exceptions" title="Link to this heading"></a></h2>
+<span id="ansible-collections-amazon-aws-docsite-dev-exceptions"></span><h2><a class="toc-backref" href="#id20" role="doc-backlink">Handling exceptions</a><a class="headerlink" href="#handling-exceptions" title="Link to this heading"></a></h2>
 <p>You should wrap any boto3 or botocore call in a try block. If an exception is thrown, then there
 are a number of possibilities for handling it.</p>
 <ul class="simple">
@@ -503,7 +530,7 @@ are a number of possibilities for handling it.</p>
 </ul>
 <p>For more information on botocore exception handling see the <a class="reference external" href="https://botocore.readthedocs.io/en/latest/client_upgrades.html#error-handling">botocore error documentation</a>.</p>
 <section id="using-is-boto3-error-code">
-<h3><a class="toc-backref" href="#id20" role="doc-backlink">Using is_boto3_error_code</a><a class="headerlink" href="#using-is-boto3-error-code" title="Link to this heading"></a></h3>
+<h3><a class="toc-backref" href="#id21" role="doc-backlink">Using is_boto3_error_code</a><a class="headerlink" href="#using-is-boto3-error-code" title="Link to this heading"></a></h3>
 <p>To use <code class="docutils literal notranslate"><span class="pre">ansible_collections.amazon.aws.plugins.module_utils.botocore.is_boto3_error_code</span></code> to catch a single
 AWS error code, call it in place of <code class="docutils literal notranslate"><span class="pre">ClientError</span></code> in your except clauses. In
 this example, <em>only</em> the <code class="docutils literal notranslate"><span class="pre">InvalidGroup.NotFound</span></code> error code will be caught here,
@@ -517,7 +544,7 @@ and any other error will be raised for handling elsewhere in the program.</p>
 </div>
 </section>
 <section id="using-fail-json-aws">
-<h3><a class="toc-backref" href="#id21" role="doc-backlink">Using fail_json_aws()</a><a class="headerlink" href="#using-fail-json-aws" title="Link to this heading"></a></h3>
+<h3><a class="toc-backref" href="#id22" role="doc-backlink">Using fail_json_aws()</a><a class="headerlink" href="#using-fail-json-aws" title="Link to this heading"></a></h3>
 <p>In the AnsibleAWSModule there is a special method, <code class="docutils literal notranslate"><span class="pre">module.fail_json_aws()</span></code> for nice reporting of
 exceptions.  Call this on your exception and it will report the error together with a traceback for
 use in Ansible verbose mode.</p>
@@ -554,7 +581,7 @@ expect anything other than botocore exceptions you should test everything works
 </div>
 </section>
 <section id="using-fail-json-and-avoiding-ansibleawsmodule">
-<h3><a class="toc-backref" href="#id22" role="doc-backlink">using fail_json() and avoiding AnsibleAWSModule</a><a class="headerlink" href="#using-fail-json-and-avoiding-ansibleawsmodule" title="Link to this heading"></a></h3>
+<h3><a class="toc-backref" href="#id23" role="doc-backlink">using fail_json() and avoiding AnsibleAWSModule</a><a class="headerlink" href="#using-fail-json-and-avoiding-ansibleawsmodule" title="Link to this heading"></a></h3>
 <p>Boto3 provides lots of useful information when an exception is thrown so pass this to the user
 along with the message.</p>
 <div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="kn">from</span><span class="w"> </span><span class="nn">ansible_collections.amazon.aws.plugins.module_utils.botocore</span><span class="w"> </span><span class="kn">import</span> <span class="n">HAS_BOTO3</span>
@@ -597,7 +624,7 @@ work with python3</p>
 </section>
 </section>
 <section id="api-throttling-rate-limiting-and-pagination">
-<span id="ansible-collections-amazon-aws-docsite-dev-ratelimits"></span><h2><a class="toc-backref" href="#id23" role="doc-backlink">API throttling (rate limiting) and pagination</a><a class="headerlink" href="#api-throttling-rate-limiting-and-pagination" title="Link to this heading"></a></h2>
+<span id="ansible-collections-amazon-aws-docsite-dev-ratelimits"></span><h2><a class="toc-backref" href="#id24" role="doc-backlink">API throttling (rate limiting) and pagination</a><a class="headerlink" href="#api-throttling-rate-limiting-and-pagination" title="Link to this heading"></a></h2>
 <p>For methods that return a lot of results, boto3 often provides
 <a class="reference external" href="https://boto3.readthedocs.io/en/latest/guide/paginators.html">paginators</a>. If the method
 you’re calling has <code class="docutils literal notranslate"><span class="pre">NextToken</span></code> or <code class="docutils literal notranslate"><span class="pre">Marker</span></code> parameters, you should probably
@@ -674,7 +701,7 @@ the <code class="docutils literal notranslate"><span class="pre">cloudformation<
 </dl>
 </section>
 <section id="returning-values">
-<span id="ansible-collections-amazon-aws-docsite-dev-return"></span><h2><a class="toc-backref" href="#id24" role="doc-backlink">Returning Values</a><a class="headerlink" href="#returning-values" title="Link to this heading"></a></h2>
+<span id="ansible-collections-amazon-aws-docsite-dev-return"></span><h2><a class="toc-backref" href="#id25" role="doc-backlink">Returning Values</a><a class="headerlink" href="#returning-values" title="Link to this heading"></a></h2>
 <p>When you make a call using boto3, you will probably get back some useful information that you
 should return in the module.  As well as information related to the call itself, you will also have
 some response metadata.  It is OK to return this to the user as well as they may find it useful.</p>
@@ -699,7 +726,7 @@ of a dict.  This is particularly useful for tags, where keys are case-sensitive.
 should be a sensible approximation of the resource name.  For example, <code class="docutils literal notranslate"><span class="pre">volume</span></code> for <code class="docutils literal notranslate"><span class="pre">ec2_vol</span></code>,
 <code class="docutils literal notranslate"><span class="pre">volumes</span></code> for <code class="docutils literal notranslate"><span class="pre">ec2_vol_info</span></code>.</p>
 <section id="tags">
-<h3><a class="toc-backref" href="#id25" role="doc-backlink">Tags</a><a class="headerlink" href="#tags" title="Link to this heading"></a></h3>
+<h3><a class="toc-backref" href="#id26" role="doc-backlink">Tags</a><a class="headerlink" href="#tags" title="Link to this heading"></a></h3>
 <p>Tags should be returned as a dictionary of key: value pairs, with each key being the tag’s
 key and value being the tag’s value.  It should be noted, however, that boto3 often returns tags
 as a list of dictionaries.</p>
@@ -722,7 +749,7 @@ from boto3’s returned tag list to the desired dictionary of tags to be returne
 </div>
 </section>
 <section id="info-modules">
-<h3><a class="toc-backref" href="#id26" role="doc-backlink">Info modules</a><a class="headerlink" href="#info-modules" title="Link to this heading"></a></h3>
+<h3><a class="toc-backref" href="#id27" role="doc-backlink">Info modules</a><a class="headerlink" href="#info-modules" title="Link to this heading"></a></h3>
 <p>Info modules that can return information on multiple resources should return a list of
 dictionaries, with each dictionary containing information about that particular resource
 (i.e. <code class="docutils literal notranslate"><span class="pre">security_groups</span></code> in <code class="docutils literal notranslate"><span class="pre">ec2_group_info</span></code>).</p>
@@ -775,7 +802,7 @@ match either the parameter name itself, or an alias of that parameter.</p>
 </div>
 </section>
 <section id="deprecating-return-values">
-<h3><a class="toc-backref" href="#id27" role="doc-backlink">Deprecating return values</a><a class="headerlink" href="#deprecating-return-values" title="Link to this heading"></a></h3>
+<h3><a class="toc-backref" href="#id28" role="doc-backlink">Deprecating return values</a><a class="headerlink" href="#deprecating-return-values" title="Link to this heading"></a></h3>
 <p>If changes need to be made to current return values, the new/”correct” keys should be
 returned <strong>in addition to</strong> the existing keys to preserve compability with existing
 playbooks. A deprecation should be added to the return values being replaced, initially
@@ -789,7 +816,7 @@ placed at least 2 years out, on the 1st of a month.</p>
 </section>
 </section>
 <section id="dealing-with-iam-json-policy">
-<span id="ansible-collections-amazon-aws-docsite-dev-policies"></span><h2><a class="toc-backref" href="#id28" role="doc-backlink">Dealing with IAM JSON policy</a><a class="headerlink" href="#dealing-with-iam-json-policy" title="Link to this heading"></a></h2>
+<span id="ansible-collections-amazon-aws-docsite-dev-policies"></span><h2><a class="toc-backref" href="#id29" role="doc-backlink">Dealing with IAM JSON policy</a><a class="headerlink" href="#dealing-with-iam-json-policy" title="Link to this heading"></a></h2>
 <p>If your module accepts IAM JSON policies then set the type to ‘json’ in the module spec. For
 example:</p>
 <div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="n">argument_spec</span><span class="o">.</span><span class="n">update</span><span class="p">(</span>
@@ -824,7 +851,7 @@ and returns True if they are different.</p>
 </div>
 </section>
 <section id="dealing-with-tags">
-<span id="ansible-collections-amazon-aws-docsite-dev-tags"></span><h2><a class="toc-backref" href="#id29" role="doc-backlink">Dealing with tags</a><a class="headerlink" href="#dealing-with-tags" title="Link to this heading"></a></h2>
+<span id="ansible-collections-amazon-aws-docsite-dev-tags"></span><h2><a class="toc-backref" href="#id30" role="doc-backlink">Dealing with tags</a><a class="headerlink" href="#dealing-with-tags" title="Link to this heading"></a></h2>
 <p>AWS has a concept of resource tags. Usually the boto3 API has separate calls
 for tagging and untagging a resource.  For example, the EC2 API has
 <code class="docutils literal notranslate"><span class="pre">create_tags</span></code> and <code class="docutils literal notranslate"><span class="pre">delete_tags</span></code> calls.</p>
@@ -853,11 +880,11 @@ for more detail.</p>
 included when adding tagging support.</p>
 </section>
 <section id="helper-functions">
-<span id="ansible-collections-amazon-aws-docsite-dev-helpers"></span><h2><a class="toc-backref" href="#id30" role="doc-backlink">Helper functions</a><a class="headerlink" href="#helper-functions" title="Link to this heading"></a></h2>
+<span id="ansible-collections-amazon-aws-docsite-dev-helpers"></span><h2><a class="toc-backref" href="#id31" role="doc-backlink">Helper functions</a><a class="headerlink" href="#helper-functions" title="Link to this heading"></a></h2>
 <p>Along with the connection functions in Ansible ec2.py module_utils, there are some other useful
 functions detailed below.</p>
 <section id="camel-dict-to-snake-dict">
-<h3><a class="toc-backref" href="#id31" role="doc-backlink">camel_dict_to_snake_dict</a><a class="headerlink" href="#camel-dict-to-snake-dict" title="Link to this heading"></a></h3>
+<h3><a class="toc-backref" href="#id32" role="doc-backlink">camel_dict_to_snake_dict</a><a class="headerlink" href="#camel-dict-to-snake-dict" title="Link to this heading"></a></h3>
 <p>boto3 returns results in a dict.  The keys of the dict are in CamelCase format. In keeping with
 Ansible format, this function will convert the keys to snake_case.</p>
 <p><code class="docutils literal notranslate"><span class="pre">camel_dict_to_snake_dict</span></code> takes an optional parameter called <code class="docutils literal notranslate"><span class="pre">ignore_list</span></code> which is a list of
@@ -868,23 +895,23 @@ which would then be converted by <code class="docutils literal notranslate"><spa
 Passing <code class="docutils literal notranslate"><span class="pre">reversible=True</span></code> converts HTTPEndpoint to <code class="docutils literal notranslate"><span class="pre">h_t_t_p_endpoint</span></code> which converts back to <code class="docutils literal notranslate"><span class="pre">HTTPEndpoint</span></code>.</p>
 </section>
 <section id="snake-dict-to-camel-dict">
-<h3><a class="toc-backref" href="#id32" role="doc-backlink">snake_dict_to_camel_dict</a><a class="headerlink" href="#snake-dict-to-camel-dict" title="Link to this heading"></a></h3>
+<h3><a class="toc-backref" href="#id33" role="doc-backlink">snake_dict_to_camel_dict</a><a class="headerlink" href="#snake-dict-to-camel-dict" title="Link to this heading"></a></h3>
 <p><code class="docutils literal notranslate"><span class="pre">snake_dict_to_camel_dict</span></code> converts snake cased keys to camel case. By default, because it was
 first introduced for ECS purposes, this converts to dromedaryCase. An optional
 parameter called <code class="docutils literal notranslate"><span class="pre">capitalize_first</span></code>, which defaults to <code class="docutils literal notranslate"><span class="pre">False</span></code>, can be used to convert to CamelCase.</p>
 </section>
 <section id="ansible-dict-to-boto3-filter-list">
-<h3><a class="toc-backref" href="#id33" role="doc-backlink">ansible_dict_to_boto3_filter_list</a><a class="headerlink" href="#ansible-dict-to-boto3-filter-list" title="Link to this heading"></a></h3>
+<h3><a class="toc-backref" href="#id34" role="doc-backlink">ansible_dict_to_boto3_filter_list</a><a class="headerlink" href="#ansible-dict-to-boto3-filter-list" title="Link to this heading"></a></h3>
 <p>Converts a an Ansible list of filters to a boto3 friendly list of dicts.  This is useful for any
 boto3 <code class="docutils literal notranslate"><span class="pre">_facts</span></code> modules.</p>
 </section>
 <section id="boto-exception">
-<h3><a class="toc-backref" href="#id34" role="doc-backlink">boto_exception</a><a class="headerlink" href="#boto-exception" title="Link to this heading"></a></h3>
+<h3><a class="toc-backref" href="#id35" role="doc-backlink">boto_exception</a><a class="headerlink" href="#boto-exception" title="Link to this heading"></a></h3>
 <p>Pass an exception returned from boto or boto3, and this function will consistently get the message from the exception.</p>
 <p>Deprecated: use <code class="docutils literal notranslate"><span class="pre">AnsibleAWSModule</span></code>’s <code class="docutils literal notranslate"><span class="pre">fail_json_aws</span></code> instead.</p>
 </section>
 <section id="boto3-tag-list-to-ansible-dict">
-<h3><a class="toc-backref" href="#id35" role="doc-backlink">boto3_tag_list_to_ansible_dict</a><a class="headerlink" href="#boto3-tag-list-to-ansible-dict" title="Link to this heading"></a></h3>
+<h3><a class="toc-backref" href="#id36" role="doc-backlink">boto3_tag_list_to_ansible_dict</a><a class="headerlink" href="#boto3-tag-list-to-ansible-dict" title="Link to this heading"></a></h3>
 <p>Converts a boto3 tag list to an Ansible dict. Boto3 returns tags as a list of dicts containing keys
 called ‘Key’ and ‘Value’ by default.  This key names can be overridden when calling the function.
 For example, if you have already camel_cased your list of tags you may want to pass lowercase key
@@ -893,25 +920,25 @@ names instead, in other words, ‘key’ and ‘value’.</p>
 value is the tag value.</p>
 </section>
 <section id="ansible-dict-to-boto3-tag-list">
-<h3><a class="toc-backref" href="#id36" role="doc-backlink">ansible_dict_to_boto3_tag_list</a><a class="headerlink" href="#ansible-dict-to-boto3-tag-list" title="Link to this heading"></a></h3>
+<h3><a class="toc-backref" href="#id37" role="doc-backlink">ansible_dict_to_boto3_tag_list</a><a class="headerlink" href="#ansible-dict-to-boto3-tag-list" title="Link to this heading"></a></h3>
 <p>Opposite of above. Converts an Ansible dict to a boto3 tag list of dicts. You can again override
 the key names used if ‘Key’ and ‘Value’ is not suitable.</p>
 </section>
 <section id="get-ec2-security-group-ids-from-names">
-<h3><a class="toc-backref" href="#id37" role="doc-backlink">get_ec2_security_group_ids_from_names</a><a class="headerlink" href="#get-ec2-security-group-ids-from-names" title="Link to this heading"></a></h3>
+<h3><a class="toc-backref" href="#id38" role="doc-backlink">get_ec2_security_group_ids_from_names</a><a class="headerlink" href="#get-ec2-security-group-ids-from-names" title="Link to this heading"></a></h3>
 <p>Pass this function a list of security group names or combination of security group names and IDs
 and this function will return a list of IDs.  You should also pass the VPC ID if known because
 security group names are not necessarily unique across VPCs.</p>
 </section>
 <section id="compare-policies">
-<h3><a class="toc-backref" href="#id38" role="doc-backlink">compare_policies</a><a class="headerlink" href="#compare-policies" title="Link to this heading"></a></h3>
+<h3><a class="toc-backref" href="#id39" role="doc-backlink">compare_policies</a><a class="headerlink" href="#compare-policies" title="Link to this heading"></a></h3>
 <p>Pass two dicts of policies to check if there are any meaningful differences and returns true
 if there are. This recursively sorts the dicts and makes them hashable before comparison.</p>
 <p>This method should be used any time policies are being compared so that a change in order
 doesn’t result in unnecessary changes.</p>
 </section>
 <section id="compare-aws-tags">
-<h3><a class="toc-backref" href="#id39" role="doc-backlink">compare_aws_tags</a><a class="headerlink" href="#compare-aws-tags" title="Link to this heading"></a></h3>
+<h3><a class="toc-backref" href="#id40" role="doc-backlink">compare_aws_tags</a><a class="headerlink" href="#compare-aws-tags" title="Link to this heading"></a></h3>
 <p>Pass two dicts of tags and an optional purge parameter and this function will return a dict
 containing key pairs you need to modify and a list of tag key names that you need to remove.  Purge
 is True by default.  If purge is False then any existing tags will not be modified.</p>
@@ -923,7 +950,7 @@ without modification for some (Lambda) and others may need modification before u
 </section>
 </section>
 <section id="integration-tests-for-aws-modules">
-<span id="ansible-collections-amazon-aws-docsite-dev-tests"></span><h2><a class="toc-backref" href="#id40" role="doc-backlink">Integration Tests for AWS Modules</a><a class="headerlink" href="#integration-tests-for-aws-modules" title="Link to this heading"></a></h2>
+<span id="ansible-collections-amazon-aws-docsite-dev-tests"></span><h2><a class="toc-backref" href="#id41" role="doc-backlink">Integration Tests for AWS Modules</a><a class="headerlink" href="#integration-tests-for-aws-modules" title="Link to this heading"></a></h2>
 <p>All new AWS modules should include integration tests to ensure that any changes in AWS APIs that
 affect the module are detected. At a minimum this should cover the key API calls and check the
 documented return values are present in the module result.</p>
@@ -937,7 +964,7 @@ continuous integration build.</p>
 <p>Tests for new modules should be added to the <code class="docutils literal notranslate"><span class="pre">cloud/aws</span></code> group. In general just copy
 an existing aliases file such as the <a class="reference external" href="https://github.com/ansible-collections/amazon.aws/blob/master/tests/integration/targets/aws_s3/aliases">aws_s3 tests aliases file</a>.</p>
 <section id="custom-sdk-versions-for-integration-tests">
-<h3><a class="toc-backref" href="#id41" role="doc-backlink">Custom SDK versions for Integration Tests</a><a class="headerlink" href="#custom-sdk-versions-for-integration-tests" title="Link to this heading"></a></h3>
+<h3><a class="toc-backref" href="#id42" role="doc-backlink">Custom SDK versions for Integration Tests</a><a class="headerlink" href="#custom-sdk-versions-for-integration-tests" title="Link to this heading"></a></h3>
 <p>By default integration tests will run against the earliest supported version of
 the AWS SDK.  The current supported versions can be found in
 <code class="docutils literal notranslate"><span class="pre">tests/integration/constraints.txt</span></code> and should not be updated.  Where a module
@@ -952,7 +979,7 @@ the <code class="docutils literal notranslate"><span class="pre">meta/main.yml</
 </div>
 </section>
 <section id="creating-ec2-instances-in-integration-tests">
-<h3><a class="toc-backref" href="#id42" role="doc-backlink">Creating EC2 instances in Integration Tests</a><a class="headerlink" href="#creating-ec2-instances-in-integration-tests" title="Link to this heading"></a></h3>
+<h3><a class="toc-backref" href="#id43" role="doc-backlink">Creating EC2 instances in Integration Tests</a><a class="headerlink" href="#creating-ec2-instances-in-integration-tests" title="Link to this heading"></a></h3>
 <p>When started, the integration tests will be passed <code class="docutils literal notranslate"><span class="pre">aws_region</span></code> as an extra var.
 Any resources created should be created in in this region, this includes EC2
 instances.  Since AMIs are region specific there is a role which can be
@@ -982,7 +1009,7 @@ dependency in the <code class="docutils literal notranslate"><span class="pre">m
 role and the fact it provides to chose an AMI to use.</p>
 </section>
 <section id="resource-naming-in-integration-tests">
-<h3><a class="toc-backref" href="#id43" role="doc-backlink">Resource naming in Integration Tests</a><a class="headerlink" href="#resource-naming-in-integration-tests" title="Link to this heading"></a></h3>
+<h3><a class="toc-backref" href="#id44" role="doc-backlink">Resource naming in Integration Tests</a><a class="headerlink" href="#resource-naming-in-integration-tests" title="Link to this heading"></a></h3>
 <p>AWS has a range of limitations for the name of resources.  Where possible,
 resource names should include a string which makes the resource names unique
 to the test.</p>
@@ -991,7 +1018,7 @@ helpful extra vars: <code class="docutils literal notranslate"><span class="pre"
 test set, and should generally used as part of the name.  <code class="docutils literal notranslate"><span class="pre">resource_prefix</span></code> will generate a prefix based on the host the test is being run on.  Sometimes this may result in a resource name that exceeds the character limit allowed by AWS.  In these cases, <code class="docutils literal notranslate"><span class="pre">tiny_prefix</span></code> will provide a 12-character randomly generated prefix.</p>
 </section>
 <section id="aws-credentials-for-integration-tests">
-<h3><a class="toc-backref" href="#id44" role="doc-backlink">AWS Credentials for Integration Tests</a><a class="headerlink" href="#aws-credentials-for-integration-tests" title="Link to this heading"></a></h3>
+<h3><a class="toc-backref" href="#id45" role="doc-backlink">AWS Credentials for Integration Tests</a><a class="headerlink" href="#aws-credentials-for-integration-tests" title="Link to this heading"></a></h3>
 <p>The testing framework handles running the test with appropriate AWS credentials, these are made available
 to your test in the following variables:</p>
 <ul class="simple">
@@ -1023,7 +1050,7 @@ for every call, it’s preferable to use <a class="reference external" href="htt
 </div>
 </section>
 <section id="aws-permissions-for-integration-tests">
-<h3><a class="toc-backref" href="#id45" role="doc-backlink">AWS Permissions for Integration Tests</a><a class="headerlink" href="#aws-permissions-for-integration-tests" title="Link to this heading"></a></h3>
+<h3><a class="toc-backref" href="#id46" role="doc-backlink">AWS Permissions for Integration Tests</a><a class="headerlink" href="#aws-permissions-for-integration-tests" title="Link to this heading"></a></h3>
 <p>As explained in the <a class="reference external" href="https://docs.ansible.com/projects/ansible-core/devel/dev_guide/testing_integration.html#testing-integration" title="(in Ansible Core devel)"><span class="xref std std-ref">Integration Test guide</span></a>
 there are defined IAM policies in <a class="reference external" href="https://github.com/mattclay/aws-terminator">mattclay/aws-terminator</a> that contain the necessary permissions
 to run the AWS integration test.</p>
@@ -1083,7 +1110,7 @@ run the tests.</p>
 <li><p>Open a pull request proposing the minimum required policy to the <a class="reference external" href="https://github.com/mattclay/aws-terminator/tree/master/aws/policy">CI policies</a>.</p></li>
 </ol>
 <section id="troubleshooting-iam-policies">
-<h4><a class="toc-backref" href="#id46" role="doc-backlink">Troubleshooting IAM policies</a><a class="headerlink" href="#troubleshooting-iam-policies" title="Link to this heading"></a></h4>
+<h4><a class="toc-backref" href="#id47" role="doc-backlink">Troubleshooting IAM policies</a><a class="headerlink" href="#troubleshooting-iam-policies" title="Link to this heading"></a></h4>
 <ul class="simple">
 <li><p>When you make changes to a policy, wait a few minutes for the policy to update before re-running the tests.</p></li>
 <li><p>Use the <a class="reference external" href="https://policysim.aws.amazon.com/">policy simulator</a> to verify that each action (limited by resource when applicable) in your policy is allowed.</p></li>
@@ -1097,7 +1124,7 @@ run the tests.</p>
 </ul>
 </section>
 <section id="unsupported-integration-tests">
-<h4><a class="toc-backref" href="#id47" role="doc-backlink">Unsupported Integration tests</a><a class="headerlink" href="#unsupported-integration-tests" title="Link to this heading"></a></h4>
+<h4><a class="toc-backref" href="#id48" role="doc-backlink">Unsupported Integration tests</a><a class="headerlink" href="#unsupported-integration-tests" title="Link to this heading"></a></h4>
 <p>There are a limited number of reasons why it may not be practical to run integration
 tests for a module within CI.  Where these apply you should add the keyword
 <code class="docutils literal notranslate"><span class="pre">unsupported</span></code> to the aliases file in <code class="docutils literal notranslate"><span class="pre">test/integration/targets/MODULE_NAME/aliases</span></code>.</p>
@@ -1116,14 +1143,14 @@ someone performing a PR review or writing a patch.</p>
 </section>
 </section>
 <section id="unit-tests-for-aws-plugins">
-<h2><a class="toc-backref" href="#id48" role="doc-backlink">Unit-tests for AWS plugins</a><a class="headerlink" href="#unit-tests-for-aws-plugins" title="Link to this heading"></a></h2>
+<h2><a class="toc-backref" href="#id49" role="doc-backlink">Unit-tests for AWS plugins</a><a class="headerlink" href="#unit-tests-for-aws-plugins" title="Link to this heading"></a></h2>
 <section id="why-do-we-need-unit-tests-when-we-ve-got-functional-tests">
-<h3><a class="toc-backref" href="#id49" role="doc-backlink">Why do we need unit-tests when we’ve got functional tests</a><a class="headerlink" href="#why-do-we-need-unit-tests-when-we-ve-got-functional-tests" title="Link to this heading"></a></h3>
+<h3><a class="toc-backref" href="#id50" role="doc-backlink">Why do we need unit-tests when we’ve got functional tests</a><a class="headerlink" href="#why-do-we-need-unit-tests-when-we-ve-got-functional-tests" title="Link to this heading"></a></h3>
 <p>Unit-tests are much faster and more suitable to test corner cases. They also don’t depend on a third party service
 and thus, a failure is less likely to be a false positive.</p>
 </section>
 <section id="how-to-keep-my-code-simple">
-<h3><a class="toc-backref" href="#id50" role="doc-backlink">How to keep my code simple?</a><a class="headerlink" href="#how-to-keep-my-code-simple" title="Link to this heading"></a></h3>
+<h3><a class="toc-backref" href="#id51" role="doc-backlink">How to keep my code simple?</a><a class="headerlink" href="#how-to-keep-my-code-simple" title="Link to this heading"></a></h3>
 <p>Ideally, you should break up your code in tiny functions. Each function should have a limited number of parameters
 and a low amount of cross dependencies with the rest of the code (low coupling):</p>
 <ul class="simple">
@@ -1138,7 +1165,7 @@ Pass the parameter directly to your function. By doing so, you’re explicit abo
 </ul>
 </section>
 <section id="unit-tests-guidelines">
-<h3><a class="toc-backref" href="#id51" role="doc-backlink">Unit-tests guidelines</a><a class="headerlink" href="#unit-tests-guidelines" title="Link to this heading"></a></h3>
+<h3><a class="toc-backref" href="#id52" role="doc-backlink">Unit-tests guidelines</a><a class="headerlink" href="#unit-tests-guidelines" title="Link to this heading"></a></h3>
 <p>Ideally, all the <code class="docutils literal notranslate"><span class="pre">module_utils</span></code> should be covered by unit-tests. However we acknowledge that writing unit-tests may
 be challenging and we also accept contribution with no unit-test. Generally speaking, unit-tests are recommended and likely to speed up the PR reviews.</p>
 <ul class="simple">
@@ -1151,7 +1178,7 @@ be challenging and we also accept contribution with no unit-test. Generally spea
 <p>Pytest is well documented and you will find some example in its <a class="reference external" href="https://docs.pytest.org/en/latest/how-to/index.html">how-to guides</a></p>
 </section>
 <section id="how-to-run-my-unit-tests">
-<h3><a class="toc-backref" href="#id52" role="doc-backlink">How to run my unit-tests</a><a class="headerlink" href="#how-to-run-my-unit-tests" title="Link to this heading"></a></h3>
+<h3><a class="toc-backref" href="#id53" role="doc-backlink">How to run my unit-tests</a><a class="headerlink" href="#how-to-run-my-unit-tests" title="Link to this heading"></a></h3>
 <p>In our CI, the testing is done by <code class="docutils literal notranslate"><span class="pre">ansible-test</span></code>. You can run the tests locally with the following command:</p>
 <div class="highlight-shell notranslate"><div class="highlight"><pre><span></span>$<span class="w"> </span>ansible-test<span class="w"> </span>units<span class="w"> </span>--docker
 </pre></div>
@@ -1163,7 +1190,7 @@ be challenging and we also accept contribution with no unit-test. Generally spea
 </section>
 </section>
 <section id="code-formatting">
-<h2><a class="toc-backref" href="#id53" role="doc-backlink">Code formatting</a><a class="headerlink" href="#code-formatting" title="Link to this heading"></a></h2>
+<h2><a class="toc-backref" href="#id54" role="doc-backlink">Code formatting</a><a class="headerlink" href="#code-formatting" title="Link to this heading"></a></h2>
 <p>To improve the consistency of our code we use a number of formatters and linters.  These tools can
 be run locally by using tox:</p>
 <div class="highlight-shell notranslate"><div class="highlight"><pre><span></span>$<span class="w"> </span>tox<span class="w"> </span>-m<span class="w"> </span>format
diff --git a/home/runner/work/amazon.aws/amazon.aws/docsbuild/base/collections/amazon/aws/s3_object_module.html b/home/runner/work/amazon.aws/amazon.aws/docsbuild/head/collections/amazon/aws/s3_object_module.html
index c906654..aefae9d 100644
--- a/home/runner/work/amazon.aws/amazon.aws/docsbuild/base/collections/amazon/aws/s3_object_module.html
+++ b/home/runner/work/amazon.aws/amazon.aws/docsbuild/head/collections/amazon/aws/s3_object_module.html
@@ -408,7 +408,7 @@ see <a class="reference internal" href="#ansible-collections-amazon-aws-s3-objec
 <a class="ansibleOptionLink" href="#parameter-headers" title="Permalink to this option"></a><p class="ansible-option-type-line"><span class="ansible-option-type">dictionary</span></p>
 </div></td>
 <td><div class="ansible-option-cell"><p>Custom headers to use when <code class="ansible-option-value docutils literal notranslate"><a class="reference internal" href="#ansible-collections-amazon-aws-s3-object-module-parameter-mode"><span class="std std-ref"><span class="pre">mode=put</span></span></a></code> as a dictionary of key value pairs.</p>
-<p>Keys that match supported S3 object arguments are promoted to top‑level ExtraArgs. Supported keys include <code class="docutils literal notranslate"><span class="pre">ContentType</span></code>, <code class="docutils literal notranslate"><span class="pre">Content-Disposition</span></code>, <code class="docutils literal notranslate"><span class="pre">ContentEncoding</span></code>, <code class="docutils literal notranslate"><span class="pre">ContentLanguage</span></code>, <code class="docutils literal notranslate"><span class="pre">CacheControl</span></code>, <code class="docutils literal notranslate"><span class="pre">Expires</span></code>, <code class="docutils literal notranslate"><span class="pre">ACL</span></code>, <code class="docutils literal notranslate"><span class="pre">GrantFullControl</span></code>, <code class="docutils literal notranslate"><span class="pre">GrantRead</span></code>, <code class="docutils literal notranslate"><span class="pre">GrantReadACP</span></code>, <code class="docutils literal notranslate"><span class="pre">GrantWriteACP</span></code>, <code class="docutils literal notranslate"><span class="pre">StorageClass</span></code>, <code class="docutils literal notranslate"><span class="pre">ServerSideEncryption</span></code>, <code class="docutils literal notranslate"><span class="pre">SSECustomerAlgorithm</span></code>, <code class="docutils literal notranslate"><span class="pre">SSECustomerKey</span></code>, <code class="docutils literal notranslate"><span class="pre">SSECustomerKeyMD5</span></code>, <code class="docutils literal notranslate"><span class="pre">SSEKMSKeyId</span></code>, and <code class="docutils literal notranslate"><span class="pre">WebsiteRedirectLocation</span></code>. Remaining keys are stored under <code class="docutils literal notranslate"><span class="pre">Metadata</span></code>.</p>
+<p>Keys that match supported S3 object arguments are promoted to top-level ExtraArgs. Supported keys include <code class="docutils literal notranslate"><span class="pre">ContentType</span></code>, <code class="docutils literal notranslate"><span class="pre">Content-Disposition</span></code>, <code class="docutils literal notranslate"><span class="pre">ContentEncoding</span></code>, <code class="docutils literal notranslate"><span class="pre">ContentLanguage</span></code>, <code class="docutils literal notranslate"><span class="pre">CacheControl</span></code>, <code class="docutils literal notranslate"><span class="pre">Expires</span></code>, <code class="docutils literal notranslate"><span class="pre">ACL</span></code>, <code class="docutils literal notranslate"><span class="pre">GrantFullControl</span></code>, <code class="docutils literal notranslate"><span class="pre">GrantRead</span></code>, <code class="docutils literal notranslate"><span class="pre">GrantReadACP</span></code>, <code class="docutils literal notranslate"><span class="pre">GrantWriteACP</span></code>, <code class="docutils literal notranslate"><span class="pre">StorageClass</span></code>, <code class="docutils literal notranslate"><span class="pre">ServerSideEncryption</span></code>, <code class="docutils literal notranslate"><span class="pre">SSECustomerAlgorithm</span></code>, <code class="docutils literal notranslate"><span class="pre">SSECustomerKey</span></code>, <code class="docutils literal notranslate"><span class="pre">SSECustomerKeyMD5</span></code>, <code class="docutils literal notranslate"><span class="pre">SSEKMSKeyId</span></code>, and <code class="docutils literal notranslate"><span class="pre">WebsiteRedirectLocation</span></code>. Remaining keys are stored under <code class="docutils literal notranslate"><span class="pre">Metadata</span></code>.</p>
 <p>Both header name styles <code class="docutils literal notranslate"><span class="pre">ContentType</span></code> and <code class="docutils literal notranslate"><span class="pre">Content-Type</span></code> are supported. Underscore variants like <code class="docutils literal notranslate"><span class="pre">content_type</span></code> are not promoted and will be placed under <code class="docutils literal notranslate"><span class="pre">Metadata</span></code>.</p>
 <p>If <code class="docutils literal notranslate"><span class="pre">ContentType</span></code> is not provided, a type is guessed from <code class="docutils literal notranslate"><span class="pre">src</span></code> when uploading from a file. When uploading using <code class="docutils literal notranslate"><span class="pre">content</span></code> or <code class="docutils literal notranslate"><span class="pre">content_base64</span></code> the default is <code class="docutils literal notranslate"><span class="pre">binary/octet-stream</span></code>.</p>
 <p>Ignored when <code class="ansible-option docutils literal notranslate"><strong><a class="reference internal" href="#ansible-collections-amazon-aws-s3-object-module-parameter-mode"><span class="std std-ref"><span class="pre">mode</span></span></a></strong></code> is not <code class="ansible-value docutils literal notranslate"><span class="pre">put</span></code>.</p>
@@ -448,7 +448,7 @@ see <a class="reference internal" href="#ansible-collections-amazon-aws-s3-objec
 <a class="ansibleOptionLink" href="#parameter-metadata" title="Permalink to this option"></a><p class="ansible-option-type-line"><span class="ansible-option-type">dictionary</span></p>
 </div></td>
 <td><div class="ansible-option-cell"><p>Metadata to use when <code class="ansible-option-value docutils literal notranslate"><a class="reference internal" href="#ansible-collections-amazon-aws-s3-object-module-parameter-mode"><span class="std std-ref"><span class="pre">mode=copy</span></span></a></code>, <code class="ansible-option-value docutils literal notranslate"><a class="reference internal" href="#ansible-collections-amazon-aws-s3-object-module-parameter-mode"><span class="std std-ref"><span class="pre">mode=create</span></span></a></code> or <code class="ansible-option-value docutils literal notranslate"><a class="reference internal" href="#ansible-collections-amazon-aws-s3-object-module-parameter-mode"><span class="std std-ref"><span class="pre">mode=put</span></span></a></code> as a dictionary of key value pairs.</p>
-<p>Keys that match supported S3 object arguments are promoted to top‑level ExtraArgs (for example <code class="docutils literal notranslate"><span class="pre">ContentType</span></code>). Remaining keys are applied as user metadata under <code class="docutils literal notranslate"><span class="pre">Metadata</span></code>.</p>
+<p>Keys that match supported S3 object arguments are promoted to top-level ExtraArgs (for example <code class="docutils literal notranslate"><span class="pre">ContentType</span></code>). Remaining keys are applied as user metadata under <code class="docutils literal notranslate"><span class="pre">Metadata</span></code>.</p>
 </div></td>
 </tr>
 <tr class="row-even"><td><div class="ansible-option-cell">

@tremble
tremble force-pushed the ruff/bulk-cleanup branch 3 times, most recently from a25f591 to 0619af0 Compare June 14, 2026 11:25
@centosinfra-prod-github-app

Copy link
Copy Markdown
Contributor

Build failed.
https://gateway-cloud-softwarefactory.apps.ocp.cloud.ci.centos.org/zuul/t/ansible/buildset/6104c5d436fe4c89810ee26570ce94f4

✔️ ansible-galaxy-importer SUCCESS in 3m 53s
✔️ build-ansible-collection SUCCESS in 10m 57s
✔️ ansible-test-splitter SUCCESS in 4m 23s
integration-amazon.aws-1 FAILURE in 50m 45s
✔️ integration-amazon.aws-2 SUCCESS in 31m 06s
✔️ integration-amazon.aws-3 SUCCESS in 29m 18s
✔️ integration-amazon.aws-4 SUCCESS in 20m 49s
✔️ integration-amazon.aws-5 SUCCESS in 32m 03s
✔️ integration-amazon.aws-6 SUCCESS in 33m 01s
✔️ integration-amazon.aws-7 SUCCESS in 37m 45s
✔️ integration-amazon.aws-8 SUCCESS in 17m 03s
✔️ integration-amazon.aws-9 SUCCESS in 21m 04s
✔️ integration-amazon.aws-10 SUCCESS in 43m 39s
✔️ integration-amazon.aws-11 SUCCESS in 34m 51s
✔️ integration-amazon.aws-12 SUCCESS in 43m 08s
✔️ integration-amazon.aws-13 SUCCESS in 46m 55s
✔️ integration-amazon.aws-14 SUCCESS in 43m 44s
✔️ integration-amazon.aws-15 SUCCESS in 39m 07s
✔️ integration-amazon.aws-16 SUCCESS in 40m 33s
✔️ integration-amazon.aws-17 SUCCESS in 26m 50s
✔️ integration-amazon.aws-18 SUCCESS in 45m 51s
✔️ integration-amazon.aws-19 SUCCESS in 32m 00s
✔️ integration-amazon.aws-20 SUCCESS in 27m 30s
✔️ integration-amazon.aws-21 SUCCESS in 22m 02s
✔️ integration-amazon.aws-22 SUCCESS in 31m 13s
✔️ integration-community.aws-1 SUCCESS in 22m 57s
integration-community.aws-2 FAILURE in 32m 39s
✔️ integration-community.aws-3 SUCCESS in 37m 12s
✔️ integration-community.aws-4 SUCCESS in 3m 10s
✔️ integration-community.aws-5 SUCCESS in 21m 22s
✔️ integration-community.aws-6 SUCCESS in 16m 25s
✔️ integration-community.aws-7 SUCCESS in 14m 29s
✔️ integration-community.aws-8 SUCCESS in 22m 14s
✔️ integration-community.aws-9 SUCCESS in 11m 55s
✔️ integration-community.aws-10 SUCCESS in 10m 12s
✔️ integration-community.aws-11 SUCCESS in 22m 12s
✔️ integration-community.aws-12 SUCCESS in 9m 21s
integration-community.aws-13 FAILURE in 10m 48s
✔️ integration-community.aws-14 SUCCESS in 6m 49s
✔️ integration-community.aws-15 SUCCESS in 18m 21s
✔️ integration-community.aws-16 SUCCESS in 4m 17s
Skipped 6 jobs

tremble added 2 commits June 19, 2026 14:34
Adds extensions/ to the list of directories checked by:
- tox: format_dirs and lint_dirs (affects black, isort, flynt, flake8,
  pylint, ruff, ansible-lint)
- darker: src paths for incremental formatting
- isort: src_paths for import sorting

The extensions/ directory is a new location for Python code (EDA event
sources and audit plugins) that was previously not covered by formatters
and some linters.

Assisted-by: Claude Sonnet 4.5
Adds comprehensive ruff configuration to pyproject.toml with rules aligned
to existing flake8, pylint, and black configurations.

Configuration includes:
- Rule categories: E, W, F, I, UP, C90, N, RUF
- Ansible-specific ignores (E402 for DOCUMENTATION, UP009 for encoding)
- Python 3.9+ compatibility (defer 3.10+ syntax upgrades)
- AWS/Ansible naming exceptions (N802/N803/N806 for CamelCase APIs)
- Complexity threshold matching pylint (max-complexity 15)
- Custom isort sections for ansible-core/amazon.aws/community.aws
- Per-file ignores for ec2_metadata_facts, ec2_security_group, conftest

Marks F401 (unused-import) and F841 (unused-variable) as unfixable to
preserve backwards-compatibility re-exports and avoid breaking changes.

Removes obsolete black exclude for plugins/module_utils/_version.py
(file no longer exists).

Assisted-by: Claude Sonnet 4.5
@tremble
tremble force-pushed the ruff/bulk-cleanup branch from 0619af0 to 91d87f1 Compare June 19, 2026 13:47
@centosinfra-prod-github-app

Copy link
Copy Markdown
Contributor

tremble and others added 10 commits June 19, 2026 17:06
Converts backwards-compatibility re-exports to use PEP 484 pattern
(import X as X) for consistent linter recognition while maintaining
full backwards compatibility.

Changes by file:
- core.py: Updated docstring, kept all 9 re-exports with PEP 484 pattern
- ec2.py: Moved deprecated re-exports to top, added deprecation notice,
  restored all 22 backwards-compatibility imports
- elbv2.py: Converted 47+ re-exports to PEP 484 pattern
- elb_utils.py: Converted API function re-exports to PEP 484 pattern
- s3.py: Converted re-exports to PEP 484 pattern
- route53.py: Converted re-exports to assignment pattern (compatibility)
- version.py: Added @deprecated decorator wrapper for static analysis

Documentation:
- dev_guidelines.rst: New section documenting deprecated vs correct
  import patterns with examples and migration guide

The PEP 484 pattern allows ruff to recognise intentional re-exports
without F401 warnings while flake8 compatibility is maintained via
ignore list. All previously exported symbols remain available.

Version.py uses typing_extensions.deprecated (or warnings.deprecated
in Python 3.13+) to enable type checkers to flag deprecated usage.

Assisted-by: Claude Sonnet 4.5
Applies automated fixes from ruff covering import sorting, type
annotations, and code modernisation.

Changes include:
- Import sorting (I001): Organised imports into standard sections
  with ansible-core/amazon.aws/community.aws custom sections
- Import deduplication: Removed redundant typing imports
- pyupgrade (UP):
  - UP030-032: Modernised format strings
  - UP037: Removed quoted type annotations
- ClassVar annotations (RUF012): Added to class-level constants
- Implicit optional (RUF013): Made Optional[] explicit
- List/dict literal concatenation (RUF015): Simplified expressions
- Non-breaking hyphen fixes: Replaced U+2011 with ASCII hyphen
- Unused import removal: Cleaned up TYPE_CHECKING imports and
    removed incorrect import paths

All fixes are backwards-compatible and do not change runtime behaviour.
Import sorting follows collection-specific sections to maintain proper
dependency order.

Assisted-by: Claude Sonnet 4.5
Add minimal unit tests for backwards-compatibility re-export modules
to ensure imports are present and provide line coverage:

- test_core.py: Tests core.py re-exports (parse_aws_arn, HAS_BOTO3, etc.)
- test_ec2.py: Tests ec2.py re-exports (dict transformations, boto3 helpers, etc.)
- test_elb_utils.py: Tests elb_utils.py re-exports and wrapper functions
- test_version.py: Tests version.py LooseVersion re-export and deprecation

These tests verify attribute presence only, not functionality, as the
actual implementations are tested elsewhere. This provides coverage for
import statements in compatibility modules without duplicating functional tests.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Add empty commands_pre to diff-cover testenv to prevent inherited
rsync commands from running. diff-cover analyzes existing coverage.xml
files and doesn't need collection setup.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Replace deprecated typing imports with modern Python 3.9+ syntax:
- Optional[X] → X | None
- Union[X, Y] → X | Y
- Dict/List/Set/Tuple/Type → dict/list/set/tuple/type

Compatible with Python 3.9+ via 'from __future__ import annotations'.
Removes unused typing imports (Dict, List, Optional, Set, Tuple, Type).

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Correct type hints to be more precise:
- get_aws_connection_info: Add dict[str, Any] instead of bare dict
- gather_sdk_versions: Return dict[str, str] for version strings
- is_boto3_error_code: Accept str | list[str] | tuple[str, ...] | set[str]
- is_boto3_error_httpstatus: Accept int | list[int] | tuple[int, ...] | set[int]
- normalize_boto3_result: Remove set (json doesn't support sets)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Replace deprecated typing imports with modern Python 3.9+ syntax:
- Optional[X] → X | None
- Dict → dict

Compatible with Python 3.9+ via 'from __future__ import annotations'.
Removes unused typing imports (Dict, Optional).

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Add trivial entries for:
- Type hint modernization (botocore, modules)
- elb_utils re-export layering fix
- version.py LooseVersion class decorator fix
- Unit tests for compatibility re-exports

Add elb_utils to deprecated_features list.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
@tremble
tremble force-pushed the ruff/bulk-cleanup branch from 91d87f1 to c969cd4 Compare June 19, 2026 16:46
@centosinfra-prod-github-app

Copy link
Copy Markdown
Contributor

@centosinfra-prod-github-app

Copy link
Copy Markdown
Contributor

@centosinfra-prod-github-app

Copy link
Copy Markdown
Contributor

Breaking circular dependency between elb_utils and elbv2 by refactoring
internal _elbv2 modules to call API functions directly with exception
handling.

Changes:
- Promote get_load_balancer_by_name, get_target_group_arn_by_name, and
  get_listener_rules to public API in elbv2.py
- Update elb_utils.py to re-export from public elbv2 interface instead
  of private _elbv2 modules
- Refactor ElasticLoadBalancerV2.__init__ and update() to call API
  directly with try/except blocks instead of using get_elb wrapper
- Update _elbv2/listeners.py _prepare_listeners() to call API directly
  with exception handling
- Update _elbv2/rules.py _ensure_rules_action_has_arn() to call API
  directly with exception handling
- All exception handling properly encapsulated in methods with access
  to module.fail_json_aws()

This reduces code duplication, establishes clearer API boundaries, and
fixes the import loop: elb_utils → elbv2 → _elbv2/listeners → elb_utils

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
@tremble
tremble force-pushed the ruff/bulk-cleanup branch from 4b86fe4 to e00d3dc Compare June 19, 2026 18:16
@centosinfra-prod-github-app

Copy link
Copy Markdown
Contributor

Replace bare `_` with `_version` in test_version.py to satisfy pylint
disallowed-name check while maintaining intent to indicate unused variable.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
@centosinfra-prod-github-app

Copy link
Copy Markdown
Contributor

…_config

Convert to parameterised pytest tests covering all DHCP option types:
- Empty configuration
- domain-name (single and multiple values)
- domain-name-servers (single and multiple values)
- ntp-servers (single and multiple values)
- netbios-name-servers (single and multiple values)
- netbios-node-type (both integer and list formats)
- Combined configuration

This achieves 100% coverage for ec2.py:normalize_ec2_vpc_dhcp_config,
specifically covering the netbios-node-type list format path (line 1785).

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
@centosinfra-prod-github-app

Copy link
Copy Markdown
Contributor

tremble added 2 commits June 20, 2026 11:40
Move hostname and availability zone tests to dedicated test files for
better organisation. Add new test coverage for Route53 hostname
matching and Jinja2 filter template formatting.

Assisted-by: Claude Sonnet 4.5
Remove unused AnsibleError import from test_aws_ec2.py and unused
_get_boto_attr_chain/_get_tag_hostname imports from
test_hostname_helpers.py (functions are monkeypatched instead).

Also remove unused _elbv2.api import from elbv2.py module_utils.

Assisted-by: Claude Sonnet 4.5
@sonarqubecloud

Copy link
Copy Markdown

@centosinfra-prod-github-app

Copy link
Copy Markdown
Contributor

@tremble
tremble marked this pull request as ready for review June 20, 2026 19:59
@tremble

tremble commented Jun 20, 2026

Copy link
Copy Markdown
Contributor Author

recheck

@centosinfra-prod-github-app

Copy link
Copy Markdown
Contributor

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant