Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[backport] [stable-5] fix/elbv2-tg: KeyError when getting ProtocolVersion #1808

Conversation

tremble
Copy link
Contributor

@tremble tremble commented May 5, 2023

Manual backport of #1800

SUMMARY

Fix KeyError on module elb_target_group.

Given the task:

- name: Target | AWS | Create {{ tg.name }}
  community.aws.elb_target_group:
    state: present
    region: "{{ tg.region | d(omit) }}"
    name: "{{ tg.name }}"
    vpc_id: "{{ vpc_id }}"
    tags: "{{ tg.tags | d(omit) }}"
    purge_tags: "{{ tg.purge_tags | d('no') }}"
    protocol: "{{ tg.protocol }}"
    port: "{{ tg.port }}"
    # Health Check
    health_check_protocol: "{{ tg.health_check_protocol }}"
    health_check_path: "{{ tg.health_check_path | d(omit) }}"
    health_check_port: "{{ tg.health_check_port | d(omit) }}"
    successful_response_codes: "{{ tg.successful_response_codes | d(omit) }}"
    health_check_interval: "{{ tg.health_check_interval | d(omit) }}"
    health_check_timeout: "{{ tg.health_check_timeout | d(omit) }}"
    healthy_threshold_count: "{{ tg.healthy_threshold_count | d(omit) }}"
    unhealthy_threshold_count: "{{ tg.unhealthy_threshold_count | d(omit) }}"
    # Targets
    target_type: "{{ tg.target_type | d(omit) }}"
    modify_targets: "{{ tg.modify_targets | d(omit) }}"
    tagets: "{{ tg.tagets | d(omit) }}"
    # Config
    stickiness_enabled: "{{ tg.stickiness_enabled | d(omit) }}"
    stickiness_app_cookie_duration: "{{ tg.stickiness_app_cookie_duration | d(omit) }}"
    stickiness_app_cookie_name: "{{ tg.stickiness_app_cookie_name | d(omit) }}"
    stickiness_lb_cookie_duration: "{{ tg.stickiness_lb_cookie_duration | d(omit) }}"
    stickiness_type: "{{ tg.stickiness_type | d(omit) }}"
    proxy_protocol_v2_enabled: "{{ tg.proxy_protocol_v2_enabled | d(omit) }}"
    preserve_client_ip_enabled: "{{ tg.preserve_client_ip_enabled | d(omit) }}"
    deregistration_delay_timeout: "{{ tg.deregistration_delay_timeout | d(omit) }}"
  register: tg_out
  until: "tg_out is not failed"
  retries: 10
  delay: 5
  • The following error when running against existing resources (after creation/idempotency) that do not use ProtocolVersion (optional*, E.g: TCP):
The full traceback is:
Traceback (most recent call last):
  File "/home/myuser/.ansible/tmp/ansible-tmp-1683068584.628099-3322568-261537723315094/AnsiballZ_elb_target_group.py", line 107, in <module>
    _ansiballz_main()
  File "/home/myuser/.ansible/tmp/ansible-tmp-1683068584.628099-3322568-261537723315094/AnsiballZ_elb_target_group.py", line 99, in _ansiballz_main
    invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)
  File "/home/myuser/.ansible/tmp/ansible-tmp-1683068584.628099-3322568-261537723315094/AnsiballZ_elb_target_group.py", line 47, in invoke_module
    runpy.run_module(mod_name='ansible_collections.community.aws.plugins.modules.elb_target_group', init_globals=dict(_module_fqn='ansible_collections.community.aws.plugins.modules.elb_target_group', _modlib_path=modlib_path),
  File "/usr/lib64/python3.9/runpy.py", line 225, in run_module
    return _run_module_code(code, init_globals, run_name, mod_spec)
  File "/usr/lib64/python3.9/runpy.py", line 97, in _run_module_code
    _run_code(code, mod_globals, init_globals,
  File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/tmp/ansible_community.aws.elb_target_group_payload_y5gefe7e/ansible_community.aws.elb_target_group_payload.zip/ansible_collections/community/aws/plugins/modules/elb_target_group.py", line 989, in <module>
  File "/tmp/ansible_community.aws.elb_target_group_payload_y5gefe7e/ansible_community.aws.elb_target_group_payload.zip/ansible_collections/community/aws/plugins/modules/elb_target_group.py", line 983, in main
  File "/tmp/ansible_community.aws.elb_target_group_payload_y5gefe7e/ansible_community.aws.elb_target_group_payload.zip/ansible_collections/community/aws/plugins/modules/elb_target_group.py", line 685, in create_or_update_target_group
KeyError: 'ProtocolVersion'
fatal: [localhost]: FAILED! => {
    "attempts": 10,
    "changed": false,
    "module_stderr": "Traceback (most recent call last):\n  File \"/home/myuser/.ansible/tmp/ansible-tmp-1683068584.628099-3322568-261537723315094/AnsiballZ_elb_target_group.py\", line 107, in <module>\n    _ansiballz_main()\n  File \"/home/myuser/.ansible/tmp/ansible-tmp-1683068584.628099-3322568-261537723315094/AnsiballZ_elb_target_group.py\", line 99, in _ansiballz_main\n    invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)\n  File \"/home/myuser/.ansible/tmp/ansible-tmp-1683068584.628099-3322568-261537723315094/AnsiballZ_elb_target_group.py\", line 47, in invoke_module\n    runpy.run_module(mod_name='ansible_collections.community.aws.plugins.modules.elb_target_group', init_globals=dict(_module_fqn='ansible_collections.community.aws.plugins.modules.elb_target_group', _modlib_path=modlib_path),\n  File \"/usr/lib64/python3.9/runpy.py\", line 225, in run_module\n    return _run_module_code(code, init_globals, run_name, mod_spec)\n  File \"/usr/lib64/python3.9/runpy.py\", line 97, in _run_module_code\n    _run_code(code, mod_globals, init_globals,\n  File \"/usr/lib64/python3.9/runpy.py\", line 87, in _run_code\n    exec(code, run_globals)\n  File \"/tmp/ansible_community.aws.elb_target_group_payload_y5gefe7e/ansible_community.aws.elb_target_group_payload.zip/ansible_collections/community/aws/plugins/modules/elb_target_group.py\", line 989, in <module>\n  File \"/tmp/ansible_community.aws.elb_target_group_payload_y5gefe7e/ansible_community.aws.elb_target_group_payload.zip/ansible_collections/community/aws/plugins/modules/elb_target_group.py\", line 983, in main\n  File \"/tmp/ansible_community.aws.elb_target_group_payload_y5gefe7e/ansible_community.aws.elb_target_group_payload.zip/ansible_collections/community/aws/plugins/modules/elb_target_group.py\", line 685, in create_or_update_target_group\nKeyError: 'ProtocolVersion'\n",
    "module_stdout": "",
    "msg": "MODULE FAILURE\nSee stdout/stderr for the exact error",
    "rc": 1
}
  • ProtocolVersion is not returned a valid attribute from the resource using TCP Protocol
$ aws elbv2  describe-target-groups --target-group-arns arn:aws:elasticloadbalancing:us-east-1:[redacted:AWS_ACCOUNT_ID]:targetgroup/okipr3601-lfmwl-aext/513dc1925ba2791f
{
    "TargetGroups": [
        {
            "TargetGroupArn": "arn:aws:elasticloadbalancing:us-east-1:[redacted:AWS_ACCOUNT_ID]:targetgroup/okipr3601-lfmwl-aext/513dc1925ba2791f",
            "TargetGroupName": "okipr3601-lfmwl-aext",
            "Protocol": "TCP",
            "Port": 6443,
            "VpcId": "vpc-0eef74291b5613c03",
            "HealthCheckProtocol": "HTTPS",
            "HealthCheckPort": "6443",
            "HealthCheckEnabled": true,
            "HealthCheckIntervalSeconds": 10,
            "HealthCheckTimeoutSeconds": 10,
            "HealthyThresholdCount": 2,
            "UnhealthyThresholdCount": 2,
            "HealthCheckPath": "/readyz",
            "Matcher": {
                "HttpCode": "200-399"
            },
            "LoadBalancerArns": [
                "arn:aws:elasticloadbalancing:us-east-1:[redacted:AWS_ACCOUNT_ID]:loadbalancer/net/okipr3601-lfmwl-ext/93bc672ccecb2bd7"
            ],
            "TargetType": "ip",
            "IpAddressType": "ipv4"
        }
    ]
}

  • Versions
$ ansible-galaxy collection list
Collection           Version     
-------------------- ------------
amazon.aws           5.4.0       
community.aws        5.4.0       

$ python -V
Python 3.9.13

 ansible --version
ansible [core 2.14.5]
  config file = /home/ansible.cfg
  configured module search path = ['/home/myuser/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /home/myuser/.venvs/vpy3.9/lib64/python3.9/site-packages/ansible
  ansible collection location = /home/collections
  executable location = /home/myuser/.venvs/vpy3.9/bin/ansible
  python version = 3.9.13 (main, Nov  9 2022, 13:16:24) [GCC 8.5.0 20210514 (Red Hat 8.5.0-15)] (/home/myuser/.venvs/vpy3.9/bin/python3.9)
  jinja version = 3.1.2
  libyaml = True

 $ pip freeze |grep boto
boto3==1.26.125
botocore==1.29.125

*ProtocolVersion is not Required, according to the API definition: https://docs.aws.amazon.com/elasticloadbalancing/latest/APIReference/API_TargetGroup.html

ISSUE TYPE
  • Bugfix Pull Request
COMPONENT NAME
  • elb_target_group
ADDITIONAL INFORMATION

SUMMARY

Fix KeyError on module elb_target_group.

Given the task:
- name: Target | AWS | Create {{ tg.name }}
  community.aws.elb_target_group:
    state: present
    region: "{{ tg.region | d(omit) }}"
    name: "{{ tg.name }}"
    vpc_id: "{{ vpc_id }}"
    tags: "{{ tg.tags | d(omit) }}"
    purge_tags: "{{ tg.purge_tags | d('no') }}"
    protocol: "{{ tg.protocol }}"
    port: "{{ tg.port }}"
    # Health Check
    health_check_protocol: "{{ tg.health_check_protocol }}"
    health_check_path: "{{ tg.health_check_path | d(omit) }}"
    health_check_port: "{{ tg.health_check_port | d(omit) }}"
    successful_response_codes: "{{ tg.successful_response_codes | d(omit) }}"
    health_check_interval: "{{ tg.health_check_interval | d(omit) }}"
    health_check_timeout: "{{ tg.health_check_timeout | d(omit) }}"
    healthy_threshold_count: "{{ tg.healthy_threshold_count | d(omit) }}"
    unhealthy_threshold_count: "{{ tg.unhealthy_threshold_count | d(omit) }}"
    # Targets
    target_type: "{{ tg.target_type | d(omit) }}"
    modify_targets: "{{ tg.modify_targets | d(omit) }}"
    tagets: "{{ tg.tagets | d(omit) }}"
    # Config
    stickiness_enabled: "{{ tg.stickiness_enabled | d(omit) }}"
    stickiness_app_cookie_duration: "{{ tg.stickiness_app_cookie_duration | d(omit) }}"
    stickiness_app_cookie_name: "{{ tg.stickiness_app_cookie_name | d(omit) }}"
    stickiness_lb_cookie_duration: "{{ tg.stickiness_lb_cookie_duration | d(omit) }}"
    stickiness_type: "{{ tg.stickiness_type | d(omit) }}"
    proxy_protocol_v2_enabled: "{{ tg.proxy_protocol_v2_enabled | d(omit) }}"
    preserve_client_ip_enabled: "{{ tg.preserve_client_ip_enabled | d(omit) }}"
    deregistration_delay_timeout: "{{ tg.deregistration_delay_timeout | d(omit) }}"
  register: tg_out
  until: "tg_out is not failed"
  retries: 10
  delay: 5

The following error when running against existing resources (after creation/idempotency) that do not use ProtocolVersion (optional*, E.g: TCP):

The full traceback is:
Traceback (most recent call last):
  File "/home/myuser/.ansible/tmp/ansible-tmp-1683068584.628099-3322568-261537723315094/AnsiballZ_elb_target_group.py", line 107, in <module>
    _ansiballz_main()
  File "/home/myuser/.ansible/tmp/ansible-tmp-1683068584.628099-3322568-261537723315094/AnsiballZ_elb_target_group.py", line 99, in _ansiballz_main
    invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)
  File "/home/myuser/.ansible/tmp/ansible-tmp-1683068584.628099-3322568-261537723315094/AnsiballZ_elb_target_group.py", line 47, in invoke_module
    runpy.run_module(mod_name='ansible_collections.community.aws.plugins.modules.elb_target_group', init_globals=dict(_module_fqn='ansible_collections.community.aws.plugins.modules.elb_target_group', _modlib_path=modlib_path),
  File "/usr/lib64/python3.9/runpy.py", line 225, in run_module
    return _run_module_code(code, init_globals, run_name, mod_spec)
  File "/usr/lib64/python3.9/runpy.py", line 97, in _run_module_code
    _run_code(code, mod_globals, init_globals,
  File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/tmp/ansible_community.aws.elb_target_group_payload_y5gefe7e/ansible_community.aws.elb_target_group_payload.zip/ansible_collections/community/aws/plugins/modules/elb_target_group.py", line 989, in <module>
  File "/tmp/ansible_community.aws.elb_target_group_payload_y5gefe7e/ansible_community.aws.elb_target_group_payload.zip/ansible_collections/community/aws/plugins/modules/elb_target_group.py", line 983, in main
  File "/tmp/ansible_community.aws.elb_target_group_payload_y5gefe7e/ansible_community.aws.elb_target_group_payload.zip/ansible_collections/community/aws/plugins/modules/elb_target_group.py", line 685, in create_or_update_target_group
KeyError: 'ProtocolVersion'
fatal: [localhost]: FAILED! => {
    "attempts": 10,
    "changed": false,
    "module_stderr": "Traceback (most recent call last):\n  File \"/home/myuser/.ansible/tmp/ansible-tmp-1683068584.628099-3322568-261537723315094/AnsiballZ_elb_target_group.py\", line 107, in <module>\n    _ansiballz_main()\n  File \"/home/myuser/.ansible/tmp/ansible-tmp-1683068584.628099-3322568-261537723315094/AnsiballZ_elb_target_group.py\", line 99, in _ansiballz_main\n    invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)\n  File \"/home/myuser/.ansible/tmp/ansible-tmp-1683068584.628099-3322568-261537723315094/AnsiballZ_elb_target_group.py\", line 47, in invoke_module\n    runpy.run_module(mod_name='ansible_collections.community.aws.plugins.modules.elb_target_group', init_globals=dict(_module_fqn='ansible_collections.community.aws.plugins.modules.elb_target_group', _modlib_path=modlib_path),\n  File \"/usr/lib64/python3.9/runpy.py\", line 225, in run_module\n    return _run_module_code(code, init_globals, run_name, mod_spec)\n  File \"/usr/lib64/python3.9/runpy.py\", line 97, in _run_module_code\n    _run_code(code, mod_globals, init_globals,\n  File \"/usr/lib64/python3.9/runpy.py\", line 87, in _run_code\n    exec(code, run_globals)\n  File \"/tmp/ansible_community.aws.elb_target_group_payload_y5gefe7e/ansible_community.aws.elb_target_group_payload.zip/ansible_collections/community/aws/plugins/modules/elb_target_group.py\", line 989, in <module>\n  File \"/tmp/ansible_community.aws.elb_target_group_payload_y5gefe7e/ansible_community.aws.elb_target_group_payload.zip/ansible_collections/community/aws/plugins/modules/elb_target_group.py\", line 983, in main\n  File \"/tmp/ansible_community.aws.elb_target_group_payload_y5gefe7e/ansible_community.aws.elb_target_group_payload.zip/ansible_collections/community/aws/plugins/modules/elb_target_group.py\", line 685, in create_or_update_target_group\nKeyError: 'ProtocolVersion'\n",
    "module_stdout": "",
    "msg": "MODULE FAILURE\nSee stdout/stderr for the exact error",
    "rc": 1
}

ProtocolVersion is not returned a valid attribute from the resource using TCP Protocol

$ aws elbv2  describe-target-groups --target-group-arns arn:aws:elasticloadbalancing:us-east-1:[redacted:AWS_ACCOUNT_ID]:targetgroup/okipr3601-lfmwl-aext/513dc1925ba2791f
{
    "TargetGroups": [
        {
            "TargetGroupArn": "arn:aws:elasticloadbalancing:us-east-1:[redacted:AWS_ACCOUNT_ID]:targetgroup/okipr3601-lfmwl-aext/513dc1925ba2791f",
            "TargetGroupName": "okipr3601-lfmwl-aext",
            "Protocol": "TCP",
            "Port": 6443,
            "VpcId": "vpc-0eef74291b5613c03",
            "HealthCheckProtocol": "HTTPS",
            "HealthCheckPort": "6443",
            "HealthCheckEnabled": true,
            "HealthCheckIntervalSeconds": 10,
            "HealthCheckTimeoutSeconds": 10,
            "HealthyThresholdCount": 2,
            "UnhealthyThresholdCount": 2,
            "HealthCheckPath": "/readyz",
            "Matcher": {
                "HttpCode": "200-399"
            },
            "LoadBalancerArns": [
                "arn:aws:elasticloadbalancing:us-east-1:[redacted:AWS_ACCOUNT_ID]:loadbalancer/net/okipr3601-lfmwl-ext/93bc672ccecb2bd7"
            ],
            "TargetType": "ip",
            "IpAddressType": "ipv4"
        }
    ]
}

Versions

$ ansible-galaxy collection list
Collection           Version
-------------------- ------------
amazon.aws           5.4.0
community.aws        5.4.0

$ python -V
Python 3.9.13

 ansible --version
ansible [core 2.14.5]
  config file = /home/ansible.cfg
  configured module search path = ['/home/myuser/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /home/myuser/.venvs/vpy3.9/lib64/python3.9/site-packages/ansible
  ansible collection location = /home/collections
  executable location = /home/myuser/.venvs/vpy3.9/bin/ansible
  python version = 3.9.13 (main, Nov  9 2022, 13:16:24) [GCC 8.5.0 20210514 (Red Hat 8.5.0-15)] (/home/myuser/.venvs/vpy3.9/bin/python3.9)
  jinja version = 3.1.2
  libyaml = True

 $ pip freeze |grep boto
boto3==1.26.125
botocore==1.29.125

*ProtocolVersion is not Required, according to the API definition: https://docs.aws.amazon.com/elasticloadbalancing/latest/APIReference/API_TargetGroup.html
ISSUE TYPE

Bugfix Pull Request

COMPONENT NAME

elb_target_group

ADDITIONAL INFORMATION

Reviewed-by: Mark Chappell
Reviewed-by: Alina Buzachis
@tremble tremble marked this pull request as ready for review May 5, 2023 12:33
@tremble tremble added the mergeit Merge the PR (SoftwareFactory) label May 5, 2023
@softwarefactory-project-zuul
Copy link
Contributor

Build succeeded (gate pipeline).
https://ansible.softwarefactory-project.io/zuul/buildset/8a08d33ec7b84c85a3d6608d8818b07d

✔️ ansible-galaxy-importer SUCCESS in 3m 40s
✔️ build-ansible-collection SUCCESS in 13m 23s
ansible-test-sanity-docker-devel FAILURE in 14m 31s (non-voting)
✔️ ansible-test-sanity-docker-milestone SUCCESS in 12m 46s (non-voting)
✔️ ansible-test-sanity-docker-stable-2.12 SUCCESS in 12m 21s
✔️ ansible-test-sanity-docker-stable-2.13 SUCCESS in 9m 24s
✔️ ansible-test-sanity-docker-stable-2.14 SUCCESS in 8m 56s
✔️ ansible-test-units-amazon-aws-python36 SUCCESS in 7m 57s
✔️ ansible-test-units-amazon-aws-python38 SUCCESS in 7m 46s
✔️ ansible-test-units-amazon-aws-python39 SUCCESS in 8m 31s
✔️ ansible-test-units-amazon-aws-python310 SUCCESS in 8m 57s
✔️ ansible-test-changelog SUCCESS in 4m 47s
✔️ ansible-test-splitter SUCCESS in 5m 05s
✔️ integration-community.aws-1 SUCCESS in 26m 18s
Skipped 21 jobs

@softwarefactory-project-zuul softwarefactory-project-zuul bot merged commit c975a80 into ansible-collections:stable-5 May 5, 2023
@tremble tremble deleted the backport/stable-5/1800 branch September 29, 2023 08:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
mergeit Merge the PR (SoftwareFactory)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants