Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

lambda.py invokes update_function_configuration() and afterwards update_function_code() without checking Configuration.State and Configuration.LastUpdateStatus fields in between #830

Closed
1 task done
rradecki-migo opened this issue Dec 9, 2021 · 0 comments · Fixed by #857

Comments

@rradecki-migo
Copy link

rradecki-migo commented Dec 9, 2021

Summary

lambda.py invokes update_function_configuration() here and afterwards update_function_code() here without checking Configuration.State and Configuration.LastUpdateStatus fields in between. If lambda function was updated in scope of the first call an error similar to below one is created during the second call:

An exception occurred during task execution. To see the full traceback, use -vvv. The error was: botocore.errorfactory.ResourceConflictException: An error occurred (ResourceConflictException) when calling the UpdateFunctionCode operation: The operation cannot be performed at this time. An update is in progress for resource: arn:aws:lambda:MY_REGION:MY_AWS_ACCOUNT_ID:function:MY_LAMBDA_FUNC_NAME
fatal: [localhost]: FAILED! => {"boto3_version": "1.20.22", "botocore_version": "1.23.22", "changed": false, "error": {"code": "ResourceConflictException", "message": "The operation cannot be performed at this time. An update is in progress for resource: arn:aws:lambda:MY_REGION:MY_AWS_ACCOUNT_ID:function:MY_LAMBDA_FUNC_NAME"}, "message": "The operation cannot be performed at this time. An update is in progress for resource: arn:aws:lambda:MY_REGION:MY_AWS_ACCOUNT_ID:function:MY_LAMBDA_FUNC_NAME", "msg": "Trying to upload new code: An error occurred (ResourceConflictException) when calling the UpdateFunctionCode operation: The operation cannot be performed at this time. An update is in progress for resource: arn:aws:lambda:MY_REGION:MY_AWS_ACCOUNT_ID:function:MY_LAMBDA_FUNC_NAME", "response_metadata": {"http_headers": {"connection": "keep-alive", "content-length": "201", "content-type": "application/json", "date": "Thu, 09 Dec 2021 13:23:20 GMT", "x-amzn-errortype": "ResourceConflictException", "x-amzn-requestid": "abfc47a6-6c1c-42e4-b19d-853da87674bc"}, "http_status_code": 409, "request_id": "abfc47a6-6c1c-42e4-b19d-853da87674bc", "retry_attempts": 0}, "type": "User"}

Issue Type

Bug Report

Component Name

lambda

Ansible Version

$ ansible --version
ansible [core 2.12.0]
  config file = /home/my_dir/ansible.cfg
  configured module search path = ['/home/my_dir/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /home/my_dir/.venv/lib/python3.8/site-packages/ansible
  ansible collection location = /home/my_dir/.ansible/my_company/collections
  executable location = /home/my_dir/.venv/bin/ansible
  python version = 3.8.5 (v3.8.5:580fbb018f, Jul 20 2020, 12:11:27) [Clang 6.0 (clang-600.0.57)]
  jinja version = 2.11.3
  libyaml = True```


### Collection Versions

```console (paste below)
$ ansible-galaxy collection list
# /home/my_dir/.ansible/my_company/collections/ansible_collections
Collection            Version
--------------------- -------
amazon.aws            3.0.0
ansible.netcommon     1.4.1
ansible.posix         1.3.0
community.aws         2.1.0
community.general     4.0.0
community.hashi_vault 1.3.2
community.kubernetes  1.2.1
community.sops        1.1.0
google.cloud          1.0.1
kubernetes.core       2.2.1

# /home/my_dir/.venv/lib/python3.8/site-packages/ansible_collections
Collection                    Version
----------------------------- -------
amazon.aws                    2.1.0
ansible.netcommon             2.4.0
ansible.posix                 1.3.0
ansible.utils                 2.4.2
ansible.windows               1.8.0
arista.eos                    3.1.0
awx.awx                       19.4.0
azure.azcollection            1.10.0
check_point.mgmt              2.1.1
chocolatey.chocolatey         1.1.0
cisco.aci                     2.1.0
cisco.asa                     2.1.0
cisco.intersight              1.0.17
cisco.ios                     2.5.0
cisco.iosxr                   2.5.0
cisco.ise                     1.2.1
cisco.meraki                  2.5.0
cisco.mso                     1.2.0
cisco.nso                     1.0.3
cisco.nxos                    2.7.1
cisco.ucs                     1.6.0
cloud.common                  2.1.0
cloudscale_ch.cloud           2.2.0
community.aws                 2.1.0
community.azure               1.1.0
community.ciscosmb            1.0.4
community.crypto              2.0.1
community.digitalocean        1.12.0
community.dns                 2.0.3
community.docker              2.0.1
community.fortios             1.0.0
community.general             4.0.2
community.google              1.0.0
community.grafana             1.2.3
community.hashi_vault         2.0.0
community.hrobot              1.2.1
community.kubernetes          2.0.1
community.kubevirt            1.0.0
community.libvirt             1.0.2
community.mongodb             1.3.2
community.mysql               2.3.1
community.network             3.0.0                                                                                                                                                                                                                                                                                 [0/1887]
community.okd                 2.1.0
community.postgresql          1.5.0
community.proxysql            1.3.0
community.rabbitmq            1.1.0
community.routeros            2.0.0
community.skydive             1.0.0
community.sops                1.2.0
community.vmware              1.16.0
community.windows             1.8.0
community.zabbix              1.5.0
containers.podman             1.8.2
cyberark.conjur               1.1.0
cyberark.pas                  1.0.13
dellemc.enterprise_sonic      1.1.0
dellemc.openmanage            4.2.0
dellemc.os10                  1.1.1
dellemc.os6                   1.0.7
dellemc.os9                   1.0.4
f5networks.f5_modules         1.12.0
fortinet.fortimanager         2.1.4
fortinet.fortios              2.1.3
frr.frr                       1.0.3
gluster.gluster               1.0.2
google.cloud                  1.0.2
hetzner.hcloud                1.6.0
hpe.nimble                    1.1.3
ibm.qradar                    1.0.3
infinidat.infinibox           1.3.0
infoblox.nios_modules         1.1.2
inspur.sm                     1.3.0
junipernetworks.junos         2.6.0
kubernetes.core               2.2.1
mellanox.onyx                 1.0.0
netapp.aws                    21.7.0
netapp.azure                  21.10.0
netapp.cloudmanager           21.12.0
netapp.elementsw              21.7.0
netapp.ontap                  21.13.1
netapp.storagegrid            21.7.0
netapp.um_info                21.8.0
netapp_eseries.santricity     1.2.13
netbox.netbox                 3.3.0
ngine_io.cloudstack           2.2.2
ngine_io.exoscale             1.0.0
ngine_io.vultr                1.1.0
openstack.cloud               1.5.3
openvswitch.openvswitch       2.0.2
ovirt.ovirt                   1.6.5
purestorage.flasharray        1.11.0
purestorage.flashblade        1.8.1
sensu.sensu_go                1.12.0
servicenow.servicenow         1.0.6
splunk.es                     1.0.2
t_systems_mms.icinga_director 1.24.0
theforeman.foreman            2.2.0
vyos.vyos                     2.6.0
wti.remote                    1.0.3

AWS SDK versions

$ pip show boto boto3 botocore
Name: boto
Version: 2.49.0
Summary: Amazon Web Services Library
Home-page: https://github.com/boto/boto/
Author: Mitch Garnaat
Author-email: [email protected]
License: MIT
Location:/home/my_dir/.venv/lib/python3.8/site-packages
Requires:
Required-by:
---
Name: boto3
Version: 1.20.22
Summary: The AWS SDK for Python
Home-page: https://github.com/boto/boto3
Author: Amazon Web Services
Author-email:
License: Apache License 2.0
Location: /home/my_dir/.venv/lib/python3.8/site-packages
Requires: botocore, jmespath, s3transfer
Required-by:
---
Name: botocore
Version: 1.23.22
Summary: Low-level, data-driven core of boto 3.
Home-page: https://github.com/boto/botocore
Author: Amazon Web Services
Author-email:
License: Apache License 2.0
Location: /home/my_dir/.venv/lib/python3.8/site-packages
Requires: jmespath, python-dateutil, urllib3
Required-by: awscli, boto3, s3transfer

Configuration

$ ansible-config dump --only-changed
COLLECTIONS_PATHS(/home/my_dir/ansible.cfg) = ['/home/my_dir/.ansible/my_company/collections']
DEFAULT_HOST_LIST(/home/my_dir/ansible.cfg) = ['/dev/null']
DEFAULT_VAULT_PASSWORD_FILE(env: ANSIBLE_VAULT_PASSWORD_FILE) = /home/my_dir/.vault-ansible
LOCALHOST_WARNING(/home/my_dir/ansible.cfg) = False
RETRY_FILES_ENABLED(/home/my_dir/ansible.cfg) = False

OS / Environment

macOS Big Sur 11.6

Steps to Reproduce

  1. Change memory assignment in aws lambda gui to value different than the one set by Ansible
  2. Wait until aws lambda get-function --function-name YOUR_FUNCTION --query 'Configuration.[State, LastUpdateStatus]' will return
[
    "Active",
    "Successful"
]
  1. Invoke below ansible snippet
- name: "Deploy lambda function"
  lambda:
    dead_letter_arn: "{{ 'arn:aws:sqs:{}:{}:{}-{}-lambda-deadletter'.format(aws_region, account, x, lambda_name) }}"
    description: "{{ lambda_description }}"
    environment_variables: "{{ lambda_env }}"
    handler: "{{ lambda_handler }}"
    memory_size: "{{ lambda_memory }}"
    name: "{{ lambda_full_name }}"
    role: "arn:aws:iam::{{ account }}:role/service-role/{{ lambda_iam_name }}"
    runtime: "{{ lambda_runtime }}"
    s3_bucket: "{{ lambda_s3_bucket }}"
    s3_key: "{{ lambda_s3_url }}
    timeout: "{{ lambda_timeout }}"
    vpc_security_group_ids: "{{ group_id }}"
    vpc_subnet_ids: "{{ subnets }}"
    region: "{{ aws_region }}"
    aws_access_key: "{{ access_key }}"
    aws_secret_key: "{{ secret_key }}"
    security_token: "{{ session_token }}"

Expected Results

  1. Get the below error due to lambda function being still updated after update_function_configuration() call:
An exception occurred during task execution. To see the full traceback, use -vvv. The error was: botocore.errorfactory.ResourceConflictException: An error occurred (ResourceConflictException) when calling the UpdateFunctionCode operation: The operation cannot be performed at this time. An update is in progress for resource: arn:aws:lambda:MY_REGION:MY_AWS_ACCOUNT_ID:function:MY_LAMBDA_FUNC_NAME
fatal: [localhost]: FAILED! => {"boto3_version": "1.20.22", "botocore_version": "1.23.22", "changed": false, "error": {"code": "ResourceConflictException", "message": "The operation cannot be performed at this time. An update is in progress for resource: arn:aws:lambda:MY_REGION:MY_AWS_ACCOUNT_ID:function:MY_LAMBDA_FUNC_NAME"}, "message": "The operation cannot be performed at this time. An update is in progress for resource: arn:aws:lambda:MY_REGION:MY_AWS_ACCOUNT_ID:function:MY_LAMBDA_FUNC_NAME", "msg": "Trying to upload new code: An error occurred (ResourceConflictException) when calling the UpdateFunctionCode operation: The operation cannot be performed at this time. An update is in progress for resource: arn:aws:lambda:MY_REGION:MY_AWS_ACCOUNT_ID:function:MY_LAMBDA_FUNC_NAME", "response_metadata": {"http_headers": {"connection": "keep-alive", "content-length": "201", "content-type": "application/json", "date": "Thu, 09 Dec 2021 13:23:20 GMT", "x-amzn-errortype": "ResourceConflictException", "x-amzn-requestid": "abfc47a6-6c1c-42e4-b19d-853da87674bc"}, "http_status_code": 409, "request_id": "abfc47a6-6c1c-42e4-b19d-853da87674bc", "retry_attempts": 0}, "type": "User"}

Actual Results

Code of Conduct

  • I agree to follow the Ansible Code of Conduct
softwarefactory-project-zuul bot pushed a commit that referenced this issue Jan 30, 2022
Lambda - Wait before updating

SUMMARY

Updated lambda module to wait for State = Active & LastUpdateStatus = Successful based on https://aws.amazon.com/blogs/compute/coming-soon-expansion-of-aws-lambda-states-to-all-functions/

Fixes #830
ISSUE TYPE


Bugfix Pull Request

COMPONENT NAME

module: lambda
ADDITIONAL INFORMATION

Reviewed-by: Markus Bergholz <[email protected]>
Reviewed-by: Alina Buzachis <None>
patchback bot pushed a commit that referenced this issue Jan 30, 2022
Lambda - Wait before updating

SUMMARY

Updated lambda module to wait for State = Active & LastUpdateStatus = Successful based on https://aws.amazon.com/blogs/compute/coming-soon-expansion-of-aws-lambda-states-to-all-functions/

Fixes #830
ISSUE TYPE

Bugfix Pull Request

COMPONENT NAME

module: lambda
ADDITIONAL INFORMATION

Reviewed-by: Markus Bergholz <[email protected]>
Reviewed-by: Alina Buzachis <None>
(cherry picked from commit ab4bda2)
patchback bot pushed a commit that referenced this issue Jan 30, 2022
Lambda - Wait before updating

SUMMARY

Updated lambda module to wait for State = Active & LastUpdateStatus = Successful based on https://aws.amazon.com/blogs/compute/coming-soon-expansion-of-aws-lambda-states-to-all-functions/

Fixes #830
ISSUE TYPE

Bugfix Pull Request

COMPONENT NAME

module: lambda
ADDITIONAL INFORMATION

Reviewed-by: Markus Bergholz <[email protected]>
Reviewed-by: Alina Buzachis <None>
(cherry picked from commit ab4bda2)
tremble pushed a commit that referenced this issue Jan 30, 2022
Lambda - Wait before updating

SUMMARY

Updated lambda module to wait for State = Active & LastUpdateStatus = Successful based on https://aws.amazon.com/blogs/compute/coming-soon-expansion-of-aws-lambda-states-to-all-functions/

Fixes #830
ISSUE TYPE

Bugfix Pull Request

COMPONENT NAME

module: lambda
ADDITIONAL INFORMATION

Reviewed-by: Markus Bergholz <[email protected]>
Reviewed-by: Alina Buzachis <None>
(cherry picked from commit ab4bda2)

Co-authored-by: Andreas Jonsson <[email protected]>
softwarefactory-project-zuul bot pushed a commit that referenced this issue Jan 31, 2022
[PR #857/ab4bda24 backport][stable-2] Lambda - Wait before updating

This is a backport of PR #857 as merged into main (ab4bda2).
SUMMARY

Updated lambda module to wait for State = Active & LastUpdateStatus = Successful based on https://aws.amazon.com/blogs/compute/coming-soon-expansion-of-aws-lambda-states-to-all-functions/

Fixes #830
ISSUE TYPE


Bugfix Pull Request

COMPONENT NAME

module: lambda
ADDITIONAL INFORMATION
abikouo pushed a commit to abikouo/community.aws that referenced this issue Oct 24, 2023
…lections#830)

ec2_key - remove unused wait and wait_timeout parameters

SUMMARY
Remove unused wait and wait_timeout parameters from the ec2_key module.  These have had no effect since Ansible 2.5 and were deprecated in release 1.0.0.
ISSUE TYPE

Feature Pull Request

COMPONENT NAME
ec2_key
ADDITIONAL INFORMATION
See also: ansible/ansible#64230

Reviewed-by: Alina Buzachis <None>
abikouo pushed a commit to abikouo/community.aws that referenced this issue Oct 24, 2023
Lambda - Wait before updating

SUMMARY

Updated lambda module to wait for State = Active & LastUpdateStatus = Successful based on https://aws.amazon.com/blogs/compute/coming-soon-expansion-of-aws-lambda-states-to-all-functions/

Fixes ansible-collections#830
ISSUE TYPE

Bugfix Pull Request

COMPONENT NAME

module: lambda
ADDITIONAL INFORMATION

Reviewed-by: Markus Bergholz <[email protected]>
Reviewed-by: Alina Buzachis <None>

This commit was initially merged in https://github.com/ansible-collections/community.aws
See: ansible-collections@ab4bda2
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant