warning about duplicate dict key with 6.16.2

[MallocArray]

Summary

When using 6.16.2 in an Ubuntu pipeline agent or a Stream9 container, when scanning my entire repo, I get pages full of errors similar this:

WARNING  /usr/local/lib/python3.9/site-packages/ansible/parsing/yaml/constructor.py:76 AnsibleWarning While constructing a mapping from <unicode string>, line 53, column 3, found a duplicate dict key (name). Using last defined value only.
WARNING  /usr/local/lib/python3.9/site-packages/ansible/parsing/yaml/constructor.py:76 AnsibleWarning While constructing a mapping from <unicode string>, line 57, column 3, found a duplicate dict key (name). Using last defined value only.

When it finishes, it is showing 0 failures and 0 warnings, when I know there are issues in the repo. If I give it a single subfolder, it works as expected and I do not see these Warnings.

Trying to find a minimal use-case, I have a folder with some playbooks in it and then another subfolder off the root
collections/ansible_collections/ansible/windows
which contains the 1.14.0 version of this collection.

Running ansible-lint against the entire directory yields the errors, removing this windows folder does not log the errors.

I do have this folder in my exclude_paths parameters of .ansible-lint which I understand is not working properly in this version and is scanning the folder, but I would not have expected these issues when it does scan this collection folder

Issue Type

• Bug Report

OS / ENVIRONMENT

ansible-lint --version
ansible-lint 6.16.2 using ansible-core:2.14.6 ruamel-yaml:None ruamel-yaml-clib:None

• ansible installation method: pip3 in pipeline, or requirements.txt in ansible-builder for Execution Environment
• ansible-lint installation method: pip

STEPS TO REPRODUCE

Git project folder with .ansible-lint in the root excluding - collections/ansible_collections/ansible
Folder with 4 playbooks inside of it
collections folder with Windows collection at collections/ansible_collections/ansible

 ansible-lint 
WARNING  Skipped installing collection dependencies due to running in offline mode.
WARNING  /usr/local/lib/python3.9/site-packages/ansible/parsing/yaml/constructor.py:76 AnsibleWarning While constructing a mapping from <unicode string>, line 87, column 5, found a duplicate dict key (members). Using last defined value only.
WARNING  /usr/local/lib/python3.9/site-packages/ansible/parsing/yaml/constructor.py:76 AnsibleWarning While constructing a mapping from <unicode string>, line 111, column 25, found a duplicate dict key (members). Using last defined value only.
WARNING  /usr/local/lib/python3.9/site-packages/ansible/parsing/yaml/constructor.py:76 AnsibleWarning While constructing a mapping from <unicode string>, line 148, column 25, found a duplicate dict key (state). Using last defined value only.

Desired Behavior

No warnings about parsing yaml files outside of this project folder, and only valid issues from the playbook folder are shown
Possible security bugs should be reported via email to [email protected]

Actual Behavior

In this minimal case where I've deleted nearly ever folder I see the duplicate dict key messages but it still shows valid issues in the remaining playbooks. When running against a full repository with multiple folder, playbooks, and collections, I get more of the duplicate dict key messages, but no issues are shown, as if all errors are addressed which is not accurate.

Please give some details of what is happening.
Include a minimum complete verifiable example with:

• minimized playbook to reproduce the error
• the output of running ansible-lint including the command line used
• if you're getting a stack trace, also the output of
  ansible-playbook --syntax-check playbook

DEBUG    data set to None for collections/ansible_collections/ansible/windows/tests/integration/targets/win_certificate_store/aliases due to being of unknown kind.
DEBUG    data set to None for collections/ansible_collections/ansible/windows/plugins/plugin_utils/_quote.py due to being of unknown kind.
DEBUG    data set to None for collections/ansible_collections/ansible/windows/plugins/modules/async_status.ps1 due to being of unknown kind.
INFO     Executing syntax check on playbook AWX/awx_capture_fatal_failure.yml (0.81s)
INFO     Executing syntax check on playbook AWX/awx_export.yml (0.82s)
INFO     Executing syntax check on playbook AWX/awx_configuration.yml (0.86s)
WARNING  /usr/local/lib/python3.9/site-packages/ansible/parsing/yaml/constructor.py:76 AnsibleWarning While constructing a mapping from <unicode string>, line 87, column 5, found a duplicate dict key (members). Using last defined value only.
WARNING  /usr/local/lib/python3.9/site-packages/ansible/parsing/yaml/constructor.py:76 AnsibleWarning While constructing a mapping from <unicode string>, line 111, column 25, found a duplicate dict key (members). Using last defined value only.
WARNING  /usr/local/lib/python3.9/site-packages/ansible/parsing/yaml/constructor.py:76 AnsibleWarning While constructing a mapping from <unicode string>, line 148, column 25, found a duplicate dict key (state). Using last defined value only.

[netopsengineer]

Same here @ssbarnea

Let me know if you need additional info:

Mac OS Ventura 13.4 and Python 3.11.3

ansible-lint/lib/python3.11/site-packages/ansible/parsing/yaml/constructor.py:76 AnsibleWarning While constructing a mapping from <unicode string>, line 256, column 5, found a duplicate dict key (password). Using last defined value only.

ansible-lint --version output:

ansible-lint 6.16.2 using ansible-core:2.14.4 ruamel-yaml:0.17.26 ruamel-yaml-clib:0.2.7

I noticed that it seems to be ignoring my ignore list, back to back runs on the same project from 16.6.0 vs 16.6.2 show a significantly larger amount of files being checked given the same .ansible-lint config file. I suspect it is picking up the Collections folder which is ignored.

With 16.6.0:

Passed with production profile: 0 failure(s), 0 warning(s) on 111 files.

With 16.6.2:

Passed with production profile: 0 failure(s), 0 warning(s) on 2836 files.

[MallocArray]

@netopsengineer the issue with excluded directories being scanned is known and a fix is on the way
#3477
Not sure if the duplicate key thing is related or not.

[netopsengineer]

Thanks for the heads up on the excluded directories @MallocArray will keep an eye on the issue here on the duplicate keys, for me atleast my project previously passed with production profile, and still does, just with those errors spewing, so I'm not sure if its skewing the results or not.