Wildcards / globbing not working for --exclude anymore (since 6.12)

[frittentheke]

Summary

We use the globbing pattern ansible-lint --exclude '**/secrets.yml to exclude binary files (which are binary due to applied git-crypt encryption). This worked fine until creator-ee:v0.13.0 (ansible-lint 6.11) and broke with creator-ee:v0.14.0 (ansible-lint 6.12.2)

Issue Type

• Bug Report

OS / ENVIRONMENT

ansible-lint --version

root@creator-ee:v0.16.0: /builds (main)
# ansible-lint --version
ansible-lint 6.14.6 using ansible 2.14.4

We are using the quay.io/ansible/creator-ee:v0.16.0 container within a GitlabCI job

STEPS TO REPRODUCE

# Create a binary file which causes ansible-lint to fail
dd if=/dev/urandom of=secrets.yml bs=1 count=1000

2. Add that file to your group_vars

3. Run ansible-lint

4. You see a stack trace with a UnicodeDecodeError exception:

Traceback (most recent call last):                                                        
  File "/usr/local/bin/ansible-lint", line 8, in <module>                                 
    sys.exit(_run_cli_entrypoint())                                                       
             ^^^^^^^^^^^^^^^^^^^^^                                                        
  File "/usr/local/lib/python3.11/site-packages/ansiblelint/__main__.py", line 354, in _run_cli_entrypoint                                                                           
    sys.exit(main(sys.argv))                                                              
             ^^^^^^^^^^^^^^                                                               
  File "/usr/local/lib/python3.11/site-packages/ansiblelint/__main__.py", line 244, in main                                                                                          
    result = _get_matches(rules, options)                                                 
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^                                                 
  File "/usr/local/lib/python3.11/site-packages/ansiblelint/runner.py", line 216, in _get_matches                                                                                    
    lintables = ansiblelint.utils.get_lintables(opts=options, args=options.lintables)                                                                                                
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^                                                                                                
  File "/usr/local/lib/python3.11/site-packages/ansiblelint/utils.py", line 965, in get_lintables                                                                                    
    lintables.append(Lintable(path))                                                      
                     ^^^^^^^^^^^^^^                                                       
  File "/usr/local/lib/python3.11/site-packages/ansiblelint/file_utils.py", line 247, in __init__                                                                                    
    self.data  # pylint: disable=pointless-statement                                      
    ^^^^^^^^^                                                                             
  File "/usr/local/lib/python3.11/site-packages/ansiblelint/file_utils.py", line 375, in data                                                                                        
    self._data = parse_yaml_linenumbers(self)                                             
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^                                             
  File "/usr/local/lib/python3.11/site-packages/ansiblelint/utils.py", line 842, in parse_yaml_linenumbers                                                                           
    loader = AnsibleLoader(lintable.content, **kwargs)                                    
                           ^^^^^^^^^^^^^^^^                                               
  File "/usr/local/lib/python3.11/site-packages/ansiblelint/file_utils.py", line 295, in content                                                                                     
    self._populate_content_cache_from_disk()                                              
  File "/usr/local/lib/python3.11/site-packages/ansiblelint/file_utils.py", line 282, in _populate_content_cache_from_disk                                                           
    self._content = self.path.expanduser().resolve().read_text(encoding="utf-8")                                                                                                     
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^                                                                                                     
  File "/usr/lib64/python3.11/pathlib.py", line 1059, in read_text                                                                                                                   
    return f.read()                                                                       
           ^^^^^^^^                                                                       
  File "<frozen codecs>", line 322, in decode                                             
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xc5 in position 0: invalid continuation byte

5. Test with exclude glob ansible-lint --exclude '**/secrets.yml with in versions 6.11 and with 6.12.

6. If you give ansible-lint the full path e.g. ansible-lint --exclude 'group_vars/all/secrets.yml it works even with 6.12

Desired Behavior

We expect the selected (wildcard/globbed) files to be excluded as they contain binary.
This is now the debug log looks like with creator-ee:v0.13.0:

[...]                              
DEBUG    Excluded environments/production/group_vars/haproxy/secrets.yml (yaml)                                        
DEBUG    Excluded environments/integration/group_vars/monitoring/secrets.yml (yaml
[...]

or when using a non globbed path with creator-ee:v0.16.0:

[...]
DEBUG    Ignored environments/integration/group_vars/monitoring/secrets.yml due to: File /builds/environments/integration/group_vars/monitoring matched exclusion entry: environments/integration/group_vars/monitoring/secrets.yml
[...]

Actual Behavior

See steps to reproduce.

[frittentheke]

@ssbarnea @ajinkyau can I provide any more info on this issue? Can you reproduce this and agree this is an issue?