Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Wildcards / globbing not working for --exclude anymore (since 6.12) #3310

Closed
frittentheke opened this issue Apr 20, 2023 · 1 comment · Fixed by #3401
Closed

Wildcards / globbing not working for --exclude anymore (since 6.12) #3310

frittentheke opened this issue Apr 20, 2023 · 1 comment · Fixed by #3401
Assignees
Labels

Comments

@frittentheke
Copy link

frittentheke commented Apr 20, 2023

Summary

We use the globbing pattern ansible-lint --exclude '**/secrets.yml to exclude binary files (which are binary due to applied git-crypt encryption). This worked fine until creator-ee:v0.13.0 (ansible-lint 6.11) and broke with creator-ee:v0.14.0 (ansible-lint 6.12.2)

Issue Type
  • Bug Report
OS / ENVIRONMENT
ansible-lint --version
root@creator-ee:v0.16.0: /builds (main)
# ansible-lint --version
ansible-lint 6.14.6 using ansible 2.14.4

We are using the quay.io/ansible/creator-ee:v0.16.0 container within a GitlabCI job

STEPS TO REPRODUCE
# Create a binary file which causes ansible-lint to fail
dd if=/dev/urandom of=secrets.yml bs=1 count=1000
  1. Add that file to your group_vars

  2. Run ansible-lint

  3. You see a stack trace with a UnicodeDecodeError exception:

Traceback (most recent call last):                                                        
  File "/usr/local/bin/ansible-lint", line 8, in <module>                                 
    sys.exit(_run_cli_entrypoint())                                                       
             ^^^^^^^^^^^^^^^^^^^^^                                                        
  File "/usr/local/lib/python3.11/site-packages/ansiblelint/__main__.py", line 354, in _run_cli_entrypoint                                                                           
    sys.exit(main(sys.argv))                                                              
             ^^^^^^^^^^^^^^                                                               
  File "/usr/local/lib/python3.11/site-packages/ansiblelint/__main__.py", line 244, in main                                                                                          
    result = _get_matches(rules, options)                                                 
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^                                                 
  File "/usr/local/lib/python3.11/site-packages/ansiblelint/runner.py", line 216, in _get_matches                                                                                    
    lintables = ansiblelint.utils.get_lintables(opts=options, args=options.lintables)                                                                                                
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^                                                                                                
  File "/usr/local/lib/python3.11/site-packages/ansiblelint/utils.py", line 965, in get_lintables                                                                                    
    lintables.append(Lintable(path))                                                      
                     ^^^^^^^^^^^^^^                                                       
  File "/usr/local/lib/python3.11/site-packages/ansiblelint/file_utils.py", line 247, in __init__                                                                                    
    self.data  # pylint: disable=pointless-statement                                      
    ^^^^^^^^^                                                                             
  File "/usr/local/lib/python3.11/site-packages/ansiblelint/file_utils.py", line 375, in data                                                                                        
    self._data = parse_yaml_linenumbers(self)                                             
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^                                             
  File "/usr/local/lib/python3.11/site-packages/ansiblelint/utils.py", line 842, in parse_yaml_linenumbers                                                                           
    loader = AnsibleLoader(lintable.content, **kwargs)                                    
                           ^^^^^^^^^^^^^^^^                                               
  File "/usr/local/lib/python3.11/site-packages/ansiblelint/file_utils.py", line 295, in content                                                                                     
    self._populate_content_cache_from_disk()                                              
  File "/usr/local/lib/python3.11/site-packages/ansiblelint/file_utils.py", line 282, in _populate_content_cache_from_disk                                                           
    self._content = self.path.expanduser().resolve().read_text(encoding="utf-8")                                                                                                     
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^                                                                                                     
  File "/usr/lib64/python3.11/pathlib.py", line 1059, in read_text                                                                                                                   
    return f.read()                                                                       
           ^^^^^^^^                                                                       
  File "<frozen codecs>", line 322, in decode                                             
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xc5 in position 0: invalid continuation byte

  1. Test with exclude glob ansible-lint --exclude '**/secrets.yml with in versions 6.11 and with 6.12.

  2. If you give ansible-lint the full path e.g. ansible-lint --exclude 'group_vars/all/secrets.yml it works even with 6.12

Desired Behavior

We expect the selected (wildcard/globbed) files to be excluded as they contain binary.
This is now the debug log looks like with creator-ee:v0.13.0:

[...]                              
DEBUG    Excluded environments/production/group_vars/haproxy/secrets.yml (yaml)                                        
DEBUG    Excluded environments/integration/group_vars/monitoring/secrets.yml (yaml
[...]

or when using a non globbed path with creator-ee:v0.16.0:

[...]
DEBUG    Ignored environments/integration/group_vars/monitoring/secrets.yml due to: File /builds/environments/integration/group_vars/monitoring matched exclusion entry: environments/integration/group_vars/monitoring/secrets.yml
[...]
Actual Behavior

See steps to reproduce.

@frittentheke frittentheke added bug new Triage required labels Apr 20, 2023
@audgirka audgirka self-assigned this Apr 20, 2023
@audgirka audgirka removed the new Triage required label Apr 20, 2023
@frittentheke
Copy link
Author

@ssbarnea @ajinkyau can I provide any more info on this issue? Can you reproduce this and agree this is an issue?

ssbarnea added a commit that referenced this issue May 5, 2023
ssbarnea added a commit that referenced this issue May 5, 2023
ssbarnea added a commit that referenced this issue May 5, 2023
ssbarnea added a commit that referenced this issue May 5, 2023
ssbarnea added a commit that referenced this issue May 5, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants