Cannot apply deployment with error "Failed to load resource definition: [Errno 13] Permission denied: '/root/ansible/Ansible/k8s_deployments/nginx.yaml'"

[ibrahim-osama-amin]

Hello guys,

Hope you are doing great.
This is my first post here, so I hope I am doing things correctly. I checked previous issues and I did not find this one.

SUMMARY

The first task works normally. The second task gets stuck with this weird error:

An exception occurred during task execution. To see the full traceback, use -vvv. The error was: ansible_collections.kubernetes.core.plugins.module_utils.k8s.exceptions.CoreException: Failed to load resource definition: [Errno 13] Permission denied: '/root/ansible/Ansible/k8s_deployments/nginx.yaml'
fatal: [192.168.111.138]: FAILED! => {"changed": false, "msg": "Failed to load resource definition: [Errno 13] Permission denied: '/root/ansible/Ansible/k8s_deployments/nginx.yaml'"}

It is weird because I am running as root so I shouldn't face any permission issues.

The ansible target user is kube and it has all the requirements installed:

[kube@localhost ~]$ python3 -c "import openshift"
[kube@localhost ~]$ python3 -c "import yaml"
[kube@localhost ~]$ python3 -c "import jsonpatch"

I tried to work with community.kubernetes.k8s and same error
Also, I tried to play around with the names of the nginx deployment and it is the same issue.
This is my repo if you want to check and take a look at the history:
https://github.com/ibrahim-osama-amin/Ansible

ISSUE TYPE

• Bug Report

COMPONENT NAME

community.kubernetes.k8s and kubernetes.core.k8s

ANSIBLE VERSION

ansible [core 2.14.14]
  config file = /root/ansible/Ansible/ansible.cfg
  configured module search path = ['/root/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python3.9/site-packages/ansible
  ansible collection location = /root/.ansible/collections:/usr/share/ansible/collections
  executable location = /usr/bin/ansible
  python version = 3.9.18 (main, May 16 2024, 00:00:00) [GCC 11.4.1 20231218 (Red Hat 11.4.1-3)] (/usr/bin/python3)
  jinja version = 3.1.2
  libyaml = True

COLLECTION VERSION

[root@localhost Ansible]# ansible-config dump --only-changed
CONFIG_FILE() = /root/ansible/Ansible/ansible.cfg
DEFAULT_HOST_LIST(/root/ansible/Ansible/ansible.cfg) = ['/root/ansible/Ansible/hosts']
HOST_KEY_CHECKING(/root/ansible/Ansible/ansible.cfg) = False

CONFIGURATION

[root@localhost Ansible]# ansible-config dump --only-changed
CONFIG_FILE() = /root/ansible/Ansible/ansible.cfg
DEFAULT_HOST_LIST(/root/ansible/Ansible/ansible.cfg) = ['/root/ansible/Ansible/hosts']
HOST_KEY_CHECKING(/root/ansible/Ansible/ansible.cfg) = False

OS / ENVIRONMENT

Ansible host:

[root@localhost Ansible]# hostnamectl
   Static hostname: (unset)
Transient hostname: localhost
         Icon name: computer-vm
           Chassis: vm 🖴
        Machine ID: 67bcc410514841869f3234ef45756747
           Boot ID: 0532904d3785438ab6526e10c1d1e147
    Virtualization: vmware
  Operating System: Red Hat Enterprise Linux 9.4 (Plow)
       CPE OS Name: cpe:/o:redhat:enterprise_linux:9::baseos
            Kernel: Linux 5.14.0-427.13.1.el9_4.x86_64
      Architecture: x86-64
   Hardware Vendor: VMware, Inc.
    Hardware Model: VMware 

Ansible target:

   Static hostname: (unset)
Transient hostname: localhost
         Icon name: computer-vm
           Chassis: vm 🖴
        Machine ID: 1ac6a73bd74247f898bfa530e6d526c8
           Boot ID: 3aa46e091ef14b4da82473701a7daa95
    Virtualization: vmware
  Operating System: Red Hat Enterprise Linux 9.3 (Plow)
       CPE OS Name: cpe:/o:redhat:enterprise_linux:9::baseos
            Kernel: Linux 5.14.0-362.8.1.el9_3.x86_64
      Architecture: x86-64
   Hardware Vendor: VMware, Inc.
    Hardware Model: VMware Virtual Platform
  Firmware Version: 6.00

STEPS TO REPRODUCE

Create two Redhat VMs, install requirements on the source and the destination.
Run kubectl on the destination with user kube

---
- name: Deploy app in a new namespace
  hosts: local
  tasks:
    - name: Ensure my-app namespace is present 
      community.kubernetes.k8s:
        name: my-app
        api_version: v1
        kind: Namespace
        state: present  

    - name: Ensure nginx deployment with its service are running
      community.kubernetes.k8s:
        src: /root/ansible/Ansible/k8s_deployments/nginx.yaml
        state: present
        kubeconfig: /home/kube/.kube/config
        namespace: my-app

EXPECTED RESULTS

I should see a pod created and a service created on the target.

ACTUAL RESULTS

TASK [Ensure nginx deployment with its service are running] **********************************************************************************************************************************************
task path: /root/ansible/Ansible/deploy-to-k8s.yaml:12
redirecting (type: modules) community.kubernetes.k8s to kubernetes.core.k8s
redirecting (type: action) community.kubernetes.k8s to kubernetes.core.k8s_info
redirecting (type: action) community.kubernetes.k8s to kubernetes.core.k8s_info
<192.168.111.138> ESTABLISH SSH CONNECTION FOR USER: kube
<192.168.111.138> SSH: EXEC ssh -vvv -C -o ControlMaster=auto -o ControlPersist=60s -o StrictHostKeyChecking=no -o 'IdentityFile="/root/.ssh/kube"' -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o 'User="kube"' -o ConnectTimeout=10 -o 'ControlPath="/root/.ansible/cp/8e11ae4af7"' 192.168.111.138 '/bin/sh -c '"'"'echo ~kube && sleep 0'"'"''
<192.168.111.138> (0, b'/home/kube\n', b"OpenSSH_8.7p1, OpenSSL 3.0.7 1 Nov 2022\r\ndebug1: Reading configuration data /etc/ssh/ssh_config\r\ndebug3: /etc/ssh/ssh_config line 55: Including file /etc/ssh/ssh_config.d/50-redhat.conf depth 0\r\ndebug1: Reading configuration data /etc/ssh/ssh_config.d/50-redhat.conf\r\ndebug2: checking match for 'final all' host 192.168.111.138 originally 192.168.111.138\r\ndebug3: /etc/ssh/ssh_config.d/50-redhat.conf line 3: not matched 'final'\r\ndebug2: match not found\r\ndebug3: /etc/ssh/ssh_config.d/50-redhat.conf line 5: Including file /etc/crypto-policies/back-ends/openssh.config depth 1 (parse only)\r\ndebug1: Reading configuration data /etc/crypto-policies/back-ends/openssh.config\r\ndebug3: gss kex names ok: [gss-curve25519-sha256-,gss-nistp256-sha256-,gss-group14-sha256-,gss-group16-sha512-]\r\ndebug3: kex names ok: [curve25519-sha256,[email protected],ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group14-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512]\r\ndebug1: configuration requests final Match pass\r\ndebug2: resolve_canonicalize: hostname 192.168.111.138 is address\r\ndebug1: re-parsing configuration\r\ndebug1: Reading configuration data /etc/ssh/ssh_config\r\ndebug3: /etc/ssh/ssh_config line 55: Including file /etc/ssh/ssh_config.d/50-redhat.conf depth 0\r\ndebug1: Reading configuration data /etc/ssh/ssh_config.d/50-redhat.conf\r\ndebug2: checking match for 'final all' host 192.168.111.138 originally 192.168.111.138\r\ndebug3: /etc/ssh/ssh_config.d/50-redhat.conf line 3: matched 'final'\r\ndebug2: match found\r\ndebug3: /etc/ssh/ssh_config.d/50-redhat.conf line 5: Including file /etc/crypto-policies/back-ends/openssh.config depth 1\r\ndebug1: Reading configuration data /etc/crypto-policies/back-ends/openssh.config\r\ndebug3: gss kex names ok: [gss-curve25519-sha256-,gss-nistp256-sha256-,gss-group14-sha256-,gss-group16-sha512-]\r\ndebug3: kex names ok: [curve25519-sha256,[email protected],ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group14-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512]\r\ndebug3: expanded UserKnownHostsFile '~/.ssh/known_hosts' -> '/root/.ssh/known_hosts'\r\ndebug3: expanded UserKnownHostsFile '~/.ssh/known_hosts2' -> '/root/.ssh/known_hosts2'\r\ndebug1: auto-mux: Trying existing master\r\ndebug2: fd 3 setting O_NONBLOCK\r\ndebug2: mux_client_hello_exchange: master version 4\r\ndebug3: mux_client_forwards: request forwardings: 0 local, 0 remote\r\ndebug3: mux_client_request_session: entering\r\ndebug3: mux_client_request_alive: entering\r\ndebug3: mux_client_request_alive: done pid = 4378\r\ndebug3: mux_client_request_session: session request sent\r\ndebug1: mux_client_request_session: master session id: 2\r\ndebug3: mux_client_read_packet: read header failed: Broken pipe\r\ndebug2: Received exit status from master 0\r\n")
<192.168.111.138> ESTABLISH SSH CONNECTION FOR USER: kube
<192.168.111.138> SSH: EXEC ssh -vvv -C -o ControlMaster=auto -o ControlPersist=60s -o StrictHostKeyChecking=no -o 'IdentityFile="/root/.ssh/kube"' -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o 'User="kube"' -o ConnectTimeout=10 -o 'ControlPath="/root/.ansible/cp/8e11ae4af7"' 192.168.111.138 '/bin/sh -c '"'"'( umask 77 && mkdir -p "` echo /home/kube/.ansible/tmp `"&& mkdir "` echo /home/kube/.ansible/tmp/ansible-tmp-1723209062.7628653-4397-109600890829893 `" && echo ansible-tmp-1723209062.7628653-4397-109600890829893="` echo /home/kube/.ansible/tmp/ansible-tmp-1723209062.7628653-4397-109600890829893 `" ) && sleep 0'"'"''
<192.168.111.138> (0, b'ansible-tmp-1723209062.7628653-4397-109600890829893=/home/kube/.ansible/tmp/ansible-tmp-1723209062.7628653-4397-109600890829893\n', b"OpenSSH_8.7p1, OpenSSL 3.0.7 1 Nov 2022\r\ndebug1: Reading configuration data /etc/ssh/ssh_config\r\ndebug3: /etc/ssh/ssh_config line 55: Including file /etc/ssh/ssh_config.d/50-redhat.conf depth 0\r\ndebug1: Reading configuration data /etc/ssh/ssh_config.d/50-redhat.conf\r\ndebug2: checking match for 'final all' host 192.168.111.138 originally 192.168.111.138\r\ndebug3: /etc/ssh/ssh_config.d/50-redhat.conf line 3: not matched 'final'\r\ndebug2: match not found\r\ndebug3: /etc/ssh/ssh_config.d/50-redhat.conf line 5: Including file /etc/crypto-policies/back-ends/openssh.config depth 1 (parse only)\r\ndebug1: Reading configuration data /etc/crypto-policies/back-ends/openssh.config\r\ndebug3: gss kex names ok: [gss-curve25519-sha256-,gss-nistp256-sha256-,gss-group14-sha256-,gss-group16-sha512-]\r\ndebug3: kex names ok: [curve25519-sha256,[email protected],ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group14-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512]\r\ndebug1: configuration requests final Match pass\r\ndebug2: resolve_canonicalize: hostname 192.168.111.138 is address\r\ndebug1: re-parsing configuration\r\ndebug1: Reading configuration data /etc/ssh/ssh_config\r\ndebug3: /etc/ssh/ssh_config line 55: Including file /etc/ssh/ssh_config.d/50-redhat.conf depth 0\r\ndebug1: Reading configuration data /etc/ssh/ssh_config.d/50-redhat.conf\r\ndebug2: checking match for 'final all' host 192.168.111.138 originally 192.168.111.138\r\ndebug3: /etc/ssh/ssh_config.d/50-redhat.conf line 3: matched 'final'\r\ndebug2: match found\r\ndebug3: /etc/ssh/ssh_config.d/50-redhat.conf line 5: Including file /etc/crypto-policies/back-ends/openssh.config depth 1\r\ndebug1: Reading configuration data /etc/crypto-policies/back-ends/openssh.config\r\ndebug3: gss kex names ok: [gss-curve25519-sha256-,gss-nistp256-sha256-,gss-group14-sha256-,gss-group16-sha512-]\r\ndebug3: kex names ok: [curve25519-sha256,[email protected],ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group14-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512]\r\ndebug3: expanded UserKnownHostsFile '~/.ssh/known_hosts' -> '/root/.ssh/known_hosts'\r\ndebug3: expanded UserKnownHostsFile '~/.ssh/known_hosts2' -> '/root/.ssh/known_hosts2'\r\ndebug1: auto-mux: Trying existing master\r\ndebug2: fd 3 setting O_NONBLOCK\r\ndebug2: mux_client_hello_exchange: master version 4\r\ndebug3: mux_client_forwards: request forwardings: 0 local, 0 remote\r\ndebug3: mux_client_request_session: entering\r\ndebug3: mux_client_request_alive: entering\r\ndebug3: mux_client_request_alive: done pid = 4378\r\ndebug3: mux_client_request_session: session request sent\r\ndebug1: mux_client_request_session: master session id: 2\r\ndebug3: mux_client_read_packet: read header failed: Broken pipe\r\ndebug2: Received exit status from master 0\r\n")
redirecting (type: modules) community.kubernetes.k8s to kubernetes.core.k8s
Using module file /root/.ansible/collections/ansible_collections/kubernetes/core/plugins/modules/k8s.py
<192.168.111.138> PUT /root/.ansible/tmp/ansible-local-4367vfq387sm/tmpij2nghdd TO /home/kube/.ansible/tmp/ansible-tmp-1723209062.7628653-4397-109600890829893/AnsiballZ_k8s.py
<192.168.111.138> SSH: EXEC sftp -b - -vvv -C -o ControlMaster=auto -o ControlPersist=60s -o StrictHostKeyChecking=no -o 'IdentityFile="/root/.ssh/kube"' -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o 'User="kube"' -o ConnectTimeout=10 -o 'ControlPath="/root/.ansible/cp/8e11ae4af7"' '[192.168.111.138]'
<192.168.111.138> (0, b'sftp> put /root/.ansible/tmp/ansible-local-4367vfq387sm/tmpij2nghdd /home/kube/.ansible/tmp/ansible-tmp-1723209062.7628653-4397-109600890829893/AnsiballZ_k8s.py\n', b'OpenSSH_8.7p1, OpenSSL 3.0.7 1 Nov 2022\r\ndebug1: Reading configuration data /etc/ssh/ssh_config\r\ndebug3: /etc/ssh/ssh_config line 55: Including file /etc/ssh/ssh_config.d/50-redhat.conf depth 0\r\ndebug1: Reading configuration data /etc/ssh/ssh_config.d/50-redhat.conf\r\ndebug2: checking match for \'final all\' host 192.168.111.138 originally 192.168.111.138\r\ndebug3: /etc/ssh/ssh_config.d/50-redhat.conf line 3: not matched \'final\'\r\ndebug2: match not found\r\ndebug3: /etc/ssh/ssh_config.d/50-redhat.conf line 5: Including file /etc/crypto-policies/back-ends/openssh.config depth 1 (parse only)\r\ndebug1: Reading configuration data /etc/crypto-policies/back-ends/openssh.config\r\ndebug3: gss kex names ok: [gss-curve25519-sha256-,gss-nistp256-sha256-,gss-group14-sha256-,gss-group16-sha512-]\r\ndebug3: kex names ok: [curve25519-sha256,[email protected],ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group14-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512]\r\ndebug1: configuration requests final Match pass\r\ndebug2: resolve_canonicalize: hostname 192.168.111.138 is address\r\ndebug1: re-parsing configuration\r\ndebug1: Reading configuration data /etc/ssh/ssh_config\r\ndebug3: /etc/ssh/ssh_config line 55: Including file /etc/ssh/ssh_config.d/50-redhat.conf depth 0\r\ndebug1: Reading configuration data /etc/ssh/ssh_config.d/50-redhat.conf\r\ndebug2: checking match for \'final all\' host 192.168.111.138 originally 192.168.111.138\r\ndebug3: /etc/ssh/ssh_config.d/50-redhat.conf line 3: matched \'final\'\r\ndebug2: match found\r\ndebug3: /etc/ssh/ssh_config.d/50-redhat.conf line 5: Including file /etc/crypto-policies/back-ends/openssh.config depth 1\r\ndebug1: Reading configuration data /etc/crypto-policies/back-ends/openssh.config\r\ndebug3: gss kex names ok: [gss-curve25519-sha256-,gss-nistp256-sha256-,gss-group14-sha256-,gss-group16-sha512-]\r\ndebug3: kex names ok: [curve25519-sha256,[email protected],ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group14-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512]\r\ndebug3: expanded UserKnownHostsFile \'~/.ssh/known_hosts\' -> \'/root/.ssh/known_hosts\'\r\ndebug3: expanded UserKnownHostsFile \'~/.ssh/known_hosts2\' -> \'/root/.ssh/known_hosts2\'\r\ndebug1: auto-mux: Trying existing master\r\ndebug2: fd 3 setting O_NONBLOCK\r\ndebug2: mux_client_hello_exchange: master version 4\r\ndebug3: mux_client_forwards: request forwardings: 0 local, 0 remote\r\ndebug3: mux_client_request_session: entering\r\ndebug3: mux_client_request_alive: entering\r\ndebug3: mux_client_request_alive: done pid = 4378\r\ndebug3: mux_client_request_session: session request sent\r\ndebug1: mux_client_request_session: master session id: 2\r\ndebug2: Remote version: 3\r\ndebug2: Server supports extension "[email protected]" revision 1\r\ndebug2: Server supports extension "[email protected]" revision 2\r\ndebug2: Server supports extension "[email protected]" revision 2\r\ndebug2: Server supports extension "[email protected]" revision 1\r\ndebug2: Server supports extension "[email protected]" revision 1\r\ndebug2: Server supports extension "[email protected]" revision 1\r\ndebug2: Server supports extension "[email protected]" revision 1\r\ndebug2: Server supports extension "[email protected]" revision 1\r\ndebug3: Sent message [email protected] I:1\r\ndebug3: Received limits reply T:201 I:1\r\ndebug1: Using server download size 261120\r\ndebug1: Using server upload size 261120\r\ndebug1: Server handle limit 1019; using 64\r\ndebug3: Sent message fd 3 T:16 I:2\r\ndebug3: SSH2_FXP_REALPATH . -> /home/kube\r\ndebug3: Looking up /root/.ansible/tmp/ansible-local-4367vfq387sm/tmpij2nghdd\r\ndebug3: Sent message fd 3 T:17 I:3\r\ndebug1: Couldn\'t stat remote file: No such file or directory\r\ndebug3: Sent dest message SSH2_FXP_OPEN I:4 P:/home/kube/.ansible/tmp/ansible-tmp-1723209062.7628653-4397-109600890829893/AnsiballZ_k8s.py M:0x001a\r\ndebug3: Sent message SSH2_FXP_WRITE I:6 O:0 S:235207\r\ndebug3: SSH2_FXP_STATUS 0\r\ndebug3: In write loop, ack for 6 235207 bytes at 0\r\ndebug3: Sent message SSH2_FXP_CLOSE I:5\r\ndebug3: SSH2_FXP_STATUS 0\r\ndebug3: mux_client_read_packet: read header failed: Broken pipe\r\ndebug2: Received exit status from master 0\r\n')
<192.168.111.138> ESTABLISH SSH CONNECTION FOR USER: kube
<192.168.111.138> SSH: EXEC ssh -vvv -C -o ControlMaster=auto -o ControlPersist=60s -o StrictHostKeyChecking=no -o 'IdentityFile="/root/.ssh/kube"' -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o 'User="kube"' -o ConnectTimeout=10 -o 'ControlPath="/root/.ansible/cp/8e11ae4af7"' 192.168.111.138 '/bin/sh -c '"'"'chmod u+x /home/kube/.ansible/tmp/ansible-tmp-1723209062.7628653-4397-109600890829893/ /home/kube/.ansible/tmp/ansible-tmp-1723209062.7628653-4397-109600890829893/AnsiballZ_k8s.py && sleep 0'"'"''
<192.168.111.138> (0, b'', b"OpenSSH_8.7p1, OpenSSL 3.0.7 1 Nov 2022\r\ndebug1: Reading configuration data /etc/ssh/ssh_config\r\ndebug3: /etc/ssh/ssh_config line 55: Including file /etc/ssh/ssh_config.d/50-redhat.conf depth 0\r\ndebug1: Reading configuration data /etc/ssh/ssh_config.d/50-redhat.conf\r\ndebug2: checking match for 'final all' host 192.168.111.138 originally 192.168.111.138\r\ndebug3: /etc/ssh/ssh_config.d/50-redhat.conf line 3: not matched 'final'\r\ndebug2: match not found\r\ndebug3: /etc/ssh/ssh_config.d/50-redhat.conf line 5: Including file /etc/crypto-policies/back-ends/openssh.config depth 1 (parse only)\r\ndebug1: Reading configuration data /etc/crypto-policies/back-ends/openssh.config\r\ndebug3: gss kex names ok: [gss-curve25519-sha256-,gss-nistp256-sha256-,gss-group14-sha256-,gss-group16-sha512-]\r\ndebug3: kex names ok: [curve25519-sha256,[email protected],ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group14-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512]\r\ndebug1: configuration requests final Match pass\r\ndebug2: resolve_canonicalize: hostname 192.168.111.138 is address\r\ndebug1: re-parsing configuration\r\ndebug1: Reading configuration data /etc/ssh/ssh_config\r\ndebug3: /etc/ssh/ssh_config line 55: Including file /etc/ssh/ssh_config.d/50-redhat.conf depth 0\r\ndebug1: Reading configuration data /etc/ssh/ssh_config.d/50-redhat.conf\r\ndebug2: checking match for 'final all' host 192.168.111.138 originally 192.168.111.138\r\ndebug3: /etc/ssh/ssh_config.d/50-redhat.conf line 3: matched 'final'\r\ndebug2: match found\r\ndebug3: /etc/ssh/ssh_config.d/50-redhat.conf line 5: Including file /etc/crypto-policies/back-ends/openssh.config depth 1\r\ndebug1: Reading configuration data /etc/crypto-policies/back-ends/openssh.config\r\ndebug3: gss kex names ok: [gss-curve25519-sha256-,gss-nistp256-sha256-,gss-group14-sha256-,gss-group16-sha512-]\r\ndebug3: kex names ok: [curve25519-sha256,[email protected],ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group14-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512]\r\ndebug3: expanded UserKnownHostsFile '~/.ssh/known_hosts' -> '/root/.ssh/known_hosts'\r\ndebug3: expanded UserKnownHostsFile '~/.ssh/known_hosts2' -> '/root/.ssh/known_hosts2'\r\ndebug1: auto-mux: Trying existing master\r\ndebug2: fd 3 setting O_NONBLOCK\r\ndebug2: mux_client_hello_exchange: master version 4\r\ndebug3: mux_client_forwards: request forwardings: 0 local, 0 remote\r\ndebug3: mux_client_request_session: entering\r\ndebug3: mux_client_request_alive: entering\r\ndebug3: mux_client_request_alive: done pid = 4378\r\ndebug3: mux_client_request_session: session request sent\r\ndebug1: mux_client_request_session: master session id: 2\r\ndebug3: mux_client_read_packet: read header failed: Broken pipe\r\ndebug2: Received exit status from master 0\r\n")
<192.168.111.138> ESTABLISH SSH CONNECTION FOR USER: kube
<192.168.111.138> SSH: EXEC ssh -vvv -C -o ControlMaster=auto -o ControlPersist=60s -o StrictHostKeyChecking=no -o 'IdentityFile="/root/.ssh/kube"' -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o 'User="kube"' -o ConnectTimeout=10 -o 'ControlPath="/root/.ansible/cp/8e11ae4af7"' -tt 192.168.111.138 '/bin/sh -c '"'"'/usr/bin/python3 /home/kube/.ansible/tmp/ansible-tmp-1723209062.7628653-4397-109600890829893/AnsiballZ_k8s.py && sleep 0'"'"''
<192.168.111.138> (1, b'\r\n{"exception": "Traceback (most recent call last):\\n  File \\"/tmp/ansible_community.kubernetes.k8s_payload_q_jzlnzh/ansible_community.kubernetes.k8s_payload.zip/ansible_collections/kubernetes/core/plugins/module_utils/k8s/runner.py\\", line 49, in get_definitions\\n    definitions = create_definitions(params)\\n  File \\"/tmp/ansible_community.kubernetes.k8s_payload_q_jzlnzh/ansible_community.kubernetes.k8s_payload.zip/ansible_collections/kubernetes/core/plugins/module_utils/k8s/resource.py\\", line 61, in create_definitions\\n    definitions = from_file(d)\\n  File \\"/tmp/ansible_community.kubernetes.k8s_payload_q_jzlnzh/ansible_community.kubernetes.k8s_payload.zip/ansible_collections/kubernetes/core/plugins/module_utils/k8s/resource.py\\", line 98, in from_file\\n    with open(path, \\"rb\\") as f:\\nPermissionError: [Errno 13] Permission denied: \'/root/ansible/Ansible/k8s_deployments/nginx.yaml\'\\n\\nThe above exception was the direct cause of the following exception:\\n\\nTraceback (most recent call last):\\n  File \\"/tmp/ansible_community.kubernetes.k8s_payload_q_jzlnzh/ansible_community.kubernetes.k8s_payload.zip/ansible_collections/kubernetes/core/plugins/modules/k8s.py\\", line 503, in main\\n  File \\"/tmp/ansible_community.kubernetes.k8s_payload_q_jzlnzh/ansible_community.kubernetes.k8s_payload.zip/ansible_collections/kubernetes/core/plugins/module_utils/k8s/runner.py\\", line 91, in run_module\\n    definitions = get_definitions(svc, module.params)\\n  File \\"/tmp/ansible_community.kubernetes.k8s_payload_q_jzlnzh/ansible_community.kubernetes.k8s_payload.zip/ansible_collections/kubernetes/core/plugins/module_utils/k8s/runner.py\\", line 52, in get_definitions\\n    raise CoreException(msg) from e\\nansible_collections.kubernetes.core.plugins.module_utils.k8s.exceptions.CoreException: Failed to load resource definition: [Errno 13] Permission denied: \'/root/ansible/Ansible/k8s_deployments/nginx.yaml\'\\n", "failed": true, "msg": "Failed to load resource definition: [Errno 13] Permission denied: \'/root/ansible/Ansible/k8s_deployments/nginx.yaml\'", "invocation": {"module_args": {"src": "/root/ansible/Ansible/k8s_deployments/nginx.yaml", "state": "present", "kubeconfig": "/home/kube/.kube/config", "namespace": "my-app", "api_version": "v1", "wait": false, "wait_sleep": 5, "wait_timeout": 120, "append_hash": false, "apply": false, "continue_on_error": false, "force": false, "delete_all": false, "kind": null, "name": null, "resource_definition": null, "context": null, "host": null, "api_key": null, "username": null, "password": null, "validate_certs": null, "ca_cert": null, "client_cert": null, "client_key": null, "proxy": null, "no_proxy": null, "proxy_headers": null, "persist_config": null, "impersonate_user": null, "impersonate_groups": null, "wait_condition": null, "merge_type": null, "validate": null, "template": null, "delete_options": null, "label_selectors": null, "generate_name": null, "server_side_apply": null, "hidden_fields": null}}}\r\n', b"OpenSSH_8.7p1, OpenSSL 3.0.7 1 Nov 2022\r\ndebug1: Reading configuration data /etc/ssh/ssh_config\r\ndebug3: /etc/ssh/ssh_config line 55: Including file /etc/ssh/ssh_config.d/50-redhat.conf depth 0\r\ndebug1: Reading configuration data /etc/ssh/ssh_config.d/50-redhat.conf\r\ndebug2: checking match for 'final all' host 192.168.111.138 originally 192.168.111.138\r\ndebug3: /etc/ssh/ssh_config.d/50-redhat.conf line 3: not matched 'final'\r\ndebug2: match not found\r\ndebug3: /etc/ssh/ssh_config.d/50-redhat.conf line 5: Including file /etc/crypto-policies/back-ends/openssh.config depth 1 (parse only)\r\ndebug1: Reading configuration data /etc/crypto-policies/back-ends/openssh.config\r\ndebug3: gss kex names ok: [gss-curve25519-sha256-,gss-nistp256-sha256-,gss-group14-sha256-,gss-group16-sha512-]\r\ndebug3: kex names ok: [curve25519-sha256,[email protected],ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group14-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512]\r\ndebug1: configuration requests final Match pass\r\ndebug2: resolve_canonicalize: hostname 192.168.111.138 is address\r\ndebug1: re-parsing configuration\r\ndebug1: Reading configuration data /etc/ssh/ssh_config\r\ndebug3: /etc/ssh/ssh_config line 55: Including file /etc/ssh/ssh_config.d/50-redhat.conf depth 0\r\ndebug1: Reading configuration data /etc/ssh/ssh_config.d/50-redhat.conf\r\ndebug2: checking match for 'final all' host 192.168.111.138 originally 192.168.111.138\r\ndebug3: /etc/ssh/ssh_config.d/50-redhat.conf line 3: matched 'final'\r\ndebug2: match found\r\ndebug3: /etc/ssh/ssh_config.d/50-redhat.conf line 5: Including file /etc/crypto-policies/back-ends/openssh.config depth 1\r\ndebug1: Reading configuration data /etc/crypto-policies/back-ends/openssh.config\r\ndebug3: gss kex names ok: [gss-curve25519-sha256-,gss-nistp256-sha256-,gss-group14-sha256-,gss-group16-sha512-]\r\ndebug3: kex names ok: [curve25519-sha256,[email protected],ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group14-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512]\r\ndebug3: expanded UserKnownHostsFile '~/.ssh/known_hosts' -> '/root/.ssh/known_hosts'\r\ndebug3: expanded UserKnownHostsFile '~/.ssh/known_hosts2' -> '/root/.ssh/known_hosts2'\r\ndebug1: auto-mux: Trying existing master\r\ndebug2: fd 3 setting O_NONBLOCK\r\ndebug2: mux_client_hello_exchange: master version 4\r\ndebug3: mux_client_forwards: request forwardings: 0 local, 0 remote\r\ndebug3: mux_client_request_session: entering\r\ndebug3: mux_client_request_alive: entering\r\ndebug3: mux_client_request_alive: done pid = 4378\r\ndebug3: mux_client_request_session: session request sent\r\ndebug1: mux_client_request_session: master session id: 2\r\ndebug3: mux_client_read_packet: read header failed: Broken pipe\r\ndebug2: Received exit status from master 1\r\nShared connection to 192.168.111.138 closed.\r\n")
<192.168.111.138> Failed to connect to the host via ssh: OpenSSH_8.7p1, OpenSSL 3.0.7 1 Nov 2022
debug1: Reading configuration data /etc/ssh/ssh_config
debug3: /etc/ssh/ssh_config line 55: Including file /etc/ssh/ssh_config.d/50-redhat.conf depth 0
debug1: Reading configuration data /etc/ssh/ssh_config.d/50-redhat.conf
debug2: checking match for 'final all' host 192.168.111.138 originally 192.168.111.138
debug3: /etc/ssh/ssh_config.d/50-redhat.conf line 3: not matched 'final'
debug2: match not found
debug3: /etc/ssh/ssh_config.d/50-redhat.conf line 5: Including file /etc/crypto-policies/back-ends/openssh.config depth 1 (parse only)
debug1: Reading configuration data /etc/crypto-policies/back-ends/openssh.config
debug3: gss kex names ok: [gss-curve25519-sha256-,gss-nistp256-sha256-,gss-group14-sha256-,gss-group16-sha512-]
debug3: kex names ok: [curve25519-sha256,[email protected],ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group14-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512]
debug1: configuration requests final Match pass
debug2: resolve_canonicalize: hostname 192.168.111.138 is address
debug1: re-parsing configuration
debug1: Reading configuration data /etc/ssh/ssh_config
debug3: /etc/ssh/ssh_config line 55: Including file /etc/ssh/ssh_config.d/50-redhat.conf depth 0
debug1: Reading configuration data /etc/ssh/ssh_config.d/50-redhat.conf
debug2: checking match for 'final all' host 192.168.111.138 originally 192.168.111.138
debug3: /etc/ssh/ssh_config.d/50-redhat.conf line 3: matched 'final'
debug2: match found
debug3: /etc/ssh/ssh_config.d/50-redhat.conf line 5: Including file /etc/crypto-policies/back-ends/openssh.config depth 1
debug1: Reading configuration data /etc/crypto-policies/back-ends/openssh.config
debug3: gss kex names ok: [gss-curve25519-sha256-,gss-nistp256-sha256-,gss-group14-sha256-,gss-group16-sha512-]
debug3: kex names ok: [curve25519-sha256,[email protected],ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group14-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512]
debug3: expanded UserKnownHostsFile '~/.ssh/known_hosts' -> '/root/.ssh/known_hosts'
debug3: expanded UserKnownHostsFile '~/.ssh/known_hosts2' -> '/root/.ssh/known_hosts2'
debug1: auto-mux: Trying existing master
debug2: fd 3 setting O_NONBLOCK
debug2: mux_client_hello_exchange: master version 4
debug3: mux_client_forwards: request forwardings: 0 local, 0 remote
debug3: mux_client_request_session: entering
debug3: mux_client_request_alive: entering
debug3: mux_client_request_alive: done pid = 4378
debug3: mux_client_request_session: session request sent
debug1: mux_client_request_session: master session id: 2
debug3: mux_client_read_packet: read header failed: Broken pipe
debug2: Received exit status from master 1
Shared connection to 192.168.111.138 closed.
<192.168.111.138> ESTABLISH SSH CONNECTION FOR USER: kube
<192.168.111.138> SSH: EXEC ssh -vvv -C -o ControlMaster=auto -o ControlPersist=60s -o StrictHostKeyChecking=no -o 'IdentityFile="/root/.ssh/kube"' -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o 'User="kube"' -o ConnectTimeout=10 -o 'ControlPath="/root/.ansible/cp/8e11ae4af7"' 192.168.111.138 '/bin/sh -c '"'"'rm -f -r /home/kube/.ansible/tmp/ansible-tmp-1723209062.7628653-4397-109600890829893/ > /dev/null 2>&1 && sleep 0'"'"''
<192.168.111.138> (0, b'', b"OpenSSH_8.7p1, OpenSSL 3.0.7 1 Nov 2022\r\ndebug1: Reading configuration data /etc/ssh/ssh_config\r\ndebug3: /etc/ssh/ssh_config line 55: Including file /etc/ssh/ssh_config.d/50-redhat.conf depth 0\r\ndebug1: Reading configuration data /etc/ssh/ssh_config.d/50-redhat.conf\r\ndebug2: checking match for 'final all' host 192.168.111.138 originally 192.168.111.138\r\ndebug3: /etc/ssh/ssh_config.d/50-redhat.conf line 3: not matched 'final'\r\ndebug2: match not found\r\ndebug3: /etc/ssh/ssh_config.d/50-redhat.conf line 5: Including file /etc/crypto-policies/back-ends/openssh.config depth 1 (parse only)\r\ndebug1: Reading configuration data /etc/crypto-policies/back-ends/openssh.config\r\ndebug3: gss kex names ok: [gss-curve25519-sha256-,gss-nistp256-sha256-,gss-group14-sha256-,gss-group16-sha512-]\r\ndebug3: kex names ok: [curve25519-sha256,[email protected],ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group14-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512]\r\ndebug1: configuration requests final Match pass\r\ndebug2: resolve_canonicalize: hostname 192.168.111.138 is address\r\ndebug1: re-parsing configuration\r\ndebug1: Reading configuration data /etc/ssh/ssh_config\r\ndebug3: /etc/ssh/ssh_config line 55: Including file /etc/ssh/ssh_config.d/50-redhat.conf depth 0\r\ndebug1: Reading configuration data /etc/ssh/ssh_config.d/50-redhat.conf\r\ndebug2: checking match for 'final all' host 192.168.111.138 originally 192.168.111.138\r\ndebug3: /etc/ssh/ssh_config.d/50-redhat.conf line 3: matched 'final'\r\ndebug2: match found\r\ndebug3: /etc/ssh/ssh_config.d/50-redhat.conf line 5: Including file /etc/crypto-policies/back-ends/openssh.config depth 1\r\ndebug1: Reading configuration data /etc/crypto-policies/back-ends/openssh.config\r\ndebug3: gss kex names ok: [gss-curve25519-sha256-,gss-nistp256-sha256-,gss-group14-sha256-,gss-group16-sha512-]\r\ndebug3: kex names ok: [curve25519-sha256,[email protected],ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group14-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512]\r\ndebug3: expanded UserKnownHostsFile '~/.ssh/known_hosts' -> '/root/.ssh/known_hosts'\r\ndebug3: expanded UserKnownHostsFile '~/.ssh/known_hosts2' -> '/root/.ssh/known_hosts2'\r\ndebug1: auto-mux: Trying existing master\r\ndebug2: fd 3 setting O_NONBLOCK\r\ndebug2: mux_client_hello_exchange: master version 4\r\ndebug3: mux_client_forwards: request forwardings: 0 local, 0 remote\r\ndebug3: mux_client_request_session: entering\r\ndebug3: mux_client_request_alive: entering\r\ndebug3: mux_client_request_alive: done pid = 4378\r\ndebug3: mux_client_request_session: session request sent\r\ndebug1: mux_client_request_session: master session id: 2\r\ndebug3: mux_client_read_packet: read header failed: Broken pipe\r\ndebug2: Received exit status from master 0\r\n")
The full traceback is:
Traceback (most recent call last):
  File "/tmp/ansible_community.kubernetes.k8s_payload_q_jzlnzh/ansible_community.kubernetes.k8s_payload.zip/ansible_collections/kubernetes/core/plugins/module_utils/k8s/runner.py", line 49, in get_definitions
    definitions = create_definitions(params)
  File "/tmp/ansible_community.kubernetes.k8s_payload_q_jzlnzh/ansible_community.kubernetes.k8s_payload.zip/ansible_collections/kubernetes/core/plugins/module_utils/k8s/resource.py", line 61, in create_definitions
    definitions = from_file(d)
  File "/tmp/ansible_community.kubernetes.k8s_payload_q_jzlnzh/ansible_community.kubernetes.k8s_payload.zip/ansible_collections/kubernetes/core/plugins/module_utils/k8s/resource.py", line 98, in from_file
    with open(path, "rb") as f:
PermissionError: [Errno 13] Permission denied: '/root/ansible/Ansible/k8s_deployments/nginx.yaml'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/tmp/ansible_community.kubernetes.k8s_payload_q_jzlnzh/ansible_community.kubernetes.k8s_payload.zip/ansible_collections/kubernetes/core/plugins/modules/k8s.py", line 503, in main
  File "/tmp/ansible_community.kubernetes.k8s_payload_q_jzlnzh/ansible_community.kubernetes.k8s_payload.zip/ansible_collections/kubernetes/core/plugins/module_utils/k8s/runner.py", line 91, in run_module
    definitions = get_definitions(svc, module.params)
  File "/tmp/ansible_community.kubernetes.k8s_payload_q_jzlnzh/ansible_community.kubernetes.k8s_payload.zip/ansible_collections/kubernetes/core/plugins/module_utils/k8s/runner.py", line 52, in get_definitions
    raise CoreException(msg) from e
ansible_collections.kubernetes.core.plugins.module_utils.k8s.exceptions.CoreException: Failed to load resource definition: [Errno 13] Permission denied: '/root/ansible/Ansible/k8s_deployments/nginx.yaml'
fatal: [192.168.111.138]: FAILED! => {
    "changed": false,
    "invocation": {
        "module_args": {
            "api_key": null,
            "api_version": "v1",
            "append_hash": false,
            "apply": false,
            "ca_cert": null,
            "client_cert": null,
            "client_key": null,
            "context": null,
            "continue_on_error": false,
            "delete_all": false,
            "delete_options": null,
            "force": false,
            "generate_name": null,
            "hidden_fields": null,
            "host": null,
            "impersonate_groups": null,
            "impersonate_user": null,
            "kind": null,
            "kubeconfig": "/home/kube/.kube/config",
            "label_selectors": null,
            "merge_type": null,
            "name": null,
            "namespace": "my-app",
            "no_proxy": null,
            "password": null,
            "persist_config": null,
            "proxy": null,
            "proxy_headers": null,
            "resource_definition": null,
            "server_side_apply": null,
            "src": "/root/ansible/Ansible/k8s_deployments/nginx.yaml",
            "state": "present",
            "template": null,
            "username": null,
            "validate": null,
            "validate_certs": null,
            "wait": false,
            "wait_condition": null,
            "wait_sleep": 5,
            "wait_timeout": 120
        }
    },
    "msg": "Failed to load resource definition: [Errno 13] Permission denied: '/root/ansible/Ansible/k8s_deployments/nginx.yaml'"
}

[ibrahim-osama-amin]

Tried to upgrade ansible today but I am still getting the same issue, most likely I am doing something incorrectly and this is not a bug.

[root@localhost Ansible]# ansible --version
ansible [core 2.17.0]
  config file = /root/ansible/Ansible/ansible.cfg
  configured module search path = ['/root/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/local/lib/python3.12/site-packages/ansible
  ansible collection location = /root/.ansible/collections:/usr/share/ansible/collections
  executable location = /usr/bin/ansible
  python version = 3.12.1 (main, May  3 2024, 00:00:00) [GCC 11.4.1 20231218 (Red Hat 11.4.1-3)] (/usr/bin/python3)
  jinja version = 3.1.4
  libyaml = True

[dkarpele]

Hi @ibrahim-osama-amin

It is weird because I am running as root so I shouldn't face any permission issues.

There is a line in log:
<192.168.111.138> ESTABLISH SSH CONNECTION FOR USER: kube

Are you sure that you are running as root?
You can also try to change permissions for nginx.yaml or move the file under /home/kube/. Pay attention that this file is in /root/ansible/Ansible/k8s_deployments/ .
Also you can try to experiment with become keyword