[BUG] - Fix Paramiko SSH Key Discovery when running tests locally

[viniciusdc]

Describe the bug

When running our Pytests locally, the tests under tests_deployment/test_jupyterhub_ssh.py rely on Paramiko to handle the SSH connection within the Jupyterhub-ssh plugin. While these tests run smoothly in CI due to a single SSH key being configured in the runner's environment, problems arise when users attempt to run the same tests locally. The current setup triggers SSH key discovery in the user's host configuration, leading to incorrect authentications, credential prompts, and, ultimately, failed connections.

    raise SSHException("No existing session")
paramiko.ssh_exception.SSHException: No existing session
--------------- Captured log setup ------------------
2024-05-06 11:56:00      INFO 1909 transport: Connected (version 2.0, client AsyncSSH_2.13.2)
2024-05-06 11:56:32     ERROR 1909 transport: Exception (client): key cannot be used for signing
2024-05-06 11:56:32     ERROR 1907 transport: Traceback (most recent call last):
2024-05-06 11:56:32     ERROR 1907 transport:   File "envs/nebari/lib/python3.10/site-packages/paramiko/transport.py", line 2220, in run
2024-05-06 11:56:32     ERROR 1907 transport:     handler(m)
2024-05-06 11:56:32     ERROR 1907 transport:   File "site-packages/paramiko/auth_handler.py", line 404, in _parse_service_accept
2024-05-06 11:56:32     ERROR 1907 transport:     sig = self.private_key.sign_ssh_data(blob, algorithm)
2024-05-06 11:56:32     ERROR 1907 transport:   File "envs/nebari/lib/python3.10/site-packages/paramiko/agent.py", line 496, in sign_ssh_data
2024-05-06 11:56:32     ERROR 1907 transport:     raise SSHException("key cannot be used for signing")
2024-05-06 11:56:32     ERROR 1907 transport: paramiko.ssh_exception.SSHException: key cannot be used for signing
2024-05-06 11:56:32     ERROR 1907 transport:

Since the SSH connection parameters only require the username and token (user token), there is no need for SSH key discovery or access to the host's SSH key agent. According to Paramiko's documentation on connection behavior (see Paramiko Docs), this behavior can be disabled by adjusting the following settings:

• allow_agent (bool) – Set to False to disable connecting to the SSH agent.
• look_for_keys (bool) – Set to False to disable searching for discoverable private key files in ~/.ssh/.

Expected behavior

Tests should run correctly when running locally

OS and architecture in which you are running Nebari

Linux

How to Reproduce the problem?

Run the following with a proper install of nebari and a viable/accessible local nebari deployment:

pytest tests/tests_deployment/test_jupyterhub_ssh.py -v

Make sure to set up the following environment variables prior to running any tests:

NEBARI_HOSTNAME 
NEBARI_CONFIG_PATH 
--------
KEYCLOAK_USERNAME 
KEYCLOAK_PASSWORD

For the user credentials, you can create these locally (within the same dir as the nebari config):

nebari keycloak adduser -c nebari-config.yaml --user testuser test

Command output

No response

Versions and dependencies used.

No response

Compute environment

kind

Integrations

No response

Anything else?

No response

[marcelovilla]

@viniciusdc I'm reopening this issue as we're seeing our local integration tests fail with a paramiko related error.

[viniciusdc]

After a quick talk, this might be related to the user node not being ready and the ssh connection timing out