Framework for Nebari deployment via pytest for extensive testing

[aktech]

Part of fixing #1862

This creates a framework to write tests that are designed to test things on Nebari deployed on cloud. At the moment it only deploys on DigitalOcean, which can be easily expanded to all other clouds.

It would be better to merge this small chunk and then work on other cloud providers, rather than a single PR handling all clouds making it difficult to review.

Testing on cloud would be simply adding the decorator @on_cloud("cloud_name"):

@on_cloud("do")
def test_something_on_do_deployment(deploy):
	# create notebook
    # test dask
    # test jupyterhub
    # test conda store

Other Fixes:

• Fixes fetching Kubernetes Version for DigitalOcean
• Playwright cleanups

Notes:

• More extensive testing like, running notebooks, dask, jupyterhub will be done in follow up PRs
• Example run: https://github.com/nebari-dev/nebari/actions/runs/5620616744/job/15229991582
• Will also add pytest markers for clouds/testing type, etc in subsequent PRs

What does this implement/fix?

Put a x in the boxes that apply

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds a feature)
• Breaking change (fix or feature that would cause existing features not to work as expected)
• Documentation Update
• Code style update (formatting, renaming)
• Refactoring (no functional changes, no API changes)
• Build related changes
• Other (please describe):

Testing

• Did you test the pull request locally?
• Did you add new tests?

Any other comments?

[iameskild]

Reminder for @costrouc and @iameskild to test this as soon as possible.

[iameskild]

Thanks for the great work @aktech!

Given that this will be used to deploy cloud-based Nebari clusters, will this replace some of the manual setup we do for the integration tests? That's what I recall from our previous conversation with @costrouc.

Lastly, I'm unfortunately unable to test this locally because Digital Ocean just dropped support for kubernetes 1.24...

[iameskild]

@aktech the one other thing that will likely be worthwhile adding is some sort of clean up script so in the event things are fully destroyed, we can still ensure the cloud resources are removed. We have a script for AWS (python and bash) but for the other cloud providers, we can either copy over what we have from the integration tests or write a python script that handles this for us.

[aktech]

@iameskild makes sense about the clean up script. How do you feel about I tackle that in a follow up PR? we're not going to be running the integration tests on each push at the moment.

[iameskild]

@aktech this looks good to me! As we discussed offline, we can open new PRs for any fixes or updates. Thanks again 🎉