Nebari Distribution Proposal

[Adam-D-Lewis]

In Quansight, we've been talking about the idea of a Nebari distribution. I think the first thing we should do is to define what a distribution is. Here's my proposed definition below.

What is a distribution?

A nebari distribution is a tested set of configuration that is known to work and is usually geared towards a specific use case (see examples below).

Users deploying nebari can feel confident that if they deploy a distribution, it will be a good starting point for them.

Distribution Examples

For the sake of discussion, these are examples of distributions that should be possible.

• low cost nebari - defined by cheaper node types, disabling some optional features in nebari config, etc.
• gpu nebari - defined by gpu node types and profiles
• ML nebari - defined by gpu node types/profiles, ML conda environments, nebari-mlflow-plugin
• Ragna app - automatically deploys ragna app on Nebari on initial deployment along with conda environment and a workflow for downloading a local LLM model

Supported Features of a Distribution

As motivated by the distribution examples, a distribution should be able to define the following:

1. Nebari Config Options*
   • Any option in the nebari config file.
   • e.g. this would be needed to set up gpu node types and jupyter profiles
2. Conda environments
   • environments created during nebari deployment. These could be specified in multiple ways e.g. conda project, conda env file, etc.
3. Workflows**
   • workflows which run conditionally (e.g. initial deployment only, if file doesn't exist, etc.) during deployment. These could be specified in multiple ways e.g. conda project, argo, k8s job, etc.
   • used to download the LLM model weights in the ragna example distribution
4. Jhub Apps:
   • apps deployed during nebari deployment
   • these could be specified in multiple ways e.g. conda project or direct definition
5. Nebari plugins:
   • Nebari plugins which should be installed and included in the deployment
   • e.g. needed by the ML distribution of Nebari.

* Custom docker images could be specified as part of this.
**I could see us replacing Workflows (or supplementing it) with a "Downloads" feature which downloads files and puts them where specified in the NFS drive on initial deployment. It could use some type of workflow under the hood, but that would be an implementation detail.

How to define a distribution:

Various options exist to create a distribution:
- Nebari distribution plugin (not different from a regular nebari plugin, just want user to keep in mind a distribution plugin is likely to have dependencies on other plugins (e.g. Nebari MLflow plugin))
- yaml file
- other?

Nebari Plugin defined Distribution

I see a Nebari plugin as very robust and capable choice, but it is more difficult to create a distribution since you have to create a python package. I don't see that difficulty as an issue since I think anyone considering creating a Nebari plugin is capable of creating a python package or learning how to do so. This also allows us to specify dependencies on nebari and other plugins in the usual manner (pyproject.yaml).

Yaml File defined Distribution

I see a yaml file as not robust enough and likely to result in us trying to create a DSL in yaml to handle many different distribution needs. Nevertheless, I've tried to define below for the sake of argument. I didn't put it in a code block so that the scroll bars wouldn't appear.

distro_schema_version: 1.0
supported_nebari_versions: >=2024.11.1  # **Con: More clumsy than just using pyproject.yaml**
# anything specified overrides what nebari generates
config_options:

# **Con: You need a separate distribution for each cloud provider for gpus or else you need to start creating the yaml DSL to define conditional logic (please no)**
  - google_cloud_platform.node_groups:
    - gpu-tesla-t4-x1:
        instance: "n1-standard-16"
        min_nodes: 0
        max_nodes: 4
        guest_accelerators:
          - name: nvidia-tesla-t4      # 1x 16 GB GDDR6: Nividia Tesla T4
            count: 1
  - profiles:
    - jupyterlab:
      - display_name: T4 GPU Instance 2x
        description: 16 cpu / 60GB RAM / 2 Nvidia T4 GPU (32 GB GPU RAM)
        kubespawner_override:
          image: quay.io/nebari/nebari-jupyterlab-gpu:2024.5.1
          cpu_limit: 16
          ...
          node_selector:
            "cloud.google.com/gke-nodepool": "gpu-tesla-t4-x2"
  - conda_store.extra_settings.CondaStore:
      conda_allowed_channels: ["bioconda", "conda-forge"]
conda_environments:
  - https://github.com/nebari-dev/jhub-apps-from-git-repo-example/blob/main/conda-project.yml
  - PAT_ENV_VAR: MY_GITHUB_PAT
    https://$MY_GITHUB_PAT:github.com/nebari-dev/jhub-apps/blob/main/environment-dev.yml
jhub_apps:
  - https://github.com/myOrg/myRepo/tree/myBranch
workflows:
  - condition: # e.g. check if file exists on NFS drive
    argo:
      # ARGO_WORKFLOW_YAML_SPEC
  - condition: # e.g. check if file exists on NFS drive
    cron:
      # K8S_JOB_SPEC
  - condition:
    conda_project:
      path:  # url or path
      command: 

# **Con: we have to figure out what package manager they're using (pip, conda, poetry, etc?) and install it the same way or make it configurable**
nebari_plugins:
  - nebari-plugin-mlflow
  - https://github.com/myOrg/myNebariPlugin

The 3 major cons I've noted in the yaml file are:

1. Putting supported nebari version dependencies in the yaml file is more clumsy than just using pyproject.yaml
2. Imagine making a gpu distribution, you need a separate distribution for each cloud provider or else you need to start creating the yaml DSL to define (please no)
3. If your distribution requires other plugins then we have to figure out how to install those for the user in the same manner as Nebari which is troublesome since many options for installing python packages exist.

I think we should define nebari distributions in plugins b/c of those limitations of the yaml file approach.