endpoint.update() removes environment variables of endpoint

[MoritzLaurer]

Describe the bug

I have an endpoint (e.g. TGI llama3.1 endpoint) with custom environment variables and secrets. When I update the endpoint, e.g. to get a new revision of the model, the model updates correctly, but the environment variables are lost.

The expected behaviour would be: Only the thing I'm specifying in endpoint.update (in this case, the model version from revision) should change, while all the rest of the endpoint is not updated (i.e. the env variables remain the same). Interestingly enough: secrets are not deleted, while env variables are deleted.

Reproduction

Create endpoint via endpoint interface and manually add env variable and secret.

namespace = "MoritzLaurer" 
endpoint_name = "meta-llama-3-8b-instruct-001"

all_endpoints = list_inference_endpoints(namespace=namespace)
if endpoint_name in [endpoint.name for endpoint in all_endpoints]: 
    endpoint = get_inference_endpoint(endpoint_name, namespace=namespace)
    endpoint.update(repository=endpoint.repository, revision="c4a54320a52ed5f88b7a2f84496903ea4ff07b45")

The update endpoint does not contain the environment variable anymore.

The issue can be circumvented with huggingface_hub.HfApi by passing the env variables again via custom_image.

from huggingface_hub import HfApi
api = HfApi()

namespace = "MoritzLaurer"
endpoint_name = "meta-llama-3-8b-instruct-001"

api.update_inference_endpoint(
    name = endpoint_name, 
    custom_image = {
        "health_route": "/health",
        "env": {
            "TEST_ENV": "True",
            "TEST_ENV2": "False",
            "MODEL_ID": "/repository"
        },
        "url": "ghcr.io/huggingface/text-generation-inference:latest",
    },
    namespace = namespace
)

In this case the env variables are preserved.

I think user's expected behaviour is that endpoint.update() preserves environment variables automatically (a customer reported this). If this is not possible, it would be good to add support for the custom_image argument in endpoint.update() and document explicitly what gets lost if not passed to the method.

Could you have a look at this @Wauplin ?

Logs

No response

System info

Jupyterlab Space

- huggingface_hub version: 0.24.6
- Platform: Linux-5.10.205-195.807.amzn2.x86_64-x86_64-with-glibc2.31
- Python version: 3.9.5
- Running in iPython ?: Yes
- iPython shell: ZMQInteractiveShell
- Running in notebook ?: Yes
- Running in Google Colab ?: No
- Token path ?: /home/user/.cache/huggingface/token
- Has saved token ?: True
- Who am I ?: MoritzLaurer
- Configured git credential helpers: 
- FastAI: N/A
- Tensorflow: N/A
- Torch: N/A
- Jinja2: 3.1.4
- Graphviz: N/A
- keras: N/A
- Pydot: N/A
- Pillow: N/A
- hf_transfer: N/A
- gradio: N/A
- tensorboard: N/A
- numpy: N/A
- pydantic: N/A
- aiohttp: N/A
- ENDPOINT: https://huggingface.co
- HF_HUB_CACHE: /home/user/.cache/huggingface/hub
- HF_ASSETS_CACHE: /home/user/.cache/huggingface/assets
- HF_TOKEN_PATH: /home/user/.cache/huggingface/token
- HF_HUB_OFFLINE: False
- HF_HUB_DISABLE_TELEMETRY: False
- HF_HUB_DISABLE_PROGRESS_BARS: None
- HF_HUB_DISABLE_SYMLINKS_WARNING: False
- HF_HUB_DISABLE_EXPERIMENTAL_WARNING: False
- HF_HUB_DISABLE_IMPLICIT_TOKEN: False
- HF_HUB_ENABLE_HF_TRANSFER: False
- HF_HUB_ETAG_TIMEOUT: 10
- HF_HUB_DOWNLOAD_TIMEOUT: 10

{'huggingface_hub version': '0.24.6',
 'Platform': 'Linux-5.10.205-195.807.amzn2.x86_64-x86_64-with-glibc2.31',
 'Python version': '3.9.5',
 'Running in iPython ?': 'Yes',
 'iPython shell': 'ZMQInteractiveShell',
 'Running in notebook ?': 'Yes',
 'Running in Google Colab ?': 'No',
 'Token path ?': '/home/user/.cache/huggingface/token',
 'Has saved token ?': True,
 'Who am I ?': 'MoritzLaurer',
 'Configured git credential helpers': '',
 'FastAI': 'N/A',
 'Tensorflow': 'N/A',
 'Torch': 'N/A',
 'Jinja2': '3.1.4',
 'Graphviz': 'N/A',
 'keras': 'N/A',
 'Pydot': 'N/A',
 'Pillow': 'N/A',
 'hf_transfer': 'N/A',
 'gradio': 'N/A',
 'tensorboard': 'N/A',
 'numpy': 'N/A',
 'pydantic': 'N/A',
 'aiohttp': 'N/A',
 'ENDPOINT': 'https://huggingface.co',
 'HF_HUB_CACHE': '/home/user/.cache/huggingface/hub',
 'HF_ASSETS_CACHE': '/home/user/.cache/huggingface/assets',
 'HF_TOKEN_PATH': '/home/user/.cache/huggingface/token',
 'HF_HUB_OFFLINE': False,
 'HF_HUB_DISABLE_TELEMETRY': False,
 'HF_HUB_DISABLE_PROGRESS_BARS': None,
 'HF_HUB_DISABLE_SYMLINKS_WARNING': False,
 'HF_HUB_DISABLE_EXPERIMENTAL_WARNING': False,
 'HF_HUB_DISABLE_IMPLICIT_TOKEN': False,
 'HF_HUB_ENABLE_HF_TRANSFER': False,
 'HF_HUB_ETAG_TIMEOUT': 10,
 'HF_HUB_DOWNLOAD_TIMEOUT': 10}

[Wauplin]

Thanks for reporting the issue @MoritzLaurer! The problem comes from some None values being passed by default when another field gets updated. This is now fixed by #2476. I have been able to reproduce the error and can confirm it now works as expected.

[MoritzLaurer]

Thanks for the quick fix @Wauplin !

What do you think about adding the custom_image argument to endpoint.update()? In case users want to update the entire image (I suppose that's a separate feature request)

[Wauplin]

Hi @MoritzLaurer this is indeed a separate feature request but luckily it has already been implemented: #2306 😃 You should be able to use it in 0.24.x release.

[MoritzLaurer]

Hi @MoritzLaurer this is indeed a separate feature request but luckily it has already been implemented: #2306 😃 You should be able to use it in 0.24.x release.

What I meant is the endpoint.update() alias around HfApi.update_inference_endpoint(). custom_image can currently not be passed to endpoint.update(), right (v0.24.6 docs)? So people need to directly use HfApi.update_inference_endpoint() to pass a custom_image, or do I misunderstand something? (the parameters of the two methods are not exactly the same. endpoint.update also doesn't seem to have token in the docs) @Wauplin

[Wauplin]

Correct yes! Sorry about that 🙈 Fixed it in #2477