From d10048d00bb29ca9ba4cebd3417e5ebc8d09eb5d Mon Sep 17 00:00:00 2001 From: kiersten-stokes Date: Tue, 18 Mar 2025 13:18:48 -0500 Subject: [PATCH] Add further details for Kserve support in LMEval --- docs/modules/ROOT/pages/lm-eval-tutorial.adoc | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/docs/modules/ROOT/pages/lm-eval-tutorial.adoc b/docs/modules/ROOT/pages/lm-eval-tutorial.adoc index f3f9733..c6c379a 100644 --- a/docs/modules/ROOT/pages/lm-eval-tutorial.adoc +++ b/docs/modules/ROOT/pages/lm-eval-tutorial.adoc @@ -667,6 +667,11 @@ In the case where both managed and existing PVCs are referenced in `outputs`, th === Using an `InferenceService` +[NOTE] +==== +When using Kserve `InferenceServices`, only HuggingFace- and VLLM-based model serving runtimes are supported with the following method. Additionally, ensure DNS is set up for your cluster, as there is currently no method to pass additional request headers to a served model in `lm-evaluation-harness`. +==== + [NOTE] ==== This example assumes vLLM model already deployed in your cluster.