test: Enable running the end to end tests on K8S clusters other than Kind

[shmuelk]

This PR enables the running of the End to End tests on clusters other than kind.

If the environment variable K8S_CONTEXT is set the end to end test code will not create a kind cluster for the test. Instead the cluster pointed to by the specified context in the current Kubernetes Config will be used. All created Kubernetes objects will be deleted at the end of the test. Additionally kubectl port-forward will be started to enable the tests to be run from "outside" the target cluster. The user running the tests, needs to be a cluster wide admin for the cluster in question.

In addition the following environment variables can "configure" the tests if needed:

• E2E_PORT - the local port used to send the inference requests to.
• NAMESPACE - the Kubernetes namespace to create all of the namespaced objects in.
• EPP_IMAGE - the full image name to be used for the llm-d-inference-scheduler
• SIDECAR_IMAGE - the full image name for the llm-d-routing-sidecar
• VLLM_SIMULATOR_IMAGE - the full name for the llm-d-inference-sim

[hdefazio]

A few makefile targets have envsubst '$$PROJECT_NAME $$NAMESPACE $$IMAGE_TAG_BASE $$VERSION'
Do we need to remove $IMAGE_TAG_BASE and/or add the new image env vars to this?

[hdefazio]

scripts/kind-dev-env.sh and scripts/kubernetes-dev-env.sh can also be updated to use EPP_IMAGE /SIDECAR_IMAGE /VLLM_SIMULATOR_IMAGE

[shmuelk]

A few makefile targets have envsubst '$$PROJECT_NAME $$NAMESPACE $$IMAGE_TAG_BASE $$VERSION' Do we need to remove $IMAGE_TAG_BASE and/or add the new image env vars to this?

scripts/kind-dev-env.sh and scripts/kubernetes-dev-env.sh can also be updated to use EPP_IMAGE /SIDECAR_IMAGE /VLLM_SIMULATOR_IMAGE

Both have been fixed.

[elevran]

/lgtm
/approve
/hold (in case you want to remove the unrelated (?) changes - feel free to unhold and let it merge)

[elevran]

nit: this seems like an unrelated change?

[shmuelk]

You're fundamentally correct. But upstream, the function LoadConfig no longer exists causing lint to fail in the file test/config/prefix_cache_mode_test.go. I then searched for LoadConfig and updated all references.

[elevran]

nit: unrelated?

[shmuelk]

You're fundamentally correct. But upstream, the function LoadConfig no longer exists causing lint to fail.

[shmuelk]

/unhold