Skip to content

Conversation

@danehans
Copy link
Contributor

@danehans danehans commented May 7, 2025

Running make test-e2e results in:

  STEP: Verifying connectivity through the inference extension @ 05/07/25 08:09:31.18
  [FAILED] in [It] - /Users/solo-system-dhansen/go/src/sigs.k8s.io/gateway-api-inference-extension/test/e2e/epp/e2e_test.go:89 @ 05/07/25 08:12:31.183
  STEP: Deleting the InferenceModel test resource. @ 05/07/25 08:12:31.183
• [FAILED] [180.340 seconds]
InferencePool when The Inference Extension is running [It] Should route traffic to target model servers
/Users/solo-system-dhansen/go/src/sigs.k8s.io/gateway-api-inference-extension/test/e2e/epp/e2e_test.go:47

  [FAILED] Timed out after 180.001s.
  Expected success, but got an error:
      <*errors.errorString | 0x14000a84730>:
      did not get 200 OK: HTTP/1.1 404 Not Found
      date: Wed, 07 May 2025 15:12:26 GMT
      server: uvicorn
      content-type: application/json
      x-went-into-resp-headers: true
      transfer-encoding: chunked

      {"code":404,"message":"The model `cad-fabricator` does not exist.","object":"error","param":null,"type":"NotFoundError"}  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                       Dload  Upload   Total   Spent    Left  Speed
100   239    0   120  100   119  16982  16841 --:--:-- --:--:-- --:--:-- 39833

      {
          s: "did not get 200 OK: HTTP/1.1 404 Not Found\r\ndate: Wed, 07 May 2025 15:12:26 GMT\r\nserver: uvicorn\r\ncontent-type: application/json\r\nx-went-into-resp-headers: true\r\ntransfer-encoding: chunked\r\n\r\n{\"code\":404,\"message\":\"The model `cad-fabricator` does not exist.\",\"object\":\"error\",\"param\":null,\"type\":\"NotFoundError\"}  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current\n                                 Dload  Upload   Total   Spent    Left  Speed\n\r  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0\r100   239    0   120  100   119  16982  16841 --:--:-- --:--:-- --:--:-- 39833\n",
      }
  In [It] at: /Users/solo-system-dhansen/go/src/sigs.k8s.io/gateway-api-inference-extension/test/e2e/epp/e2e_test.go:89 @ 05/07/25 08:12:31.183
...

cad-fabricator is an invalid target model name. Based on the config/manifests/vllm/gpu-deployment.yaml manifest, food-review-1is the only configured LoRA adapter.

Signed-off-by: Daneyon Hansen <[email protected]>
@netlify
Copy link

netlify bot commented May 7, 2025

Deploy Preview for gateway-api-inference-extension ready!

Name Link
🔨 Latest commit 29f3565
🔍 Latest deploy log https://app.netlify.com/sites/gateway-api-inference-extension/deploys/681b8164d6b11a0008201723
😎 Deploy Preview https://deploy-preview-793--gateway-api-inference-extension.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

@k8s-ci-robot k8s-ci-robot added the cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. label May 7, 2025
@k8s-ci-robot k8s-ci-robot requested a review from kfswain May 7, 2025 15:51
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: danehans

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot requested a review from nirrozenbaum May 7, 2025 15:51
@k8s-ci-robot k8s-ci-robot added approved Indicates a PR has been approved by an approver from all required OWNERS files. size/S Denotes a PR that changes 10-29 lines, ignoring generated files. labels May 7, 2025
@danehans
Copy link
Contributor Author

danehans commented May 7, 2025

In v0.1.0, e2e tested LoRA-based load balancing (xref), but the reference manifests have since changed to use a single targetModelName. @kfswain @ahg-g do you have additional context for this change?

@ahg-g
Copy link
Contributor

ahg-g commented May 7, 2025

/lgtm

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label May 7, 2025
@k8s-ci-robot k8s-ci-robot merged commit b4cb728 into kubernetes-sigs:main May 7, 2025
8 checks passed
rlakhtakia pushed a commit to rlakhtakia/gateway-api-inference-extension that referenced this pull request May 13, 2025
nayihz pushed a commit to nayihz/gateway-api-inference-extension that referenced this pull request May 14, 2025
kaushikmitr pushed a commit to kaushikmitr/llm-instance-gateway that referenced this pull request May 15, 2025
rlakhtakia pushed a commit to rlakhtakia/gateway-api-inference-extension that referenced this pull request Jun 11, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. lgtm "Looks good to me", indicates that a PR is ready to be merged. size/S Denotes a PR that changes 10-29 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants