feat: changed to support both v1 and v1a2 ip in EPP #1277

capri-xiyue · 2025-07-31T22:44:26Z

fixed #1278
we want to support CUJ (use case) that users can specify either "inference.networking.k8s.io" or "inference.networking.x-k8s.io" InferencePool in EPP. Please note EPP deployment and InferencePool still has 1:1 mapping.

Added --pool-group to allow users to specify whether they configure either inference.networking.x-k8s.io or http://inference.networking.k8s.io InferencePool
In reconciler, will watch either inference.networking.x-k8s.io or http://inference.networking.k8s.io InferencePool
In datastore logic, will convert v1alph2.InferencePool to v1.InferencePool as the spec and status has no change

netlify · 2025-07-31T22:44:32Z

✅ Deploy Preview for gateway-api-inference-extension ready!

Name	Link
🔨 Latest commit	`d823af9`
🔍 Latest deploy log	https://app.netlify.com/projects/gateway-api-inference-extension/deploys/689260b6c3af530009aabf02
😎 Deploy Preview	https://deploy-preview-1277--gateway-api-inference-extension.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

k8s-ci-robot · 2025-07-31T22:44:36Z

Hi @capri-xiyue. Thanks for your PR.

I'm waiting for a kubernetes-sigs member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

capri-xiyue · 2025-07-31T23:23:42Z

Please don't review it as it is just a draft

nirrozenbaum · 2025-08-01T07:32:31Z

didn't start the review. just verifying:
are we in agreement that we should keep both InfPool CRDs v1 and v1alpha2 to let the various gateways some time to adjust their code and pass conformance using v1 before we deprecate v1alpha2?

I think this is the right thing to do!
cc: @robscott @ahg-g @kfswain @danehans

pierDipi · 2025-08-01T08:12:20Z

As an additional data point, supporting both for some time will make our life a lot easier, thanks for doing this!

kfswain · 2025-08-01T14:52:11Z

👍 I'll review when @capri-xiyue pulls off the WIP tag! Thanks for your work on this! We will give you space to work for now, feel free to pull of that WIP tag when ready!

kfswain · 2025-08-01T14:54:19Z

WRT the timeline, this is intended to be transitionary & give gateways time to migrate to v1. It wont be immediate, but this is not intended to be indefinite. We will work with our upstream partners but I'm thinking deprecation of v1a2 would probably be in the v1.1-v1.2 timeline

kfswain · 2025-08-01T18:53:20Z

/ok-to-test

capri-xiyue · 2025-08-01T21:15:47Z

I feel it is too much to refactor e2e and integration test here. Will create another issue to track it. #1283

I added basic UT here to make sure it works and I also verified it e2e via manual test

# Conflicts: # pkg/epp/controller/inferencemodel_reconciler.go # pkg/epp/datastore/datastore.go # pkg/epp/server/controller_manager.go # pkg/epp/server/runserver.go # Conflicts: # cmd/epp/runner/runner.go

Signed-off-by: Xiyue Yu <[email protected]>

capri-xiyue · 2025-08-04T16:47:06Z

pkg/epp/server/runserver.go

 	DefaultCertPath                                 = ""                                // default for --cert-path
 	DefaultConfigFile                               = ""                                // default for --config-file
 	DefaultConfigText                               = ""                                // default for --config-text
+	DefaultPoolGroup                                = "inference.networking.k8s.io"     // default for --pool-group


I'm wondering maybe I should use "inference.networking.x-k8s.io" to avoid unexpected break change when users change from release 0.5 to main branch?

I personally think it's safer to switch the defaults to the new GA API and leave an option to use the older alpha API for backwards compatibility.

kfswain · 2025-08-05T18:56:20Z

pkg/epp/controller/inferencepool_reconciler.go

-			logger.Info("InferencePool not found. Clearing the datastore")
+	if c.PoolGKNN.Group == v1alpha2.GroupName {
+		infPool := &v1alpha2.InferencePool{}
+		if err := c.Get(ctx, req.NamespacedName, infPool); err != nil {


can we combine this logic so we dont repeat? The only conditionals should be:

creation of pool based on type

conversion of v1a2 pool to v1 (do we need to unstructured middle step? would a conversion function that we define be more reliable?)

LMKWYT

Updated to avoid duplicate code. To avoid duplicate code, I have to initialize the variable as a interface which both v1a2 and v1 Pool type can use, here I use client.Object.

I think using unstructured as middle step is quite reliable as the conversation function is handled by k8s runtime instead of self-authored written code. And self-written conversion code would be cumbersome as I need to self-write deep copy code by myself. Let me know what you think.

Sounds great, and looks great as well. Thanks!

kfswain · 2025-08-05T18:56:55Z

Looks good for the most part, left a single comment in the primary change (infPool reconciliation)

capri-xiyue · 2025-08-05T19:57:22Z

Looks good for the most part, left a single comment in the primary change (infPool reconciliation)

Updated the code based on the comment

kfswain · 2025-08-05T20:20:34Z

/lgtm
/approve

k8s-ci-robot · 2025-08-05T20:20:42Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: capri-xiyue, kfswain

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

~~OWNERS~~ [kfswain]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

k8s-ci-robot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jul 31, 2025

k8s-ci-robot added the cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. label Jul 31, 2025

k8s-ci-robot requested review from ahg-g and danehans July 31, 2025 22:44

k8s-ci-robot added the needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. label Jul 31, 2025

k8s-ci-robot added the size/L Denotes a PR that changes 100-499 lines, ignoring generated files. label Jul 31, 2025

k8s-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Aug 1, 2025

capri-xiyue force-pushed the capri-xiyue/epp-support-both branch from 9cec6c5 to d4a8d67 Compare August 1, 2025 16:17

k8s-ci-robot added ok-to-test Indicates a non-member PR verified by an org member that is safe to test. and removed needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Aug 1, 2025

capri-xiyue force-pushed the capri-xiyue/epp-support-both branch from 288e1b3 to 43727d1 Compare August 1, 2025 19:13

capri-xiyue changed the title ~~[WIP] changed to support both v1 and v1a2 ip~~ [WIP] changed to support both v1 and v1a2 ip in EPP Aug 1, 2025

capri-xiyue changed the title ~~[WIP] changed to support both v1 and v1a2 ip in EPP~~ feat: changed to support both v1 and v1a2 ip in EPP Aug 1, 2025

k8s-ci-robot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Aug 1, 2025

k8s-ci-robot assigned robscott Aug 1, 2025

k8s-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Aug 4, 2025

capri-xiyue added 12 commits August 4, 2025 09:42

changed to support both v1 and v1a2 ip

4425be7

# Conflicts: # pkg/epp/controller/inferencemodel_reconciler.go # pkg/epp/datastore/datastore.go # pkg/epp/server/controller_manager.go # pkg/epp/server/runserver.go # Conflicts: # cmd/epp/runner/runner.go

rebase with main

ce810ee

support both v1 and v1a2 IP

97fe365

change import order

899e22e

fixed imports

ddf7ffd

fixed pipeline

1164c2f

fixed comments

6e1d507

Signed-off-by: Xiyue Yu <[email protected]>

fixed merge failure

47cc628

fixed missing arguments

1ff3cbe

Signed-off-by: Xiyue Yu <[email protected]>

fixed boilplate

2b3e66d

pass verify

8c90f07

rebase main

3c84a17

capri-xiyue force-pushed the capri-xiyue/epp-support-both branch from 76b97a9 to 3c84a17 Compare August 4, 2025 16:45

k8s-ci-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Aug 4, 2025

capri-xiyue commented Aug 4, 2025

View reviewed changes

capri-xiyue requested a review from robscott August 4, 2025 16:59

kfswain reviewed Aug 5, 2025

View reviewed changes

changed to avoid duplicate code

0fe9223

capri-xiyue requested a review from kfswain August 5, 2025 19:49

changed logger info

d823af9

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Aug 5, 2025

k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Aug 5, 2025

k8s-ci-robot merged commit 115aa85 into kubernetes-sigs:main Aug 5, 2025
9 checks passed

nicolexin mentioned this pull request Aug 5, 2025

Filter inference objectives based on inference pool group #1306

Merged

feat: changed to support both v1 and v1a2 ip in EPP #1277

feat: changed to support both v1 and v1a2 ip in EPP #1277

Uh oh!

Conversation

capri-xiyue commented Jul 31, 2025 • edited by danehans Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

netlify bot commented Jul 31, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✅ Deploy Preview for gateway-api-inference-extension ready!

Uh oh!

k8s-ci-robot commented Jul 31, 2025

Uh oh!

capri-xiyue commented Jul 31, 2025

Uh oh!

nirrozenbaum commented Aug 1, 2025

Uh oh!

pierDipi commented Aug 1, 2025

Uh oh!

kfswain commented Aug 1, 2025

Uh oh!

kfswain commented Aug 1, 2025

Uh oh!

kfswain commented Aug 1, 2025

Uh oh!

capri-xiyue commented Aug 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

capri-xiyue Aug 4, 2025

Choose a reason for hiding this comment

Uh oh!

robscott Aug 4, 2025

Choose a reason for hiding this comment

Uh oh!

kfswain Aug 5, 2025

Choose a reason for hiding this comment

Uh oh!

capri-xiyue Aug 5, 2025

Choose a reason for hiding this comment

Uh oh!

kfswain Aug 5, 2025

Choose a reason for hiding this comment

Uh oh!

kfswain commented Aug 5, 2025

Uh oh!

capri-xiyue commented Aug 5, 2025

Uh oh!

kfswain commented Aug 5, 2025

Uh oh!

k8s-ci-robot commented Aug 5, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

capri-xiyue commented Jul 31, 2025 •

edited by danehans

Loading

netlify bot commented Jul 31, 2025 •

edited

Loading

capri-xiyue commented Aug 1, 2025 •

edited

Loading