Skip to content

feat(router): add a Global Discovery HTTP service POC - #16714

Closed
atchernych wants to merge 1 commit into
codex/global-router-long-contextfrom
atchernych/global-discovery-http-service
Closed

atchernych wants to merge 1 commit into
codex/global-router-long-contextfrom
atchernych/global-discovery-http-service

Conversation

@atchernych

@atchernych atchernych commented Oct 17, 2026 •

Copy link
Copy Markdown
Contributor

Summary

This is the Karmada Specific implementation. This will serve as a feed into the more generic Kubernetes implementation in #15949.

Fixes DEP-1290

POC of a Global Discovery HTTP service for the multi-cluster Global Router. Instead of a static pool list, the router can learn its pools from Kubernetes multi-cluster services (MCS) imported by Karmada.

  • Runs in the hub cluster. It watches the EndpointSlices that Karmada's EndpointSlice controller imports and serves the KV DC Relay pools as JSON on GET /pools. GET /healthz turns ready after the first full list.
  • A Relay exported as Service relay-<dgd> maps to pool (<cluster>, <namespace>, <dgd>). Karmada adds no source-cluster label, so the cluster comes from the imported slice name (imported-<cluster>-<service>-<suffix>).
  • One index entry per slice, updated per watch event and replaced as a whole on relist. The watch uses kube-rs backoff.
  • Behind the new global-discovery-poc feature of dynamo-llm, so default builds are unchanged.
  • Adds container/Dockerfile.global-discovery-poc and container/global-discovery-poc.yaml (read-only EndpointSlice access, Deployment, Service).

Stacked on #15579. It is a working stand-in for the Kubernetes PoolDirectory sketched in #15949; the plan is to move this logic behind that trait later.

Not in this PR: the router reading /pools.

Validation

  • cargo test -p dynamo-llm --no-default-features --features global-discovery-poc --bin global-discovery-poc: 9 tests pass (site parsing, non-Relay services skipped, not-ready pools, IPv6 URLs, delete, relist).
  • cargo clippy on the binary: no warnings in the new file. cargo fmt --check: clean.
  • Manually, on two EKS clusters joined to Karmada in Pull mode: a Relay exported from one cluster appears in the hub as derived-relay-<dgd> with an imported-<cluster>-... slice, which is the input this service reads. The service itself is not yet deployed there.

🤖 Generated with Claude Code

Serve the KV DC Relay pools found in Karmada-imported EndpointSlices as
JSON on GET /pools, so the Global Router can learn its pools from
multi-cluster services instead of a static file.

The service watches EndpointSlices labelled by Karmada's EndpointSlice
controller and keeps one index entry per slice, replaced as a whole on
relist. A Relay exported as `relay-<dgd>` maps to pool
(<cluster>, <namespace>, <dgd>); the cluster comes from the imported
slice name because Karmada adds no source-cluster label.

The binary sits behind the new `global-discovery-poc` feature, so
default builds are unchanged. A Dockerfile and a hub-cluster manifest
are included.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Signed-off-by: Anna Tchernych <atchernych@nvidia.com>
@devin-ai-integration

Copy link
Copy Markdown
Contributor

❌ Dynamo PR CI failed — run 38987872826 (attempt 1) on 77fb662dca

Gate checks: ❌ backend-status-check · ❌ deploy-status-check · ❌ dynamo-status-check

Other Jobs
changed-files ❌ 1

Failure details

1 job failed: changed-files rejects two new files that no CI filter covers, so every downstream job was skipped.

❌ changed-files: files not covered by any CI filter

Job: changed-files · Failed step: Check for changes · Logs: gh run view --job 117263440876 -R ai-dynamo/dynamo --log-failed

##[error]The following files are not covered by any CI filter:
container/Dockerfile.global-discovery-poc
container/global-discovery-poc.yaml

Please add these paths to .github/filters.yaml
See .github/FILTERS.md for documentation
##[error]Process completed with exit code 1.

The uncovered-files check failed because container/Dockerfile.global-discovery-poc and container/global-discovery-poc.yaml don't match any filter in .github/filters.yaml. Adding these paths to a filter (see .github/FILTERS.md) should fix it.

For agents
{"pr": 16714, "run_id": 38987872826, "run_attempt": 1, "head_sha": "77fb662dca4fb29d860d411c0973be57f99e155b",
 "failures": [{"job": "changed-files", "job_id": 117263440876, "failed_step": "Check for changes", "signature": "files not covered by any CI filter: container/Dockerfile.global-discovery-poc, container/global-discovery-poc.yaml", "tests": [], "log_cmd": "gh run view --job 117263440876 -R ai-dynamo/dynamo --log-failed"}]}

Posted automatically by Devin for run 38987872826. Updated on every full-CI run of this PR.

atchernych added a commit that referenced this pull request Oct 20, 2026
Fill in the router side of Kubernetes mode:

- mcs.rs: label selectors and naming rules for standard MCS and Karmada.
  Karmada's source cluster comes from the slice name
  `imported-<cluster>-<source slice>`. The `work.karmada.io/namespace`
  annotation names the source cluster only on the Karmada control plane;
  copies that Karmada propagates into members carry the destination.
- McsPoolDirectory: one watch feeds a shared cache. Each slice event
  updates only its pool, a relist rebuilds everything once, `get` reads
  the cache, and `list_and_watch` sends `Added` per pool, a `Resync`,
  then changes. A bound Relay address stays while it is Ready.
- TrustedNetworkAuthenticator, MeshAuthenticator, address_identity and
  McsPoolDiscovery::build.
- The body of pool_discovery::admit, with tests.

McsImplementation is no longer non_exhaustive, so dynamo-llm handles
every implementation. The Relay side (EndpointSliceReplicaDirectory)
stays a sketch.

The naming rules and the Karmada findings come from a test of the POC
service in #16714 on two EKS clusters joined to Karmada v1.19.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Signed-off-by: Anna Tchernych <atchernych@nvidia.com>
atchernych added a commit that referenced this pull request Oct 23, 2026
Adds the Global Discovery Service, an HTTP service that PoolRelays and
Global Routers call instead of reading Kubernetes themselves:

- POST /pools registers a pool. A PoolRelay calls it on a timer, so each
  call is also the pool's heartbeat.
- GET /routers lists the Global Router replicas a PoolRelay connects to.
- GET /pools lists pools for routers. GET /healthz is OK while the
  service can answer from its store, which for MCS means its cache.

The JSON matches the dynamo-global-discovery-client crate. The service
works through the pool_discovery traits, with two stores:

- memory (the default): registrations in this process, each expiring
  lease_duration after its last registration; router replicas come from
  --router.
- karmada and mcs: pools and router replicas from the EndpointSlices that
  MCS imports. Pools come from McsPoolDirectory, and router replicas from
  EndpointSliceReplicaDirectory, which this change implements. These
  stores take no registrations (501), because an MCS pool is announced by
  its ServiceExport.

Both EndpointSlice directories now share one cache (kubernetes/cache.rs)
and one watch-stream helper (stream.rs).

PoolRelay Services are named <DGD name><suffix>, with -pool-relay as the
default suffix. This replaces the relay-<DGD name> prefix used in #16714.

The binary needs the global-discovery feature:
cargo run -p dynamo-llm --features global-discovery --bin global-discovery

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Signed-off-by: Anna Tchernych <atchernych@nvidia.com>
@atchernych

Copy link
Copy Markdown
Contributor Author

Closing: superseded by the Global Discovery Service from #15949, which now lives in lib/global-discovery-service on global-router-v2 (3e9358c).

@atchernych atchernych closed this Oct 25, 2026
atchernych added a commit that referenced this pull request Oct 27, 2026
…l Discovery Service

Copied from the POC in #16714 and updated for this crate's binary:
- container/Dockerfile.global-discovery-service builds
  global-discovery-service from dynamo-global-discovery-service into a
  CPU-only Debian image, as the POC did for its own binary.
- container/global-discovery-service.yaml deploys it on the hub with the
  Karmada store, the router Service global-router and its port http, as
  the test deployment in #17278 does. It needs only read access to
  EndpointSlices. Logging is set with DYN_LOG, which the service reads,
  instead of RUST_LOG.

The Dockerfile is owned by the global router and ops teams, like other
component Dockerfiles, and the manifest by the global router team.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Anna Tchernych <atchernych@nvidia.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant