fix(frontend): prepend KServe batch dim for Triton metadata

[Sumit884-byte]

Summary

• Translate Triton ModelConfig.dims to KServe v2 ModelMetadata shape by prepending a variable batch dim (-1) when max_batch_size > 0, matching native Triton's KServe adapter.
• Shared helper kserve_metadata_shape() used at both Triton input and output metadata call sites in model_metadata().
• Completes the same TensorMetadata mapping as the datatype pass-through: KServe gRPC ModelMetadata leaks internal model_config.proto enum names instead of OIP wire datatype strings #14520 → fix(llm): report KServe datatypes by their wire names, not protobuf variants #14957 → dynamo.frontend Config::Triton metadata drops max_batch_size, KServe v2 clients get shape mismatch on batchable models #14969.

Fixes #14969

Validation

• pre-commit run --files lib/llm/src/grpc/service/kserve.rs lib/llm/src/grpc/service/tensor.rs
• cargo fmt -p dynamo-llm -- --check
• cargo clippy --no-default-features -p dynamo-llm --lib -- -W clippy::all
• cargo test -p dynamo-llm --no-default-features kserve_metadata_shape — 2 passed
• Live KServe v2 / perf_analyzer against a Dynamo-registered Triton model with max_batch_size > 0 — not done

Test plan

• Confirm CI dynamo-llm jobs pass
• Confirm ModelMetadata for a Triton model with max_batch_size > 0 reports shape with a leading -1
• Confirm max_batch_size == 0 still reports dims unchanged

Summary by CodeRabbit

• Bug Fixes
  • Corrected model metadata to accurately report tensor shapes for both batched and unbatched models.
  • Empty tensor dimensions are now handled consistently in metadata responses.

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[github-actions]

👋 Hi Sumit884-byte! Thank you for contributing to ai-dynamo/dynamo.

Just a reminder: The NVIDIA Test Github Validation CI runs an essential subset of the testing framework to quickly catch errors.Your PR reviewers may elect to test the changes comprehensively before approving your changes.

🚀

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 2679d680-61d1-4d3a-8f3c-41ee56d43891

📥 Commits

Reviewing files that changed from the base of the PR and between b5871f3 and ffa318e.

📒 Files selected for processing (2)

• lib/llm/src/grpc/service/kserve.rs
• lib/llm/src/grpc/service/tensor.rs

Included review availability: Your plan provides up to 12 included reviews per hour; 11 remain after this review.

---

Walkthrough

The change adds batch-aware tensor shape construction and applies it to Triton input and output metadata in the KServe service. Tests cover batched, unbatched, empty, and multidimensional shapes.

Changes

KServe shape metadata

Layer / File(s)	Summary
Batch-aware shape helper lib/llm/src/grpc/service/tensor.rs	Adds kserve_metadata_shape, which prepends -1 when max_batch_size > 0 and preserves dimensions otherwise. Tests cover batched, unbatched, empty, and multidimensional shapes.
KServe metadata integration lib/llm/src/grpc/service/kserve.rs	Uses kserve_metadata_shape for Triton input and output metadata shapes, passing each tensor’s dimensions and max_batch_size.

Priority: ➖ Normal

Estimated code review effort: 2 (Simple) | ~15 minutes

Severity of issue fixed: Medium

Merge Risk: ⚪ Minimal · up to ffa31

KServe clients should receive correct tensor shapes for both batched and unbatched Triton models, with no unresolved merge-blocking risk identified.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 71.43% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 7 functions across 2 files.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly identifies the main change: adding the KServe batch dimension to Triton metadata.
Description check	✅ Passed	The description explains the change, validation, test plan, and linked issue. It does not include a dedicated “Where should the reviewer start?” section, but the missing section is non-critical becaus…
Linked Issues check	✅ Passed	Issue #14969 requires KServe metadata to prepend -1 when max_batch_size > 0, preserve dims when batching is disabled, and apply the correction to inputs and outputs. kserve_metadata_shape impl…
Out of Scope Changes check	✅ Passed	The changes are limited to the Triton metadata shape helper, its use in input and output metadata mapping, and focused unit tests. These changes directly support issue #14969 and introduce no unrelate…

• Fix all pre-merge checks with AI

---

_{Comment @coderabbitai help to get the list of available commands.}

[Sumit884-byte]

Hi @ai-dynamo/dynamo-frontend-codeowners — this PR is waiting on a code owner review from this team to merge.

It fixes #14969: KServe ModelMetadata now prepends a variable batch dim (-1) when Triton max_batch_size > 0, matching native Triton. Would appreciate a look when you have a chance — happy to address any feedback.

[rmccorm4]

/ok to test c4467a9