chore: metrics endpoint variables renamed from HTTP_SERVER->SYSTEM

[keivenchang]

Overview:

Enabled /live and /health (when DYN_SYSTEM_ENABLED=true), and renamed from "HTTP_SERVER_" to "SYSTEM_" names as to not get confused with client HTTP requests.

Details:

• Renamed constants: DYN_HTTP_SERVER_* → DYN_SYSTEM_*
• Updated method: http_server_enabled() → system_server_enabled()
• Updated all docs/comments to use "system server" terminology, removed mentions of "HTTP"
• Added tests.

Where should the reviewer start?

• lib/runtime/src/config.rs - Main changes
• lib/runtime/src/distributed.rs - Updated references

Related Issues: (use one of the action keywords Closes / Fixes / Resolves / Relates to)

• DIS-239

Summary by CodeRabbit

• New Features

  • Added a new /live endpoint to the system server for liveness checks.
  • Introduced a fallback route that returns a 404 status for unmatched paths.

• Refactor

  • Renamed HTTP server configuration and environment variables to use "system server" terminology.
  • Updated related documentation and log messages to reflect the new naming.
  • Improved server startup to provide clearer logging and asynchronous operation.

• Tests

  • Added tests for the new /live endpoint and fallback route behavior.

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[coderabbitai]

Walkthrough

The changes rename all HTTP server configuration and code references to "system server" throughout the runtime library, updating constants, struct fields, environment variables, and documentation accordingly. The HTTP server logic is refactored to use a new spawn_http_server function, which now spawns the server as a background task and returns its address and handle. Additional endpoints and improved logging are introduced. Tests and dependencies are updated to match the new naming and functionality.

Changes

File(s)	Change Summary
lib/runtime/Cargo.toml	Added reqwest as a dev-dependency with the "json" feature.
lib/runtime/src/config.rs	Renamed all HTTP server config fields, constants, env vars, and docs to "system server"; updated related tests.
lib/runtime/src/distributed.rs	Updated to use new system server config fields and system_server_enabled; refactored to use spawn_http_server.
lib/runtime/src/http_server.rs	Renamed and refactored server startup to spawn_http_server; added /live endpoint, fallback route, and improved logging; added async test for endpoints.

Sequence Diagram(s)

sequenceDiagram
    participant DistributedRuntime
    participant Config
    participant HTTPServer

    DistributedRuntime->>Config: Check system_server_enabled()
    alt Enabled
        DistributedRuntime->>HTTPServer: spawn_http_server(host, port, ...)
        HTTPServer->>DistributedRuntime: Return (address, join_handle)
        DistributedRuntime->>DistributedRuntime: Log server started
    else Disabled
        DistributedRuntime->>DistributedRuntime: Log server disabled
    end

Loading

Poem

A hop and a skip, the server’s renamed,
From HTTP to system, its purpose proclaimed.
New endpoints now live, with logging that sings,
Tests and configs updated—such wonderful things!
With spawn in its step, the server runs free,
A rabbit’s delight in pure clarity.
🐇✨

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

• Review comments: Directly reply to a review comment made by CodeRabbit. Example:
  • I pushed a fix in commit <commit_id>, please review it.
  • Explain this complex logic.
  • Open a follow-up GitHub issue for this discussion.
• Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
  • @coderabbitai explain this code block.
  • @coderabbitai modularize this function.
• PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
  • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
  • @coderabbitai read src/utils.ts and explain its main purpose.
  • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
  • @coderabbitai help me debug CodeRabbit configuration file.

Support

Need help? Create a ticket on our support page for assistance with any issues or questions.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

• @coderabbitai pause to pause the reviews on a PR.
• @coderabbitai resume to resume the paused reviews.
• @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
• @coderabbitai full review to do a full review from scratch and review all the files again.
• @coderabbitai summary to regenerate the summary of the PR.
• @coderabbitai generate sequence diagram to generate a sequence diagram of the changes in this PR.
• @coderabbitai resolve resolve all the CodeRabbit review comments.
• @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
• @coderabbitai help to get help.

Other keywords and placeholders

• Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
• Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
• Add @coderabbitai anywhere in the PR title to generate the title automatically.

Documentation and Community

• Visit our Documentation for detailed information on how to use CodeRabbit.
• Join our Discord Community to get help, request features, and share feedback.
• Follow us on X/Twitter for updates and announcements.

[coderabbitai]

Actionable comments posted: 2

FIXED: 🧹 Nitpick comments (1)

lib/runtime/src/config.rs (1)

22-22: Minor: Inconsistent terminology in comment.

The comment mentions "system-service" while the rest of the codebase uses "system server" terminology.

-    /// Grace shutdown period for the system-service.
+    /// Grace shutdown period for the system server.

📜 Review details

Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 9e76590 and f3ff3fe.

⛔ Files ignored due to path filters (1)

• Cargo.lock is excluded by !**/*.lock

📒 Files selected for processing (4)

• lib/runtime/Cargo.toml (1 hunks)
• lib/runtime/src/config.rs (7 hunks)
• lib/runtime/src/distributed.rs (1 hunks)
• lib/runtime/src/http_server.rs (2 hunks)

🧰 Additional context used

🧠 Learnings (3)

lib/runtime/src/distributed.rs (1)

Learnt from: ryanolson
PR: ai-dynamo/dynamo#1919
File: lib/runtime/src/engine.rs:168-168
Timestamp: 2025-07-14T21:25:56.898Z
Learning: The AsyncEngineContextProvider trait in lib/runtime/src/engine.rs was intentionally changed from `Send + Sync + Debug` to `Send + Debug` because the Sync bound was overly constraining. The trait should only require Send + Debug as designed.

lib/runtime/src/http_server.rs (2)

Learnt from: ryanolson
PR: ai-dynamo/dynamo#1919
File: lib/runtime/src/engine.rs:168-168
Timestamp: 2025-07-14T21:25:56.898Z
Learning: The AsyncEngineContextProvider trait in lib/runtime/src/engine.rs was intentionally changed from `Send + Sync + Debug` to `Send + Debug` because the Sync bound was overly constraining. The trait should only require Send + Debug as designed.

Learnt from: oandreeva-nv
PR: ai-dynamo/dynamo#1195
File: lib/llm/tests/block_manager.rs:150-152
Timestamp: 2025-06-02T19:37:27.666Z
Learning: In Rust/Tokio applications, when background tasks use channels for communication, dropping the sender automatically signals task termination when the receiver gets `None`. The `start_batching_publisher` function in `lib/llm/tests/block_manager.rs` demonstrates this pattern: when the `KVBMDynamoRuntimeComponent` is dropped, its `batch_tx` sender is dropped, causing `rx.recv()` to return `None`, which triggers cleanup and task termination.

lib/runtime/src/config.rs (1)

Learnt from: ryanolson
PR: ai-dynamo/dynamo#1919
File: lib/runtime/src/engine.rs:168-168
Timestamp: 2025-07-14T21:25:56.898Z
Learning: The AsyncEngineContextProvider trait in lib/runtime/src/engine.rs was intentionally changed from `Send + Sync + Debug` to `Send + Debug` because the Sync bound was overly constraining. The trait should only require Send + Debug as designed.

🔇 Additional comments (5)

lib/runtime/src/distributed.rs (1)

81-103: LGTM! Server configuration and startup logic properly updated.

The changes correctly implement the renaming from HTTP server to system server, including:

• Updated method calls and configuration field names
• Improved async server management with the new spawn_http_server signature
• Appropriate error handling and logging

lib/runtime/src/http_server.rs (3)

73-137: Well-structured refactoring to async server spawning.

The changes improve the HTTP server implementation:

• Proper async task spawning pattern
• Better error handling and logging
• Returns both address and handle for improved control

---

92-97: Good addition of the /live endpoint.

Using the same handler for both /health and /live endpoints is a common pattern for Kubernetes readiness and liveness probes.

---

260-304: Comprehensive test coverage for HTTP endpoints.

The test properly validates:

• All endpoint responses (/health, /live, and 404 fallback)
• Server lifecycle management
• Graceful shutdown handling

lib/runtime/src/config.rs (1)

128-144: Good implementation of environment variable mapping.

The mapping logic correctly transforms DYN_SYSTEM_* environment variables to the appropriate struct fields.

[nnshah1]

@grahamking for viz

[ZichengMa]

May I know why we rename HTTP to System? System seems to be a bigger word.

[keivenchang]

@ZichengMa this was discussed in a meeting today, where people may confuse your DYN_HTTP_SERVER_* with the actual HTTP server that's used on the HTTP frontend. So to separate them out, we looked into: DYN_METRICS_AND_STATS_SERVER_THAT_IS_NOT_USED_IN_THE_FRONTEND and combinations of, and finally decided on DYN_SYSTEM_{PORT,HOST,ENABLED}.

[grahamking]

warning: patch for the non root package will be ignored, specify patch at the workspace root

[grahamking]

I'm trying to avoid us having a dependency on OpenSSL. Fix is here: #1945

[grahamking]

---- http_server::tests::test_spawn_http_server_endpoints stdout ----

thread 'http_server::tests::test_spawn_http_server_endpoints' panicked at /home/runner/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.46.0/src/runtime/blocking/shutdown.rs:51:21:
Cannot drop a runtime in a context where blocking is not allowed. This happens when a runtime is dropped from within an asynchronous context.
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

Could you take a look?