merge upstream

[overcuriousity]

No description provided.

[coderabbitai]

Caution

Review failed

The pull request is closed.

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 62bedcfd-472d-4ede-afb1-a6d3e5403c35

📥 Commits

Reviewing files that changed from the base of the PR and between 9559009 and e295d15.

⛔ Files ignored due to path filters (1)

• ui-svelte/package-lock.json is excluded by !**/package-lock.json

📒 Files selected for processing (22)

• Makefile
• README.md
• config-schema.json
• config.example.yaml
• config_embed.go
• docs/configuration.md
• llama-swap.go
• proxy/config/config.go
• proxy/config/config_test.go
• proxy/config/model_config.go
• proxy/process.go
• proxy/process_rpc_health_test.go
• proxy/process_test.go
• proxy/process_timeout_test.go
• proxy/processgroup.go
• proxy/processgroup_test.go
• proxy/proxymanager.go
• proxy/proxymanager_api.go
• ui-svelte/package.json
• ui-svelte/src/App.svelte
• ui-svelte/src/components/Header.svelte
• ui-svelte/src/routes/Config.svelte

---

Walkthrough

The pull request introduces request timeout enforcement for model inference, adds RPC health checking for distributed inference models with TCP endpoint monitoring, implements configuration API endpoints, replaces the React UI with a new Svelte-based UI with configuration editor, updates the Makefile to build the Svelte UI, and refactors configuration hot-reloading in the main application.

Changes

Cohort / File(s)	Summary
Build System & Config Metadata Makefile, config_embed.go	Switched UI build target from ui/ to ui-svelte/, updated node_modules caching. Added Go embed of config.example.yaml with public accessor function.
Configuration Schema & Documentation config-schema.json, config.example.yaml, docs/configuration.md, README.md	Added two new per-model config properties: requestTimeout (integer, enforces request deadline) and rpcHealthCheck (boolean, enables RPC endpoint health monitoring). Updated example configs and documentation accordingly.
Core App & Config Parsing llama-swap.go, proxy/config/config.go, proxy/config/config_test.go, proxy/config/model_config.go	Refactored config change event handling to always listen (not conditionally deferred), added ProxyManager config path/example setters. Implemented ParseRPCEndpoints() parser with validation for RPC endpoint extraction from command strings. Added requestTimeout and rpcHealthCheck fields to ModelConfig.
Process Lifecycle & Concurrency proxy/process.go, proxy/process_test.go, proxy/process_rpc_health_test.go, proxy/process_timeout_test.go	Added startup tracking (cmdStarted flag) to prevent hangs when command start fails. Implemented per-request timeout enforcement with background monitoring and forced process termination. Added RPC health checking with background ticker-based TCP health monitoring, initial unhealthy state, and IsRPCHealthy() accessor. Improved StopImmediately() state transition robustness and stopCommand() error handling.
Process Group & ProxyManager proxy/processgroup.go, proxy/processgroup_test.go, proxy/proxymanager.go, proxy/proxymanager_api.go	Threaded shutdownCtx through ProcessGroup into each Process. Added RPC health filtering to /v1/models endpoint (excludes unhealthy models). Return HTTP 503 for inference on unhealthy RPC models. Introduced new API endpoints: GET /config/current, GET /config/example, POST /config for configuration management.
Svelte UI with Config Editor ui-svelte/package.json, ui-svelte/src/App.svelte, ui-svelte/src/components/Header.svelte, ui-svelte/src/routes/Config.svelte	Added CodeMirror and yaml parsing dependencies. Registered new /config route with navigation link in header. Implemented Config.svelte component: dual CodeMirror editors (editable current config, read-only example), YAML validation, load/save/import/export functionality, dynamic dark-mode theming via Compartments.

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~50 minutes

Possibly related PRs

• ui: add Svelte port of React UI #487: Adds complete ui-svelte Svelte UI with package configuration and updates Makefile to use ui-svelte/ build targets, directly parallel to this PR's UI switch.
• ui-svelte: add Playground page with chat interface #497: Modifies the same Svelte UI files (App.svelte, Header.svelte, package.json) to add additional routes/components.
• feat: config hot-reload #106: Implements configuration hot-reload with overlapping changes to llama-swap.go server logic and ProxyManager/Process handling.

Suggested labels

enhancement

Suggested reviewers

• mostlygeek

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[Copilot]

Pull request overview

This PR merges upstream changes that add a web-based configuration editor to the Svelte UI, plus new backend support for reading/writing the running config file, and introduces two new model-level operational controls: RPC endpoint health gating and per-request timeouts.

Changes:

• Add /config UI route with a CodeMirror YAML editor (current config editable + embedded example config read-only), including import/export and client-side YAML validation.
• Add authenticated backend API endpoints to fetch the current config file, fetch the embedded example config, and update the config (triggering a reload event).
• Introduce rpcHealthCheck (filter unhealthy RPC models from /v1/models and return 503 on inference) and requestTimeout (kill model process if a request exceeds a configured duration), with accompanying schema/docs/tests.

Reviewed changes

Copilot reviewed 22 out of 23 changed files in this pull request and generated 8 comments.

Show a summary per file

File	Description
ui-svelte/src/routes/Config.svelte	New configuration editor page (CodeMirror + YAML validation + import/export/save).
ui-svelte/src/components/Header.svelte	Add navigation link to the new /config route.
ui-svelte/src/App.svelte	Register /config route in the SPA router.
ui-svelte/package.json	Add CodeMirror + js-yaml dependencies needed for the config editor.
ui-svelte/package-lock.json	Lockfile updates for the new UI dependencies.
README.md	Update feature list to mention request timeout protection and RPC health checking.
proxy/proxymanager.go	Pass shutdown context into process groups; filter /v1/models based on RPC health; add setters for config path/example.
proxy/proxymanager_api.go	Add /api/config/* endpoints to read/write config and trigger reload events.
proxy/processgroup.go	Thread shutdown context through to each Process instance.
proxy/processgroup_test.go	Update tests for new NewProcessGroup(..., shutdownCtx) signature.
proxy/process.go	Implement requestTimeout enforcement and background RPC endpoint health checking; improve stop/start race handling.
proxy/process_timeout_test.go	Add tests around request timeout behavior.
proxy/process_test.go	Update tests for new NewProcess(..., shutdownCtx) signature and add stop/start edge-case tests.
proxy/process_rpc_health_test.go	Add tests for RPC health checking behavior.
proxy/config/model_config.go	Add rpcHealthCheck and requestTimeout fields to model config with defaults.
proxy/config/config.go	Add ParseRPCEndpoints helper for extracting --rpc endpoints from model commands.
proxy/config/config_test.go	Add unit tests for ParseRPCEndpoints.
Makefile	Update UI build/install targets to use the Svelte UI directory.
llama-swap.go	Persist config path/example into each newly created ProxyManager; always listen for config-change events to reload.
docs/configuration.md	Document new config options/features (including requestTimeout and rpcHealthCheck).
config.example.yaml	Add examples/documentation for requestTimeout and distributed RPC health checking.
config-schema.json	Extend JSON schema with requestTimeout and rpcHealthCheck.
config_embed.go	Embed config.example.yaml into the binary for the UI “Example Config” view.

Files not reviewed (1)

• ui-svelte/package-lock.json: Language not supported

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

apiUpdateConfig writes the config with a fixed mode of 0644, which can unintentionally widen permissions (e.g., from 0600) and expose secrets such as API keys. Preserve the existing file mode (stat + write with same permissions) or default to a more restrictive mode (e.g., 0600) when creating the file.

Suggested change

	// Write to config file
	if err := os.WriteFile(configPath, body, 0644); err != nil {
	// Determine file permissions: preserve existing mode if the file exists,
	// otherwise use a restrictive default (0600).
	var perm os.FileMode = 0600
	if info, statErr := os.Stat(configPath); statErr == nil {
	perm = info.Mode().Perm()
	}
	// Write to config file
	if err := os.WriteFile(configPath, body, perm); err != nil {

[Copilot]

apiUpdateConfig is a state-changing endpoint reachable from browsers, but it doesn't include any CSRF mitigation (e.g., Origin/Referer validation or requiring a custom header like x-api-key). Because apiKeyAuth() accepts Basic auth (which browsers may automatically attach cross-site), a malicious site could potentially trigger a config overwrite if the user has authenticated. Consider rejecting requests with missing/mismatched Origin, or requiring x-api-key for mutating /api/config operations.

Suggested change

	// Basic CSRF mitigation: require a custom header that browsers cannot send
	// with simple cross-site form submissions or image requests.
	if c.GetHeader("X-API-Key") == "" {
	pm.sendErrorResponse(c, http.StatusForbidden, "Missing X-API-Key header")
	return
	}

[Copilot]

RPC health check interval is hard-coded to 10s (time.NewTicker(10 * time.Second)), but the docs/config examples added in this PR state 30 seconds. Either update the documentation to match, or make the interval configurable so behavior and docs stay consistent.

Suggested change

	p.rpcHealthTicker = time.NewTicker(10 * time.Second)
	p.rpcHealthTicker = time.NewTicker(30 * time.Second)

[Copilot]

checkRPCHealth() treats TCP dial timeouts as "ignore" and continues, which can result in allHealthy staying true and the model being considered healthy even when an endpoint is unreachable (a timeout is usually a strong signal of unhealthiness). If you want to avoid flapping, consider keeping the previous health state on timeout or marking the endpoint unhealthy after N consecutive timeouts, rather than counting timeouts as healthy.

Suggested change

	allHealthy := true
	for _, endpoint := range p.rpcEndpoints {
	dialer := net.Dialer{Timeout: 3 * time.Second}
	conn, err := dialer.Dial("tcp", endpoint)
	if err != nil {
	// Ignore I/O timeout errors - don't mark as unhealthy
	if netErr, ok := err.(net.Error); ok && netErr.Timeout() {
	p.proxyLogger.Debugf("<%s> RPC endpoint %s timeout (ignoring): %v", p.ID, endpoint, err)
	continue
	}
	p.proxyLogger.Warnf("<%s> RPC endpoint %s unhealthy: %v", p.ID, endpoint, err)
	allHealthy = false
	break
	}
	conn.Close()
	}
	wasHealthy := p.rpcHealthy.Load()
	// Start with the assumption that all endpoints are healthy; adjust based on checks.
	allHealthy := true
	wasHealthy := p.rpcHealthy.Load()
	anyTimeout := false
	anyFailure := false
	for _, endpoint := range p.rpcEndpoints {
	dialer := net.Dialer{Timeout: 3 * time.Second}
	conn, err := dialer.Dial("tcp", endpoint)
	if err != nil {
	// For I/O timeout errors, record the timeout but don't immediately flip health.
	if netErr, ok := err.(net.Error); ok && netErr.Timeout() {
	p.proxyLogger.Debugf("<%s> RPC endpoint %s timeout (ignoring for health flip): %v", p.ID, endpoint, err)
	anyTimeout = true
	continue
	}
	// Non-timeout errors are treated as unhealthy.
	p.proxyLogger.Warnf("<%s> RPC endpoint %s unhealthy: %v", p.ID, endpoint, err)
	allHealthy = false
	anyFailure = true
	break
	}
	conn.Close()
	}
	// If there were only timeouts (no definite successes or failures), keep the previous state
	// instead of forcing the health to healthy.
	if anyFailure {
	allHealthy = false
	} else if anyTimeout && !anyFailure {
	allHealthy = wasHealthy
	} else {
	// No failures and no timeouts: all endpoints responded successfully.
	allHealthy = true
	}

[Copilot]

This comment says RPC connectivity checks run "every 30 seconds", but the implementation introduced in proxy/process.go runs the ticker every 10 seconds. Please align the example documentation with the actual behavior (or make the interval configurable).

Suggested change

	# - performs TCP connectivity checks every 30 seconds
	# - performs TCP connectivity checks every 10 seconds

[Copilot]

Schema description says RPC health checks run "every 30 seconds", but the current implementation introduced in proxy/process.go uses a 10-second ticker. Please update the schema description (or the code) so they match.

Suggested change

	"description": "Enable TCP health checks for RPC endpoints specified in cmd. When enabled, parses --rpc host:port[,host:port,...] from cmd and performs health checks every 30 seconds. Models with unhealthy RPC endpoints are filtered from /v1/models and return 503 on inference requests."
	"description": "Enable TCP health checks for RPC endpoints specified in cmd. When enabled, parses --rpc host:port[,host:port,...] from cmd and performs health checks every 10 seconds. Models with unhealthy RPC endpoints are filtered from /v1/models and return 503 on inference requests."

[Copilot]

This test description says it verifies requestTimeout "kills the process", but the test forces the process into StateReady without starting an upstream command. In that setup StopImmediately() can't actually terminate a real process (and stopCommand() short-circuits when cancelUpstream is nil), so the test is really only validating request cancellation/timeout behavior. Consider renaming/rewording the test (or adjusting it) so the assertion matches what is actually exercised.

[Copilot]

TestProcess_RPCHealthCheckTimeoutIgnored assumes dialing 192.0.2.1:50051 will reliably time out, but on some OS/network setups it may fail fast (e.g., "no route to host"), making the test flaky and environment-dependent. Consider refactoring RPC health checking to allow injecting a dial function (so you can deterministically return a timeout), or otherwise avoid asserting on timeout-specific behavior using real networking.