semantic router #3: embedding executor

[kohlivrinda]

Summary

Wires the bifrost embedding client into the governance plugin so it can call out to an embedding provider for semantic complexity classification. Previously the governance plugin had no way to generate embeddings; this adds the executor interface, the embedding call logic, and the server-side wiring at both bootstrap and plugin reload time.

Changes

• Added EmbeddingRequestExecutor type and EmbeddingExecutorSetter interface to the governance plugin, mirroring the pattern already used by the semantic cache plugin.
• Implemented SetEmbeddingRequestExecutor, embeddingExecutor, CanClassifySemantically, and generateEmbedding on GovernancePlugin. The executor is stored atomically to allow safe concurrent access during plugin reloads and request classification.
• generateEmbedding handles all embedding response encodings (string JSON, float64 array, 2D float64 array, int8, int32) by normalising them to []float32 for uniform cosine similarity comparison.
• The embedding context is derived with a hard timeout from SemanticConfig.Timeout (defaulting to configstore.DefaultComplexitySemanticTimeout), skips the plugin pipeline to prevent recursion, and clears caller routing state so the internal request behaves like a fresh external embeddings call.
• Added embeddingRequestExecutor atomic.Pointer[EmbeddingRequestExecutor] field to GovernancePlugin.
• The HTTP server now wires the executor to any governance plugin implementing EmbeddingExecutorSetter during both Bootstrap and ReloadPlugin, consistent with how the semantic cache plugin is handled.

Type of change

• Bug fix
• Feature
• Refactor
• Documentation
• Chore/CI

Affected areas

• Core (Go)
• Transports (HTTP)
• Providers/Integrations
• Plugins
• UI (React)
• Docs

How to test

go test ./plugins/governance/...
go test ./transports/bifrost-http/...

Key test scenarios covered:

• TestGenerateEmbeddingDecodesAllEncodings — verifies all five embedding wire formats are correctly normalised to []float32.
• TestGenerateEmbeddingRequestShape — asserts the outbound request carries the correct provider, model, input text, skip-pipeline flag, and a bounded deadline.
• TestGenerateEmbeddingTimeoutCancelsCall — confirms a slow provider is cancelled within the configured timeout window.
• TestGenerateEmbeddingGuards — covers nil executor, nil semantic config, empty response, and executor teardown.
• TestCanClassifySemantically — validates the readiness gate across all partial-configuration combinations.

Breaking changes

• Yes
• No

Related issues

Prerequisite for semantic complexity classification (task #5 — exemplar warmup).

Security considerations

The internal embedding request explicitly clears caller key-routing and body-transport state and skips the plugin pipeline, preventing the governance plugin from inadvertently inheriting or leaking caller credentials or routing context into the embedding call.

Checklist

• I read docs/contributing/README.md and followed the guidelines
• I added/updated tests where appropriate
• I updated documentation where needed
• I verified builds succeed (Go and UI)
• I verified the CI pipeline passes locally if applicable

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: be83fdb5-a1ae-47bc-8b83-139e4c96275b

📥 Commits

Reviewing files that changed from the base of the PR and between 7f4e855 and e809390.

📒 Files selected for processing (5)

• core/utils.go
• plugins/governance/embedding.go
• plugins/governance/embedding_test.go
• plugins/governance/main.go
• transports/bifrost-http/server/server.go

🚧 Files skipped from review as they are similar to previous changes (3)

• core/utils.go
• plugins/governance/main.go
• plugins/governance/embedding_test.go

---

📝 Walkthrough

Summary by CodeRabbit

• New Features
  • Added semantic complexity classification using embedding-based analysis.
  • Supports JSON strings, numeric arrays, and integer-array embedding formats.
  • Embedding capabilities connect automatically during server startup and plugin reloads.
• Reliability
  • Added bounded timeouts, cancellation, response validation, and clear failure handling.
  • Improved internal embedding requests to avoid unnecessary processing.
• Tests
  • Added coverage for embedding formats, request handling, token usage, timeouts, errors, and readiness checks.

Walkthrough

Governance plugins now support a concurrency-safe embedding executor. They create timeout-bounded internal requests, normalize supported embedding formats to []float32, and validate semantic-classification readiness. Bootstrap and reload wire the executor.

Changes

Governance embedding execution

Layer / File(s)	Summary
Executor contract and server wiring plugins/governance/embedding.go, plugins/governance/main.go, transports/bifrost-http/server/server.go	Defines the embedding executor contract and setter. GovernancePlugin stores the executor atomically. Bootstrap and ReloadPlugin wire s.Client.EmbeddingRequest.
Internal embedding request generation core/utils.go, plugins/governance/embedding.go	Prepares an internal context that skips the plugin pipeline and clears caller state. Embedding generation applies timeouts, classifies timeout failures, validates responses, and converts supported formats to []float32.
Semantic readiness and validation plugins/governance/embedding.go, plugins/governance/embedding_test.go	CanClassifySemantically checks executor and embedding configuration. Tests cover decoding, request shape, token usage, timeout handling, error classification, empty responses, and configuration guards.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Mergeability Score: 🔵 Low · up to e8093

The PR enables embedding-based semantic classification and wires it during startup and plugin reload. It is mergeable with owner awareness, but readiness may be reported before warmup finishes and upstream 504 responses may be mislabeled as local semantic timeouts.

Sequence Diagram(s)

sequenceDiagram
  participant Bootstrap
  participant GovernancePlugin
  participant EmbeddingRequestExecutor
  Bootstrap->>GovernancePlugin: Wire s.Client.EmbeddingRequest
  GovernancePlugin->>GovernancePlugin: Prepare timeout-bounded internal context
  GovernancePlugin->>EmbeddingRequestExecutor: Send embedding request
  EmbeddingRequestExecutor-->>GovernancePlugin: Return embedding response
  GovernancePlugin->>GovernancePlugin: Normalize embedding values to []float32

Loading

Possibly related PRs

• maximhq/bifrost#5655: Introduces the semantic configuration used by this embedding generation flow.
• maximhq/bifrost#5680: Extends governance embedding generation and decoding paths.
• maximhq/bifrost#5743: Further modifies governance embedding executor behavior and timeout handling.

Suggested reviewers: akshaydeo, pratham-mishra04, madhuvod

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 46.67% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Title check	✅ Passed	The title clearly and concisely identifies the main change: adding an embedding executor for the semantic router.
Description check	✅ Passed	The description covers the purpose, changes, testing, affected areas, security, breaking changes, and checklist with sufficient detail.

✨ Finishing Touches 💡 1

📝 Generate docstrings 💡

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch 07-29-semantic_router_embedding_executor

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[kohlivrinda]

Warning

This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

• semantic router: incremental embedding + embed usage accounting #5896 : 2 dependent PRs (#5901 , #5924 )
• semantic router ui revamp #5865
• move complexity exemplars to configstore with backfill migration #5842
• feat(complexity): harness-aware complexity input extraction for cc and codex #5810
• server: export ObserveWarmupRoutingEmbedding for enterprise #5791
• weaviate + redis adapter fixes #5790
• semantic routing ui #5743
• semantic router: dimension probe + message history window #5748
• semantic cache: min similarity + fallback logging #5742
• smenatic router: budgeting+telemetry management #5686
• semantic routing #5: added semantic classifier for complexity router + handlers changes #5680
• semantic router #3: embedding executor #5656 👈 (View in Graphite)
• semantic router #2: semantic routing config #5655
• semantic router #1: add chromem backend #5628
• complexity : reducing keyword lists from 4 -> 3 lists #5602
• helm and docs changes #5530
• update complexity boundaries #5529
• expose tier/mechanism on every surface #5499
• structured logging #5497
• complexity tier merge #5494
• dev

This stack of pull requests is managed by Graphite. Learn more about stacking.

[coderabbitai]

🧹 Nitpick comments (1)

plugins/governance/embedding.go (1)

68-75: 🎯 Functional Correctness | 🔵 Trivial | ⚡ Quick win

generateEmbedding doesn't re-validate Dimension >= 2 like CanClassifySemantically.

If a caller ever invokes generateEmbedding without first gating on CanClassifySemantically, an under-specified Dimension (e.g. 0 or 1) would slip through and produce a vector whose length is never checked against the configured dimension, risking silent mismatches downstream in similarity comparisons.

🛡️ Optional defensive check

 	if semantic == nil || semantic.Provider == "" || semantic.EmbeddingModel == "" {
 		return nil, 0, fmt.Errorf("semantic classification is not configured")
 	}
+	if semantic.Dimension < 2 {
+		return nil, 0, fmt.Errorf("semantic classification dimension is not configured")
+	}

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@plugins/governance/embedding.go` around lines 68 - 75, Update
GovernancePlugin.generateEmbedding to validate semantic.Dimension is at least 2,
matching CanClassifySemantically, before issuing the embedding request; return
the existing configuration error path for invalid dimensions so direct callers
cannot produce unchecked vector-length mismatches.

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@plugins/governance/embedding.go`:
- Around line 68-75: Update GovernancePlugin.generateEmbedding to validate
semantic.Dimension is at least 2, matching CanClassifySemantically, before
issuing the embedding request; return the existing configuration error path for
invalid dimensions so direct callers cannot produce unchecked vector-length
mismatches.

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 2d0bb6f7-10f9-4ebe-8e31-7c52b27420df

📥 Commits

Reviewing files that changed from the base of the PR and between 00825bc and 44143a9.

📒 Files selected for processing (4)

• plugins/governance/embedding.go
• plugins/governance/embedding_test.go
• plugins/governance/main.go
• transports/bifrost-http/server/server.go

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@plugins/governance/embedding.go`:
- Line 97: Remove the BifrostContextKeySkipPluginPipeline assignment from the
embedding plugin and move the anti-recursion behavior into a core-owned executor
API or option. Update the relevant core execution path to apply this behavior
internally, then have the plugin invoke that API without setting reserved
context keys.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 72d93fbc-300d-4ee7-b75c-102e93e9033c

📥 Commits

Reviewing files that changed from the base of the PR and between 44143a9 and 383a002.

📒 Files selected for processing (4)

• plugins/governance/embedding.go
• plugins/governance/embedding_test.go
• plugins/governance/main.go
• transports/bifrost-http/server/server.go

🚧 Files skipped from review as they are similar to previous changes (1)

• plugins/governance/main.go

[coderabbitai]

Note

GitHub couldn't provide a complete incremental comparison for this pull request, so CodeRabbit is performing a full review instead. This review may take a little longer.

[coderabbitai]

Note

GitHub couldn't provide a complete incremental comparison for this pull request, so CodeRabbit is performing a full review instead. This review may take a little longer.