feat: add webhook endpoint admin API, in-memory store, config.json sync, and per-endpoint delivery tuning

[Pratham-Mishra04]

Summary

This PR wires up the webhook admin API and integrates webhook endpoint configuration into the config.json lifecycle. It adds a WebhookConfig global settings struct to the client config, a webhooks section to config.json for declaring endpoints declaratively, an in-memory endpoint store on Config for zero-database-read serving on the hot path, and a full set of admin HTTP handlers for endpoint CRUD, secret rotation, test delivery, delivery history, and redelivery.

Changes

• Added WebhookConfig to ClientConfig and TableClientConfig, stored as a JSON blob column (webhook_config_json) with BeforeSave/AfterFind serialization. Includes a DeliveryHistoryRetention() helper with a 30-day default on a nil receiver.
• Added a database migration (add_webhook_config_client_column) to introduce the column.
• Added GenerateWebhookEndpointHash for deterministic change detection of endpoint declarations, covering all declared fields (sorted events and headers for stability) while excluding operational counters and generated IDs.
• Added WebhookEndpointConfig to ConfigData and the loadWebhooksConfig startup step, which validates declarations with a warn-and-skip policy and reconciles them against the database using either an additive merge or a full sync depending on whether source_of_truth: config.json is set and the webhooks section is physically present.
• Added an in-memory webhook endpoint store (webhookEndpoints, webhookEndpointsByName) on Config, guarded by a dedicated muWebhooks mutex, with WebhookEndpointByID, WebhookEndpointByName, SetWebhookEndpoint, RemoveWebhookEndpoint, and replaceWebhookEndpoints methods. Handlers keep it in lockstep with every database write.
• Added WebhookHandler with routes for list, get, create, update, delete, rotate-secret, test delivery (with a per-endpoint 10-second cooldown), delivery history pagination, and redelivery. Custom header values are redacted in all API responses; masked placeholders round-tripped through an update are restored from the in-memory store to avoid corrupting stored values.
• Wired WebhookDispatcher into BifrostHTTPServer.Bootstrap, passing it to the async job executor and stopping it on shutdown. The dispatcher is only initialized when both the config store and logs store are available.
• Extended config.schema.json with the webhooks array and client.webhook_config object schemas.

Notable design decisions:

• The in-memory store deliberately does not mirror operational counters (failure counts, timestamps); list views that need them read the database directly.
• Config hash for WebhookConfig is only written when the field is non-nil to avoid hash churn on upgrade for existing deployments.
• Pruning in source-of-truth mode requires the webhooks key to be physically present in the file, preventing an absent section from wiping all database endpoints.
• The typed-nil dispatcher problem for the async job executor interface is handled explicitly to avoid a non-nil interface wrapping a nil pointer.

Type of change

• Bug fix
• Feature
• Refactor
• Documentation
• Chore/CI

Affected areas

• Core (Go)
• Transports (HTTP)
• Providers/Integrations
• Plugins
• UI (React)
• Docs

How to test

go test ./framework/configstore/... ./transports/bifrost-http/...

• Create a webhook endpoint via POST /api/webhooks and verify the secret is returned exactly once.
• Rotate the secret via POST /api/webhooks/{id}/rotate-secret and confirm the old secret is no longer used for signing.
• Add a webhooks section to config.json and restart; verify the endpoint appears in the database and is served from memory.
• Set source_of_truth: config.json, remove an endpoint from the file, and restart; verify the database row is pruned.
• Set source_of_truth: config.json without a webhooks key; verify existing database endpoints are not pruned.
• Fire a test delivery via POST /api/webhooks/{id}/test and confirm the receiver gets a signed request; fire again immediately and confirm a 429 response.
• Seed a failed delivery and redeliver via POST /api/webhooks/deliveries/{id}/redeliver; confirm the job appears in the queue under the original webhook ID and a second redeliver returns 409.

New config fields:

Field	Location	Description
client.webhook_config.delivery_history_retention_days	config.json client block	How long delivery history rows are kept (default: 30 days)
webhooks[].max_retries	config.json webhooks array	Per-endpoint retry count (default: 4)
webhooks[].attempt_timeout_seconds	config.json webhooks array	Per-attempt timeout (default: 10s)
webhooks[].max_concurrent_deliveries	config.json webhooks array	Concurrency cap per node (default: 10)

Breaking changes

• Yes
• No

Security considerations

• Signing secrets are generated server-side and returned exactly once at creation or rotation; they are never included in read responses or list views.
• Custom header values (e.g. Authorization) are encrypted at rest and redacted in all API responses; only header names are visible.
• Reserved delivery headers (webhook-id, webhook-timestamp, webhook-signature, content-type) cannot be overridden by callers.
• Plain HTTP delivery URLs require explicit allow_private_network: true; link-local and metadata addresses are always blocked regardless of that flag.
• The per-endpoint test delivery cooldown (10 seconds) limits abuse of the test endpoint against external receivers.

Checklist

• I read docs/contributing/README.md and followed the guidelines
• I added/updated tests where appropriate
• I updated documentation where needed
• I verified builds succeed (Go and UI)
• I verified the CI pipeline passes locally if applicable

[Pratham-Mishra04]

• docs: add Webhooks management API paths, schemas, and x-bf-async-webhook header parameter to async job endpoints #5270
• docs: add webhooks feature page and link it from async inference #5269
• feat: add dependency-free Go webhook receiver example with Standard Webhooks signature verification #5268
• test: add Postman e2e collection, Newman runner, and webhook-receiver sidecar for async webhook delivery #5267
• feat: add server-side pagination, search, and filtering to webhook endpoints list #5266
• feat: propagate inference request_id through async jobs and webhook deliveries for LLM log reconciliation #5265
• feat: add webhooks management UI with endpoint CRUD, delivery history, secret reveal, and sidebar entry #5264
• refactor: resolve async webhook endpoint by name at submit time via WebhookManager instead of pre-normalizing to ID in context #5252
• feat: add webhook endpoint admin API, in-memory store, config.json sync, and per-endpoint delivery tuning #5251 👈 (View in Graphite)
• feat: add WebhookDispatcher interface and wire async job terminal-state webhook notifications via BifrostContextKeyAsyncWebhook #5250
• feat: add webhook delivery dispatcher with SSRF-safe HTTP client, Standard Webhooks signing, and per-endpoint retry/concurrency tuning #5249
• feat: add webhook jobs work-queue and delivery history tables with store methods and tests #5248
• feat: add config_webhook_endpoints table, store CRUD methods, and secret rotation with Standard Webhooks signing #5247
• refactor: extract SSRF-safe dial logic into core/network package and replace duplicate implementations in fetch.go and skills_serving.go #5246
• dev

This stack of pull requests is managed by Graphite. Learn more about stacking.

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[coderabbitai]

Caution

Review failed

The pull request is closed.

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 74e31004-cc13-43f2-80b9-6e1feee87657

📥 Commits

Reviewing files that changed from the base of the PR and between 3da8f9d and 6a7c2aa.

📒 Files selected for processing (11)

• framework/configstore/clientconfig.go
• framework/configstore/migrations.go
• framework/configstore/rdb.go
• framework/configstore/rdb_test.go
• framework/configstore/tables/clientconfig.go
• transports/bifrost-http/handlers/webhooks.go
• transports/bifrost-http/handlers/webhooks_test.go
• transports/bifrost-http/lib/config.go
• transports/bifrost-http/lib/config_test.go
• transports/bifrost-http/server/server.go
• transports/config.schema.json

---

📝 Walkthrough

Summary by CodeRabbit

• New Features

  • Added webhook endpoint administration, including creation, updates, deletion, secret rotation, test deliveries, delivery history, and redelivery.
  • Added declarative webhook configuration through the main configuration file.
  • Added global delivery history retention settings and per-endpoint delivery tuning.
  • Webhook headers and secrets are masked in API responses.
  • Webhook configuration now persists across restarts and synchronizes with the database.

• Bug Fixes

  • Improved validation for webhook endpoints, headers, events, and delivery settings.
  • Preserved stored secret header values when masked values are submitted during updates.

Walkthrough

Webhook support adds persisted client settings, declarative endpoint loading, in-memory endpoint indexes, administrative HTTP routes, delivery history and redelivery operations, schema definitions, and dispatcher lifecycle wiring.

Changes

Webhook platform

Layer / File(s)	Summary
Webhook configuration persistence and hashing framework/configstore/clientconfig.go, framework/configstore/tables/clientconfig.go, framework/configstore/migrations.go, framework/configstore/rdb.go, framework/configstore/rdb_test.go	Client webhook settings and endpoint hashes are persisted, migrated, serialized, deserialized, validated, and covered by database tests.
Declarative loading and in-memory endpoint state transports/bifrost-http/lib/config.go, transports/bifrost-http/lib/config_test.go, transports/config.schema.json	Webhook declarations are parsed from configuration, reconciled with stored endpoints, and maintained in ID and name indexes with merge and pruning behavior tested.
Webhook administration and delivery operations transports/bifrost-http/handlers/webhooks.go, transports/bifrost-http/handlers/webhooks_test.go	HTTP routes implement endpoint CRUD, secret rotation, response redaction, test delivery, delivery history, and redelivery with validation and lifecycle tests.
Dispatcher startup and route integration transports/bifrost-http/server/server.go	The server starts and stops the webhook dispatcher, registers webhook routes, and passes the dispatcher into asynchronous job execution.

Estimated code review effort: 4 (Complex) | ~60 minutes

Sequence Diagram(s)

sequenceDiagram
  participant ConfigFile
  participant Config
  participant ConfigStore
  participant WebhookHandler
  participant WebhookDispatcher
  ConfigFile->>Config: load webhook declarations
  Config->>ConfigStore: reconcile endpoint rows
  ConfigStore-->>Config: reload canonical endpoints
  Config->>WebhookDispatcher: replace runtime endpoints
  WebhookHandler->>ConfigStore: persist endpoint changes
  WebhookHandler->>WebhookDispatcher: request test delivery

Loading

Possibly related PRs

• maximhq/bifrost#5247: Provides the webhook endpoint model and config-store CRUD and rotation contracts used by these changes.
• maximhq/bifrost#5248: Adds the preceding webhook jobs migration that this client-column migration follows.
• maximhq/bifrost#5266: Modifies the webhook endpoints listing handler affected by this administrative API.

Suggested reviewers: akshaydeo, danpiths

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 32.43% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Title check	✅ Passed	The title accurately summarizes the main change: webhook admin APIs, in-memory config sync, and delivery tuning.
Description check	✅ Passed	The description covers the required core sections and includes testing, security, and checklist details; only non-critical sections are missing.

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch 07-15-feat_webhook_admin_api_in-memory_endpoint_store_and_dispatcher_wiring

Warning

There were issues while running some tools. Please review the errors and either fix the tool's configuration or disable the tool if it's a critical failure.

🔧 golangci-lint (2.12.2)

level=error msg="[linters_context] typechecking error: pattern ./...: directory prefix . does not contain main module or its selected dependencies"

---

_{Comment @coderabbitai help to get the list of available commands.}

[greptile-apps]

Confidence Score: 5/5

The latest fixes look safe to merge.

• Invalid named declarations no longer delete existing endpoints during authoritative sync.
• Secret-only configuration changes no longer create a false synchronized hash.
• Successful database updates reload the endpoint into the mutex-protected in-memory store.
• No new distinct blocking issue was found in the updated code.

Important Files Changed

Filename	Overview
transports/bifrost-http/handlers/webhooks.go	Adds the webhook administration, test delivery, history, and redelivery handlers.
transports/bifrost-http/lib/config.go	Adds declarative webhook reconciliation and synchronized in-memory endpoint indexes.
framework/configstore/clientconfig.go	Adds global webhook settings and deterministic endpoint hashing.
framework/configstore/migrations.go	Adds the webhook configuration JSON column with rollback support.
transports/config.schema.json	Defines global webhook settings and declarative endpoint fields.

_{Reviews (6): Last reviewed commit: "feat: webhook admin API, in-memory endpo..." | Re-trigger Greptile}

[coderabbitai]

Actionable comments posted: 2

🧹 Nitpick comments (2)

transports/bifrost-http/lib/config.go (1)

6297-6317: 🩺 Stability & Availability | 🔵 Trivial | ⚡ Quick win

SetWebhookEndpoint's shallow copy aliases Events/Headers with the caller's struct.

copied := *endpoint copies the pointer/slice/map fields by reference. Callers must currently avoid mutating Events/Headers on the struct after calling SetWebhookEndpoint, but nothing enforces this, and only the Name field's immutability is tested (TestWebhookEndpointMemoryStore). Since WebhookEndpointByID/ByName promise read-only access, a future caller mutating a reused endpoint.Headers map post-Set (e.g. for redaction) would corrupt the cache the delivery worker reads, without any lock protecting that mutation. As per coding guidelines, "mutable config maps such as ExtraHeaders must be defensively copied."

🔒 Proposed fix: defensively copy Events/Headers

 	copied := *endpoint
+	if endpoint.Events != nil {
+		copied.Events = append([]configstoreTables.WebhookEvent(nil), endpoint.Events...)
+	}
+	if endpoint.Headers != nil {
+		copied.Headers = make(map[string]schemas.SecretVar, len(endpoint.Headers))
+		for k, v := range endpoint.Headers {
+			copied.Headers[k] = v
+		}
+	}
 	c.muWebhooks.Lock()

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@transports/bifrost-http/lib/config.go` around lines 6297 - 6317, Update
SetWebhookEndpoint to deep-copy the mutable Events and Headers fields when
creating the cached endpoint, rather than relying on the shallow copied struct.
Preserve the existing ID/name replacement and indexing behavior while ensuring
later caller mutations cannot alter values returned by WebhookEndpointByID or
WebhookEndpointByName.

Source: Coding guidelines

transports/config.schema.json (1)

1285-1292: 🗄️ Data Integrity & Integration | 🔵 Trivial | ⚡ Quick win

Schema allows empty events, but the admin API rejects it.

events has no minItems, so a config.json declaration with "events": [] passes schema validation, while the same payload sent through the admin API (webhooks.go, per TestWebhookHandlerCreateValidation's "no events" case) is rejected with 400. Add "minItems": 1 here so config.json-declared webhooks fail fast at schema validation instead of only at reconciliation time.

♻️ Proposed fix

               "events": {
                 "type": "array",
                 "items": {
                   "type": "string",
                   "enum": ["async_job.completed", "async_job.failed"]
                 },
+                "minItems": 1,
                 "description": "Events this endpoint subscribes to."
               },

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@transports/config.schema.json` around lines 1285 - 1292, Update the events
array schema in the webhook endpoint definition to require at least one item by
adding minItems: 1 alongside its existing type, items, and description
properties. Preserve the current event enum and array validation.

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@transports/bifrost-http/handlers/webhooks.go`:
- Around line 265-284: Update deleteWebhookEndpoint to remove the deleted
endpoint’s id from h.lastTestFire after DeleteWebhookEndpoint succeeds,
alongside h.store.RemoveWebhookEndpoint(id). Ensure cleanup occurs only after a
successful deletion while preserving the existing error responses.

In `@transports/bifrost-http/lib/config.go`:
- Around line 2101-2155: Update syncWebhookEndpointsFromFile to resolve the
existing endpoint match by name or ID before calling
GenerateWebhookEndpointHash. If hash generation fails, mark the matched existing
endpoint ID in keepIDs before continuing, while leaving unmatched entries
uncreated and preserving the existing update/create behavior for successful
hashes.

---

Nitpick comments:
In `@transports/bifrost-http/lib/config.go`:
- Around line 6297-6317: Update SetWebhookEndpoint to deep-copy the mutable
Events and Headers fields when creating the cached endpoint, rather than relying
on the shallow copied struct. Preserve the existing ID/name replacement and
indexing behavior while ensuring later caller mutations cannot alter values
returned by WebhookEndpointByID or WebhookEndpointByName.

In `@transports/config.schema.json`:
- Around line 1285-1292: Update the events array schema in the webhook endpoint
definition to require at least one item by adding minItems: 1 alongside its
existing type, items, and description properties. Preserve the current event
enum and array validation.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 781f0aab-2586-4a8a-a96b-b3248367eb69

📥 Commits

Reviewing files that changed from the base of the PR and between b3c59cc and bd3a160.

📒 Files selected for processing (11)

• framework/configstore/clientconfig.go
• framework/configstore/migrations.go
• framework/configstore/rdb.go
• framework/configstore/rdb_test.go
• framework/configstore/tables/clientconfig.go
• transports/bifrost-http/handlers/webhooks.go
• transports/bifrost-http/handlers/webhooks_test.go
• transports/bifrost-http/lib/config.go
• transports/bifrost-http/lib/config_test.go
• transports/bifrost-http/server/server.go
• transports/config.schema.json

[Pratham-Mishra04]

Merge activity

• Jul 17, 8:36 AM UTC: A user started a stack merge that includes this pull request via Graphite.
• Jul 17, 8:51 AM UTC: Graphite rebased this pull request as part of a merge.
• Jul 17, 8:52 AM UTC: @Pratham-Mishra04 merged this pull request with Graphite.

The base branch was changed.