feat: sentinel key verification middleware#5079
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
1 Skipped Deployment
|
📝 WalkthroughWalkthroughThis pull request introduces a sentinel middleware evaluation engine for validating API keys and enforcing rate limits, integrates error page rendering capabilities, configures Redis for the sentinel service, and adds path-skipping support to logging middleware. The changes span infrastructure, new packages, and service integrations. Changes
Sequence Diagram(s)sequenceDiagram
participant Client
participant ProxyHandler as Proxy Handler
participant Engine as Sentinel Engine
participant KeyAuth as KeyAuth Executor
participant KeyService as Key Service
participant ProxyBackend as Backend Instance
Client->>ProxyHandler: HTTP Request + Authorization
ProxyHandler->>ProxyHandler: Strip X-Unkey-Principal
ProxyHandler->>Engine: Evaluate(ctx, session, request, middleware)
Engine->>Engine: Match request against policies
Engine->>KeyAuth: Execute KeyAuth policy
KeyAuth->>KeyAuth: Extract API key from request
KeyAuth->>KeyService: Validate & verify key
KeyService-->>KeyAuth: Key validation result + rate limits
KeyAuth->>KeyAuth: Write rate limit headers
KeyAuth-->>Engine: Principal (authenticated user info)
Engine-->>ProxyHandler: Result with Principal
ProxyHandler->>ProxyHandler: Serialize & set X-Unkey-Principal header
ProxyHandler->>ProxyBackend: Forward request with headers
ProxyBackend-->>Client: Response
Estimated code review effort🎯 4 (Complex) | ⏱️ ~60 minutes 🚥 Pre-merge checks | ✅ 2 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (2 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches
🧪 Generate unit tests (beta)
Warning There were issues while running some tools. Please review the errors and either fix the tool's configuration or disable the tool if it's a critical failure. 🔧 Buf (1.65.0)svc/sentinel/proto/config/v1/config.protoFailure: no .proto files were targeted. This can occur if no .proto files are found in your input, --path points to files that do not exist, or --exclude-path excludes all files. svc/sentinel/proto/policies/v1/keyauth.protoFailure: no .proto files were targeted. This can occur if no .proto files are found in your input, --path points to files that do not exist, or --exclude-path excludes all files. svc/sentinel/proto/policies/v1/policy.protoFailure: no .proto files were targeted. This can occur if no .proto files are found in your input, --path points to files that do not exist, or --exclude-path excludes all files. 🔧 golangci-lint (2.5.0)Command failed Tip Issue Planner is now in beta. Read the docs and try it out! Share your feedback on Discord. Comment |
fc21d5f to
2964b9a
Compare
* fix error pages * remove test * move some files * Update svc/frontline/internal/errorpage/error.go.tmpl Co-authored-by: Andreas Thomas <dev@chronark.com> * [autofix.ci] apply automated fixes --------- Co-authored-by: Andreas Thomas <dev@chronark.com> Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
There was a problem hiding this comment.
Actionable comments posted: 5
🧹 Nitpick comments (4)
svc/frontline/services/proxy/forward.go (1)
229-232: Minor: redundant ContentLength assignment.Both
resp.ContentLength(field) andContent-Lengthheader are set. The reverse proxy uses the field, making the header redundant—though harmless.♻️ Optional simplification
resp.Body = io.NopCloser(bytes.NewReader(htmlBody)) resp.ContentLength = int64(len(htmlBody)) resp.Header.Set("Content-Type", "text/html; charset=utf-8") - resp.Header.Set("Content-Length", fmt.Sprintf("%d", len(htmlBody)))🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@svc/frontline/services/proxy/forward.go` around lines 229 - 232, The code redundantly sets both resp.ContentLength and the "Content-Length" header on the proxied response; remove the redundant header-setting to avoid duplication. In forward.go where you set resp.Body, resp.ContentLength and then resp.Header.Set("Content-Length", ...), delete the resp.Header.Set("Content-Length", fmt.Sprintf(...)) call (keep resp.ContentLength and the Content-Type header) so the reverse proxy relies on the ContentLength field only.svc/sentinel/engine/engine_test.go (1)
12-41: Consider grouping ParseMiddleware tests using t.Run.Per coding guidelines, use
t.Run()for scenarios. The ParseMiddleware tests cover related scenarios and could be consolidated:♻️ Optional refactor with t.Run
-func TestParseMiddleware_Nil(t *testing.T) { - t.Parallel() - assert.Nil(t, ParseMiddleware(nil)) -} - -func TestParseMiddleware_Empty(t *testing.T) { - t.Parallel() - assert.Nil(t, ParseMiddleware([]byte{})) -} - -func TestParseMiddleware_EmptyJSON(t *testing.T) { - t.Parallel() - assert.Nil(t, ParseMiddleware([]byte("{}"))) -} - -func TestParseMiddleware_InvalidProto(t *testing.T) { - t.Parallel() - assert.Nil(t, ParseMiddleware([]byte("not a valid protobuf"))) -} +func TestParseMiddleware(t *testing.T) { + t.Parallel() + t.Run("nil input", func(t *testing.T) { + assert.Nil(t, ParseMiddleware(nil)) + }) + t.Run("empty input", func(t *testing.T) { + assert.Nil(t, ParseMiddleware([]byte{})) + }) + t.Run("empty JSON", func(t *testing.T) { + assert.Nil(t, ParseMiddleware([]byte("{}"))) + }) + t.Run("invalid proto", func(t *testing.T) { + assert.Nil(t, ParseMiddleware([]byte("not a valid protobuf"))) + }) + // ... remaining subtests +}As per coding guidelines: "Organize tests using
t.Run()for scenarios".🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@svc/sentinel/engine/engine_test.go` around lines 12 - 41, Combine the separate TestParseMiddleware_* functions into a single TestParseMiddleware that uses t.Run subtests for each scenario (Nil, Empty, EmptyJSON, InvalidProto, NoPolicies), calling ParseMiddleware in each subtest; keep t.Parallel where appropriate (call t.Parallel() inside individual subtests) and reuse sentinelv1.Middleware and protojson.Marshal for the NoPolicies case to produce the raw input. Ensure each subtest uses assert/require as before and preserves error handling and nolint comments.svc/frontline/internal/errorpage/error.go.tmpl (1)
164-166: Consider parameterizing the support email.The support email
support@unkey.comis hardcoded. If this varies by deployment or needs updates, extracting it as a template variable would improve maintainability.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@svc/frontline/internal/errorpage/error.go.tmpl` around lines 164 - 166, Replace the hardcoded support email in the template with a template variable (e.g. "{{.SupportEmail}}") so the footer uses a parameter instead of "support@unkey.com"; update the data struct or map used when executing the template (e.g. the ErrorPageData or the code that renders error.go.tmpl) to provide SupportEmail and optionally supply a default value ("support@unkey.com") when none is provided.svc/sentinel/routes/proxy/handler.go (1)
81-88: Serialize principal before setting header or propagate error if serialization fails.SerializePrincipal can fail (wraps protojson.Marshal), and when it does, the X-Unkey-Principal header is silently omitted from the request forwarded to the backend service. Either propagate the serialization error to fail fast, or explicitly document that backend services must tolerate a missing principal header.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@svc/sentinel/routes/proxy/handler.go` around lines 81 - 88, The current handler silently drops the principal header when engine.SerializePrincipal fails; locate the block that checks result.Principal and calls engine.SerializePrincipal and change the error path to fail fast: if serErr != nil return or propagate an error from the handler (instead of only logger.Error) so the request is not forwarded without engine.PrincipalHeader; ensure the handler's signature (or its caller chain) is updated to accept/propagate the error and keep the existing success path that calls req.Header.Set(engine.PrincipalHeader, principalJSON).
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@svc/sentinel/engine/BUILD.bazel`:
- Around line 27-55: Replace all usages of testify/assert with testify/require
across engine_test.go, integration_test.go, keyextract_test.go, and
match_test.go: update import lines to remove
"github.com/stretchr/testify/assert" and import
"github.com/stretchr/testify/require" instead, then change all assert.X calls to
require.X (adjusting tests where behavior differs because require fails the test
immediately). After updating code, remove the
`@com_github_stretchr_testify//assert` dependency from the BUILD.bazel go_test
deps and ensure `@com_github_stretchr_testify//require` remains referenced.
In `@svc/sentinel/engine/integration_test.go`:
- Around line 286-317: Update the TestKeyAuth_ValidKey test to follow test
guidelines by wrapping the scenario in t.Run (e.g., t.Run("valid key", func(t
*testing.T) { ... })) and replace all assert.* calls in that test with require.*
(for example change assert.Equal to require.Equal). Ensure the same replacement
(assert -> require) is applied consistently across other tests in this file;
keep test setup (newTestHarness, h.seed, newSession, h.engine.Evaluate)
unchanged but move their usage inside the t.Run closure so the subtest uses
require for assertions.
In `@svc/sentinel/engine/keyextract_test.go`:
- Around line 8-9: Replace the testify/assert import with testify/require in
svc/sentinel/engine/keyextract_test.go and update every assert.* call to
require.* for the assertions referenced (occurrences at lines 19, 29, 37, 52,
69, 89, 109, 128, 151, 174) so the tests fail fast; ensure the import block uses
"github.com/stretchr/testify/require" and each assertion call (e.g., assert.Nil,
assert.Equal, assert.NoError, etc.) is renamed to the corresponding
require.Nil/Equal/NoError variant in the test file.
In `@svc/sentinel/engine/match_test.go`:
- Around line 8-10: Remove the "github.com/stretchr/testify/assert" import and
replace every use of assert.* with require.* in this test file so all assertions
become fatal (require) calls; update the import block to include
"github.com/stretchr/testify/require" (remove the assert import), change the 18
occurrences of assert.* to require.* (e.g., assert.Equal -> require.Equal,
assert.NoError -> require.NoError, etc.) across the test functions in
match_test.go, and run the tests to ensure there are no unused import errors.
In `@svc/sentinel/engine/match.go`:
- Around line 60-72: evalMatchExpr currently calls expr.GetExpr() without
guarding for a nil expr which can panic; add a defensive nil check at the top of
evalMatchExpr (similar to the nil-guard used in evalPathMatch) and return false,
nil when expr is nil so the switch below (using expr.GetExpr()) is safe; update
the function handling for expr (and keep the existing switch on expr.GetExpr()
with cases for MatchExpr_Path, MatchExpr_Method, MatchExpr_Header,
MatchExpr_QueryParam) to rely on that early return.
---
Duplicate comments:
In `@svc/sentinel/engine/integration_test.go`:
- Around line 494-519: This test (TestEvaluate_DisabledPoliciesSkipped) is a
duplicate of an existing case — locate the other equivalent test and either
remove this duplicate or merge it as a subtest using t.Run to avoid repetition;
when merging, keep the same assertions that call h.engine.Evaluate(ctx, sess,
req, mw) and the check require.NoError(t, err) followed by assert.Nil(t,
result.Principal) so behavior is preserved. Ensure the unique identifiers
(TestEvaluate_DisabledPoliciesSkipped, h.engine.Evaluate, mw policy with Id
"disabled") are consolidated and do not appear twice in the test suite.
- Around line 352-380: This test is a duplicate; consolidate it with the
existing KeyAuth tests by removing the redundant TestKeyAuth_MissingKey_Reject
or converting it into a subtest via t.Run inside the primary KeyAuth test suite:
move the body (newTestHarness, s := h.seed, building mw with
sentinelv1.Middleware and sentinelv1.Policy_Keyauth, calling h.engine.Evaluate
and the assertions) into a t.Run("MissingKey", func(t *testing.T){ ... }) block
in the existing test, ensure you use the same assertion style (require/assert)
as the parent tests, and delete the duplicate standalone
TestKeyAuth_MissingKey_Reject function to avoid repeated coverage of the same
case.
- Around line 319-350: This test duplicates behavior covered elsewhere; refactor
TestKeyAuth_ValidKey_WithIdentity by either removing the duplicate test or
converting it into a subtest with t.Run and reusing setup helpers
(newTestHarness, seed, seedKeyWithIdentity) so assertions aren't repeated;
ensure you use require for fatal checks (e.g., require.NoError, require.NotNil)
before non-fatal asserts and eliminate redundant assertions on Subject/Claims to
avoid the duplicate coverage flagged in the comment.
- Around line 382-410: This test appears to be a duplicate of an existing case;
remove or consolidate it so there's a single canonical test for the "missing key
with AllowAnonymous" scenario—locate the duplicate by searching for the test
name TestKeyAuth_MissingKey_AllowAnonymous (or an equivalent subtest) and either
delete this redundant function or merge it into the existing t.Run variant; if
merging, ensure you call newTestHarness, newSession, and engine.Evaluate
(h.engine.Evaluate) under a single test body and keep the require.NoError /
assert.Nil assertions intact for the Principal and the
sentinelv1.Middleware/KeyAuth config.
- Around line 465-490: This test (TestKeyAuth_WrongKeySpace) is a
duplicate—remove this standalone test and instead add it as a subtest (t.Run)
under the existing KeyAuth tests or merge its assertions into the existing test
harness flow; locate the TestKeyAuth_WrongKeySpace function and either delete it
or move its body into an existing TestKeyAuth t.Run block, keeping the same
setup (newTestHarness, h.seed, newSession) and assertions (require.Error +
assert.Contains) so behavior remains covered without duplicated cases.
- Around line 438-463: This test is a duplicate as noted by the "t.Run" comment;
remove or consolidate TestKeyAuth_InvalidKey_Disabled so it isn't redundant:
locate the TestKeyAuth_InvalidKey_Disabled function and either delete it or move
its assertions into the existing t.Run subtest that covers the same scenario,
ensuring you reuse h.seed, h.seedDisabledKey, the constructed middleware (Policy
with KeyAuth/KeySpaceId), the request using disabled.RawKey, and the call to
h.engine.Evaluate to preserve behavior and avoid duplicate test cases.
- Around line 412-436: This test is a duplicate per the t.Run pattern note;
remove or consolidate it so we don't have two identical cases. Locate the
TestKeyAuth_InvalidKey_NotFound function in integration_test.go and either
delete it or merge its assertions into the existing t.Run subtest for key-auth
invalid-key cases (keeping the same session/request setup and the middleware
construction using sentinelv1.Middleware and KeyAuth with s.KeySpaceID) so
behavior is covered only once.
- Around line 521-552: This test (TestEvaluate_MatchFiltering) is a duplicate;
remove the standalone duplicate and instead place its logic as a subtest under
the existing TestEvaluate test (using t.Run) or merge it into the original test
body so we don't have two identical cases; locate the block using
TestEvaluate_MatchFiltering, h.seed, newSession, and h.engine.Evaluate and
either delete this duplicate function or move its assertions into the original
TestEvaluate t.Run, preserving the same assertions (require.NoError and
assert.Nil on result.Principal).
---
Nitpick comments:
In `@svc/frontline/internal/errorpage/error.go.tmpl`:
- Around line 164-166: Replace the hardcoded support email in the template with
a template variable (e.g. "{{.SupportEmail}}") so the footer uses a parameter
instead of "support@unkey.com"; update the data struct or map used when
executing the template (e.g. the ErrorPageData or the code that renders
error.go.tmpl) to provide SupportEmail and optionally supply a default value
("support@unkey.com") when none is provided.
In `@svc/frontline/services/proxy/forward.go`:
- Around line 229-232: The code redundantly sets both resp.ContentLength and the
"Content-Length" header on the proxied response; remove the redundant
header-setting to avoid duplication. In forward.go where you set resp.Body,
resp.ContentLength and then resp.Header.Set("Content-Length", ...), delete the
resp.Header.Set("Content-Length", fmt.Sprintf(...)) call (keep
resp.ContentLength and the Content-Type header) so the reverse proxy relies on
the ContentLength field only.
In `@svc/sentinel/engine/engine_test.go`:
- Around line 12-41: Combine the separate TestParseMiddleware_* functions into a
single TestParseMiddleware that uses t.Run subtests for each scenario (Nil,
Empty, EmptyJSON, InvalidProto, NoPolicies), calling ParseMiddleware in each
subtest; keep t.Parallel where appropriate (call t.Parallel() inside individual
subtests) and reuse sentinelv1.Middleware and protojson.Marshal for the
NoPolicies case to produce the raw input. Ensure each subtest uses
assert/require as before and preserves error handling and nolint comments.
In `@svc/sentinel/routes/proxy/handler.go`:
- Around line 81-88: The current handler silently drops the principal header
when engine.SerializePrincipal fails; locate the block that checks
result.Principal and calls engine.SerializePrincipal and change the error path
to fail fast: if serErr != nil return or propagate an error from the handler
(instead of only logger.Error) so the request is not forwarded without
engine.PrincipalHeader; ensure the handler's signature (or its caller chain) is
updated to accept/propagate the error and keep the existing success path that
calls req.Header.Set(engine.PrincipalHeader, principalJSON).
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
* feat: key-sentinel-middleware * fix error pages (#5083) * fix error pages * remove test * move some files * Update svc/frontline/internal/errorpage/error.go.tmpl Co-authored-by: Andreas Thomas <dev@chronark.com> * [autofix.ci] apply automated fixes --------- Co-authored-by: Andreas Thomas <dev@chronark.com> Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com> * add rl headers. * feat: new ui and fixed a bunch of stuff * Update svc/sentinel/engine/match.go Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> * fix: coderabbit --------- Co-authored-by: Andreas Thomas <dev@chronark.com> Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com> Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
* test keys table * re org and exports * error fix * Apos * chore: remove deployment breadcrumbs (#5019) * fix: cleanup project side nav * feat: simplify deployment overview page only show build logs until it's built, then show domains and network * chore: clean up nav * fix(clickhouse): improve latest keys used queries for high volume (150M +) (#4959) * fix(clickhouse): improve clickhouse query for key logs and add new table and mv for latest keys used * fix valid/error count = 0 scenario * remove identity_id from order by * wrap identity_id with aggregating function since its removed from the order key --------- Co-authored-by: Flo <53355483+Flo4604@users.noreply.github.com> * fix: domain refetch and promotion disable rule (#5013) * fix: domain refetch and promotion disable rule * fix: regression --------- Co-authored-by: Andreas Thomas <dev@chronark.com> * refactor: move custom domains to tanstack db (#5017) * refactor: move custom domains to tanstack db * fix: comment * fix: delete mutation * remove: unnecessary query * remove agent (#5021) * remove agent * remove agent * chore: vault in dashboard (#5023) * remove agent * remove agent * use vault in dashboard * remove * project domain (#5022) * fix: cleanup project side nav * feat: simplify deployment overview page only show build logs until it's built, then show domains and network * chore: clean up nav * feat: add per-project sticky domain and only display that * chore: use vault in api (#5024) * chore: use vault in api * chore: use vault in api * fix harness * use memory test * vault container go start * [autofix.ci] apply automated fixes --------- Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com> * fix: Make GH callback dynamic (#5029) * dunno * nextjs should allow a setting that says dynamic * [autofix.ci] apply automated fixes --------- Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com> * allow longer timeouts (#5032) * docs: add ratelimit.unkey.com benchmark links to ratelimiting docs Add references to real-time performance benchmarks in: - introduction.mdx: new 'Performance at scale' accordion - modes.mdx: link after latency claim Presents benchmarks as capability demonstration rather than comparison. * docs: add description to cache store interface page (#5037) Add missing SEO description to frontmatter Generated-By: mintlify-agent Co-authored-by: mintlify[bot] <109931778+mintlify[bot]@users.noreply.github.com> Co-authored-by: Andreas Thomas <dev@chronark.com> * docs: remove orphaned SDK documentation (#5033) Remove Spring Boot Java, Rust, and Elixir SDK docs that are not linked in navigation and appear to be outdated/unmaintained. Generated-By: mintlify-agent Co-authored-by: mintlify[bot] <109931778+mintlify[bot]@users.noreply.github.com> Co-authored-by: Andreas Thomas <dev@chronark.com> * No data * add bg * rework release (#5044) * rework release * rework release * feat: generate rpc wrappers (#5028) * feat: generate rpc wrappers * bazel happyier * more changes * more changes * move path * delete old files (#5043) * fix: rabbit comments --------- Co-authored-by: Oz <21091016+ogzhanolguncu@users.noreply.github.com> * feat/gossip (#5015) * add a gossip implementation * add gossip to sentinel/frontline * add message muxing * sentinel fun * cleansings * cleansings * cleansings * cleansings * use oneof * fix bazel happiness * do some changies * exportoneof * more cool fancy thingx * change gateway choosing * add label * adjjust some more * adjjust some more * fixa test * goodbye kafka * fix: bazel * rename gateway -> ambassador * add docs * fix: rabbit comments * [autofix.ci] apply automated fixes * idfk * more changes * more changes * fix ordering * fix missing files * fix test --------- Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com> * fix: retry hubble ui (#5056) * fix: wait for cillium policy until CRDs are ready (#5059) * fix: retry cillium policy until CRDs are ready * fix: blocks until all system pods are ready * deployment build screen v1 (#5042) * fix: cleanup project side nav * feat: simplify deployment overview page only show build logs until it's built, then show domains and network * feat: new build screen for ongoing deployments * fix: table column typo * fix: update copy to remove mention of analytics deletion (#5067) * fix typo (#5039) * rfc: sentinel middlewares (#5041) * fix: cleanup project side nav * feat: simplify deployment overview page only show build logs until it's built, then show domains and network * feat: middleware rfc * Update svc/sentinel/proto/buf.gen.ts.yaml Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> --------- Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> Co-authored-by: Flo <53355483+Flo4604@users.noreply.github.com> * feat: config files (#5045) * fix: cleanup project side nav * feat: simplify deployment overview page only show build logs until it's built, then show domains and network * feat: add pkg/config for struct-tag-driven TOML/YAML/JSON configuration Introduces a new configuration package that replaces environment variable based configuration with file-based config. Features: - Load and validate config from TOML, YAML, or JSON files - Struct tag driven: required, default, min/max, oneof, nonempty - Environment variable expansion (${VAR} and ${VAR:-default}) - JSON Schema generation for editor autocompletion - Collects all validation errors instead of failing on first - Custom Validator interface for cross-field checks Also adds cmd/generate-config-docs for generating MDX documentation from Go struct tags, and a Makefile target 'config-docs'. Amp-Thread-ID: https://ampcode.com/threads/T-019c672a-0e8e-7138-b0ab-27cdbeaca7ba Co-authored-by: Amp <amp@ampcode.com> * remove gen * clean up * feat(api): migrate API service to file-based TOML config (#5046) * feat(api): migrate API service to file-based config Migrate the API service from environment variables to TOML file-based configuration using pkg/config. Replaces all UNKEY_* env vars with a structured api.toml config file. Changes: - Rewrite svc/api/config.go with tagged Config struct - Update svc/api/run.go to use new config fields - Update cmd/api/main.go to accept --config flag - Add dev/config/api.toml for docker-compose - Update dev/k8s/manifests/api.yaml with ConfigMap - Regenerate config docs from struct tags Amp-Thread-ID: https://ampcode.com/threads/T-019c672a-0e8e-7138-b0ab-27cdbeaca7ba Co-authored-by: Amp <amp@ampcode.com> * feat(vault): migrate Vault service to file-based TOML config (#5047) * feat(vault): migrate Vault service to file-based config Migrate the Vault service from environment variables to TOML file-based configuration using pkg/config. Changes: - Rewrite svc/vault/config.go with tagged Config struct - Update svc/vault/run.go to use new config fields - Update cmd/vault/main.go to accept --config flag - Add dev/config/vault.toml for docker-compose - Update dev/k8s/manifests/vault.yaml with ConfigMap - Remove UNKEY_* env vars from docker-compose and k8s Amp-Thread-ID: https://ampcode.com/threads/T-019c672a-0e8e-7138-b0ab-27cdbeaca7ba Co-authored-by: Amp <amp@ampcode.com> * feat(ctrl): migrate Ctrl API and Worker to file-based TOML config (#5048) * feat(ctrl): migrate Ctrl API and Worker services to file-based config Migrate both ctrl-api and ctrl-worker from environment variables to TOML file-based configuration using pkg/config. Changes: - Rewrite svc/ctrl/api/config.go and svc/ctrl/worker/config.go - Update run.go files to use new config fields - Update cmd/ctrl/api.go and worker.go to accept --config flag - Add dev/config/ctrl-api.toml and ctrl-worker.toml - Update dev/k8s/manifests/ctrl-api.yaml and ctrl-worker.yaml with ConfigMaps - Remove UNKEY_* env vars from docker-compose and k8s manifests * feat(krane): migrate Krane service to file-based TOML config (#5049) * feat(krane): migrate Krane service to file-based config Migrate the Krane container orchestrator from environment variables to TOML file-based configuration using pkg/config. Changes: - Rewrite svc/krane/config.go with tagged Config struct - Update svc/krane/run.go to use new config fields - Update cmd/krane/main.go to accept --config flag - Add dev/config/krane.toml for docker-compose - Update dev/k8s/manifests/krane.yaml with ConfigMap - Remove UNKEY_* env vars from docker-compose and k8s Amp-Thread-ID: https://ampcode.com/threads/T-019c672a-0e8e-7138-b0ab-27cdbeaca7ba Co-authored-by: Amp <amp@ampcode.com> * feat(frontline): migrate Frontline service to file-based TOML config (#5050) * feat(frontline): migrate Frontline service to file-based config Migrate the Frontline reverse proxy from environment variables to TOML file-based configuration using pkg/config. Changes: - Rewrite svc/frontline/config.go with tagged Config struct - Update svc/frontline/run.go to use new config fields - Update cmd/frontline/main.go to accept --config flag - Update dev/k8s/manifests/frontline.yaml with ConfigMap - Remove UNKEY_* env vars from k8s manifest Amp-Thread-ID: https://ampcode.com/threads/T-019c672a-0e8e-7138-b0ab-27cdbeaca7ba Co-authored-by: Amp <amp@ampcode.com> * feat(preflight): migrate Preflight service to file-based TOML config (#5051) * feat(preflight): migrate Preflight service to file-based config Migrate the Preflight webhook admission controller from environment variables to TOML file-based configuration using pkg/config. Changes: - Rewrite svc/preflight/config.go with tagged Config struct - Update svc/preflight/run.go to use new config fields - Update cmd/preflight/main.go to accept --config flag - Update dev/k8s/manifests/preflight.yaml with ConfigMap - Remove UNKEY_* env vars from k8s manifest Amp-Thread-ID: https://ampcode.com/threads/T-019c672a-0e8e-7138-b0ab-27cdbeaca7ba Co-authored-by: Amp <amp@ampcode.com> * feat(sentinel): migrate Sentinel service to file-based config (#5052) Migrate the Sentinel sidecar from environment variables to TOML file-based configuration using pkg/config. This is the final service migration in the config stack. Changes: - Rewrite svc/sentinel/config.go with tagged Config struct - Update svc/sentinel/run.go to use new config fields - Update cmd/sentinel/main.go to accept --config flag - Update dev/docker-compose.yaml: replace env vars with TOML volume mounts for all migrated services (api, vault, krane, ctrl-api, ctrl-worker) - Minor formatting fix in pkg/db generated code --------- Co-authored-by: Amp <amp@ampcode.com> --------- Co-authored-by: Amp <amp@ampcode.com> --------- Co-authored-by: Amp <amp@ampcode.com> --------- Co-authored-by: Amp <amp@ampcode.com> --------- Co-authored-by: Amp <amp@ampcode.com> --------- Co-authored-by: Amp <amp@ampcode.com> * fix: bad config * remove unnecessary tls config for ctrl api * fix: error * fix: do not log config content * ix: remove kafka * fix: replica * fix: return err * fix: only overwrite frontline id if missing * fix: observability * fix: otel * fix: redundant config * fix: reuse tls * fix: consolidate * fix: use shared configs * fix: config * fix: something * Update pkg/config/common.go Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> * fix: vault startup * fix: instanceid * fix: vault config * fix: make configs required * fix: everything works again --------- Co-authored-by: Amp <amp@ampcode.com> Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> * clean deployment url label (#4976) * clean deployment url * fix conversion error and maintain single source of truth --------- Co-authored-by: Andreas Thomas <dev@chronark.com> Co-authored-by: Flo <53355483+Flo4604@users.noreply.github.com> * feat: New deploy settings (#5073) * feat: add github section * feat: Add icons * feat: add new sections * feat: add settingsgroup * feat: add region selection * feat: add instances * feat: add memory and cpu section * feat: add sections * feat: add health check * feat: add scaling * fix: get rid of redundant prop * refactor: Add toasts to mutations * refactor: rename component * feat: add port section * feat: fix overlapping borders * refactor: fix healthcheck tRPC * feat: add command section * feat: add env section * fix: finalize env-vars * refactor: finalize * feat: Add custom domains * fix: overflwo * feat: make tRPC route for each mutation * fix: displayValue styles * refactor: tidy * fix: revert accidental changes * feat: add cname table * fix: github styling issues * refactor: tidy * refactor: rename * fix: linter * fix: dynamic form issue * feat: allow env selection * chore: tidy * fix: use same chevron * fix: use certmanager if availiable otherwise certfile (#5076) * fix: use certmanager if availiable otherwise certfile * feat: make tls enabled by default now you need to explicitely pass tls.disabled=true if not, we fail during startup. also renamed some port vars to make it obvious what they are used for * chore: log candidates for easier debugging * fix: use static certs first --------- Co-authored-by: chronark <dev@chronark.com> * feat: sentinel key verification middleware (#5079) * feat: key-sentinel-middleware * fix error pages (#5083) * fix error pages * remove test * move some files * Update svc/frontline/internal/errorpage/error.go.tmpl Co-authored-by: Andreas Thomas <dev@chronark.com> * [autofix.ci] apply automated fixes --------- Co-authored-by: Andreas Thomas <dev@chronark.com> Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com> * add rl headers. * feat: new ui and fixed a bunch of stuff * Update svc/sentinel/engine/match.go Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> * fix: coderabbit --------- Co-authored-by: Andreas Thomas <dev@chronark.com> Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com> Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> * clean up after sentinel middleware (#5088) * feat: key-sentinel-middleware * fix error pages (#5083) * fix error pages * remove test * move some files * Update svc/frontline/internal/errorpage/error.go.tmpl Co-authored-by: Andreas Thomas <dev@chronark.com> * [autofix.ci] apply automated fixes --------- Co-authored-by: Andreas Thomas <dev@chronark.com> Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com> * add rl headers. * feat: new ui and fixed a bunch of stuff * Update svc/sentinel/engine/match.go Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> * fix: coderabbit * chore: clean up old columns * fix: db --------- Co-authored-by: Flo <flo@unkey.com> Co-authored-by: Flo <53355483+Flo4604@users.noreply.github.com> Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com> Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> * fix proto type (#5093) * fix: cleanup project side nav * feat: simplify deployment overview page only show build logs until it's built, then show domains and network * fix: runtime exception due to gaslighting type * fix: Modals with combo box work again (#5002) * chore: remove chproxy routes (#5101) * chore: remove chproxy routes * refactor: move prometheus metrics to scoped packages (#5102) * remove the hand holding (#5108) * feat: gossip metrics (#5107) * fix: Make identity slugs copyable (#5100) * fix: make me copy * Update web/apps/dashboard/app/(app)/[workspaceSlug]/authorization/permissions/components/table/components/assigned-items-cell.tsx Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> --------- Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> * refactor: useDeployment hook usage (#5086) * refactor: useDeployment hook usage * fix: remove redundant check * feat: i need more metrics (#5115) * docs: Python and Go examples (#5058) * Add Python and Go SDK documentation - Add Go quickstart guide with stdlib, Gin, and Echo examples - Add Python quickstart guide with FastAPI, Flask, Django - Add Go cookbook: stdlib, Gin, Echo middleware recipes - Add Python cookbook: FastAPI rate limiting recipe All content uses Unkey v2 API only. * Add final Python cookbook recipe and 5-minutes guide * Update docs.json sidebar navigation - Add Go and Python quickstart guides to Framework Guides - Add Go and Python cookbook recipes to Recipes section - Remove duplicate 5-minutes quickstart file * Add Go examples to quickstart and reorganize cookbook by language - Add Go code examples to /quickstart/quickstart.mdx for key creation and verification - Reorganize cookbook recipes into subsections: TypeScript, Go, Python, General - Keep existing TypeScript and Python examples in quickstart * Update cookbook index with new Go and Python recipes * Fix code issues in Go and Python documentation - Fix int to string conversion in go-gin-middleware (use strconv) - Fix middleware composition in go-stdlib-middleware - Fix wait calculation in python-fastapi-ratelimit (use total_seconds) - Fix headers attachment in python-fastapi-ratelimit (use JSONResponse) - Fix nil pointer dereference in quickstart/go - Fix unsafe type assertion in quickstart/go * Fix async/sync issue and nil pointer in quickstart docs - Use verify_key_async in Python async route - Add nil check for result.Code in Go quickstart * Fix more code issues in documentation - Fix GetUnkeyResult type assertion in go-gin-middleware - Fix imports in python-fastapi-ratelimit (add JSONResponse, remove unused timedelta) - Update basic rate limit example to use async API with context manager - Add missing os import in Django settings snippet * Fix missing os import in python-flask-auth.mdx * Fix unsafe type assertions in Go middleware docs - Fix RequirePermission in go-echo-middleware with safe type assertion - Fix GetUnkeyResult in go-echo-middleware with safe type assertion - Fix RequirePermission in go-gin-middleware with safe type assertion * Fix error handling in Python docs - replace ApiError with UnkeyError * Update legacy analytics documentation - Replace outdated /apis/features/analytics.mdx with minimal reference page - Remove analytics from API Keys sidebar in docs.json - Add redirect from /apis/features/analytics to /analytics/overview * fix * Update to mint * Fix critical type assertion issues in go-gin-middleware - Store pointer to struct in context (not value) for type assertion compatibility - Add checked type assertion in RequireRole with proper error handling * Add it back * fix the comma * revert * Update go examples * cookbook update * update quickstart * remove analytics page that is redirected * Update web/apps/docs/cookbook/go-echo-middleware.mdx Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> * Update web/apps/docs/cookbook/go-echo-middleware.mdx Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> * fix --------- Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> * fix: deploy quick fixes (#5085) * fix: fetch correct deployment+sentinel * fix: add missing team switcher hover indicator * refactor: use the same empty text * fix: lock network view and fix generate dummy network * fix: safari rendering issue of network * chore: fmt * fix: build * [autofix.ci] apply automated fixes --------- Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com> Co-authored-by: Flo <53355483+Flo4604@users.noreply.github.com> Co-authored-by: James P <james@unkey.com> * docs: remove duplicate onboarding page (#5035) Remove quickstart/onboarding/onboarding-api.mdx which duplicates content from the new quickstart. Redirects already exist in docs.json. Generated-By: mintlify-agent Co-authored-by: mintlify[bot] <109931778+mintlify[bot]@users.noreply.github.com> Co-authored-by: Andreas Thomas <dev@chronark.com> Co-authored-by: James P <james@unkey.com> * docs: remove deprecated Vercel integration page (#5034) The Vercel integration is currently not supported. Remove the page to avoid confusing users. Generated-By: mintlify-agent Co-authored-by: mintlify[bot] <109931778+mintlify[bot]@users.noreply.github.com> Co-authored-by: James P <james@unkey.com> * fix: schema cache (#5116) * fix: do background gossip connect (#5119) * fix: do background gossip connect * bazel happy * chore: debug wan failures (#5124) * chore: debug wan failures * add log writer * bazel .......... * bazel .......... * fix: a user cannot click outside of the org selection modal (#5031) * fix: a user cannot click outside of the org selection modal * use errorMessage instead of hard coding messages * restore x closing functionality * fix rabbit, fix flash of empty state * clear last used workspace when auto-selection fails * remove unused conditional --------- Co-authored-by: James P <james@unkey.com> * sentinel prewarm cache (#5071) * fix: cleanup project side nav * feat: simplify deployment overview page only show build logs until it's built, then show domains and network * feat: sentinels prewarm their cache it's not optmized, but pretty neat --------- Co-authored-by: Flo <53355483+Flo4604@users.noreply.github.com> * fix: ignore empty wan (#5122) Co-authored-by: Andreas Thomas <dev@chronark.com> Co-authored-by: James P <james@unkey.com> * docs: move docs (#5125) * refactor: deploy settings tanstack (#5104) * refactor: move them to tanstack * refactor: tidy up * feat: add env provider to decide what env we are on * refactor: tidy * feat: add scroll into view for settingcard * fix: bg * refactor: remove toasts from env-vars * chore: tidy * fix: build * feat: vault bulk en/decrypt (#5127) * feat: vault bulk en/decrypt * oops wrong file * cleanup proto * [autofix.ci] apply automated fixes * cleanup * cleanup --------- Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com> * feat: trace generated rpc clients (#5128) * feat: trace generated rpc clients * ignore not found * fix: docs generator paths (#5136) * fix: retry memberlist creation (#5134) * fix: retry memberlist creation * remove comments * move to const * fix: Restore filtering on logs (#5138) * Restore filtering on logs Restores filtering on the logs. * [autofix.ci] apply automated fixes * fmt --------- Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com> * fix: resolve dns to ip (#5139) * fix: resolve dns to ip * rabbit comments * Fix/issue 5132 billing section widths (#5140) * fix: Billing page has inconsistent section widths (#5132) Standardized all SettingCard components to use consistent width classes: - Updated Usage component: contentWidth changed from 'w-full lg:w-[320px]' to 'w-full' - Updated CancelAlert component: contentWidth changed from 'w-full lg:w-[320px]' to 'w-full' - Updated Billing Portal in client.tsx: contentWidth changed from 'w-full lg:w-[320px]' to 'w-full' - Updated CurrentPlanCard component: removed min-w-[200px] from className for consistency All billing sections now use contentWidth='w-full' for consistent layout. Fixes #5132 * Fix billing and setting cards * [autofix.ci] apply automated fixes --------- Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com> * pnpm i * No virtualization * pagination footer * sorting and pagination changes * refactor * exports * move data-table to ui * install * fmt * Footer style changes * sorting and pagination changes * sorting and footer loading fix * cleanup * prefetch pages * changes for review comments from rabbit and meg * Ref fix and removed not needed import * [autofix.ci] apply automated fixes * sorting fix and key navigation * review changes mess * minor rabbit changes * Update loading-indicator animation delay * style change on pageination footer * ref type change --------- Co-authored-by: Andreas Thomas <dev@chronark.com> Co-authored-by: Meg Stepp <mcstepp@users.noreply.github.com> Co-authored-by: Flo <53355483+Flo4604@users.noreply.github.com> Co-authored-by: Oz <21091016+ogzhanolguncu@users.noreply.github.com> Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com> Co-authored-by: James P <james@unkey.com> Co-authored-by: mintlify[bot] <109931778+mintlify[bot]@users.noreply.github.com> Co-authored-by: gui martins <guilhermev2huehue@gmail.com> Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> Co-authored-by: Amp <amp@ampcode.com> Co-authored-by: Vansh Malhotra <vansh.malhotra439@gmail.com> Co-authored-by: Flo <flo@unkey.com>

What does this PR do?
This adds our first sentinel middleware engine (key verifications) with matchers etc what we need ykyk.
fixes error page.
Type of change
How should this be tested?
make devmanually add sentinel config cuz no ui
replace env_id with the env u wanna deploy and the key auth id with a created api key auth id! NOT THE API ID.
deploy
run
make tunnelcurl -H "Authorization: Bearer <REPLACE>" https://local-api-local.unkey.local/or w/e your local url isit should 401 you
replace with created key and it should not.
I used https://github.com/Flo4604/mono-repo-test to test it so feel free to use it with the /protected route it dumps out the principal header.
Checklist
Required
pnpm buildpnpm fmtmake fmton/godirectoryconsole.logsgit pull origin mainAppreciated