fix(schema): make the lenient read set open unknown fields at every depth

CLAUDE.md

@@ -136,7 +136,8 @@ The five kinds are the cross-plane taxonomy (cp-admin.yaml `kind`); this repo's
 - The per-target invariant (`crates/aisix-proxy/AGENTS.md`: "a per-model gate binds each target") is written around `resolve_attempt_models` — the routing-group trunk. **Ensemble panel/judge (`ProxyModelCaller::call`, the streaming judge) and semantic targets (`semantic::resolve`) bypass that trunk**, so a gate wired only into the trunk is silently absent there (the 2026-08 audit found member IP allowlist, health consumption, and retries all missing on the semantic path for exactly this reason — #958). A new per-target gate must be wired into the sub-dispatch paths too, or explicitly deferred with a filed issue. Prefer routing every dispatch through one shared chokepoint so the family can't drift.
 - **Strict writes, lenient loads.** `model_one_of` has two variants: the **strict** schema (declarative resources file, the published `schemas/resources/model.schema.json`, every strict validator consumer) forbids a knob a kind never resolves — accepted-but-unread config is the #962 class; the **lenient** loader keeps the base XOR so stored rows written by an older build still load, with `Model::strip_kind_inapplicable` dropping the dead knob and reporting it as `inapplicable:<field>` through the partial-compat channel. The two lists MUST mirror each other exactly (strict-forbidden ⇔ lenient-stripped) — a field forbidden-but-not-stripped half-honors; stripped-but-not-forbidden vanishes on load while the write path accepts it. A knob is enforced exactly as written or rejected, never half-honored (#963).
 - **Never make a field of a projected resource required at the TYPE level.** Requiredness belongs in the strict schema (`require_property` in `models/schema.rs`), never in the struct: the loader validates leniently and then deserializes, and a row it cannot deserialize is **skipped entirely** (`aisix-etcd/src/loader.rs`). Skipping is survivable for a resource the request path treats as optional, but an `api_key` row that fails to load stops authenticating **every** kind of traffic, not just the feature whose field changed — a far worse outcome than the field defaulting. So a new non-`Option` field, or one that loses `#[serde(default)]`, silently turns every already-projected row into a dead one. Give it a serde default whose meaning is fail-closed, and add it to `required` in the strict schema so the write path still refuses to guess. The control plane must also re-emit the affected collection once (`ReprojectMcpAclOnce` is the pattern) — the stored shape changed, but nothing else re-projects a row whose *content* did not. (Lesson from #993: `allow` was required at the type level in #992, which made every key still projected as `mcp_access: {"mode": "inherit"}` unloadable.)
-- **Never change a projected field's shape or value domain in place — the previous release must keep loading the row.** The supported upgrade order is control plane first, then data planes, with an arbitrarily long window; a released DP is immutable, so whatever the new CP projects must still parse one release back. Unknown *fields* are tolerated by design (serde_ignored → partial-compat report), but a malformed *known* field — a reshape, a lost default, a new enum value — fails the row and the loader skips it whole (the blast radius of the rule above). A reshape therefore ships under a NEW field name (the old one is never reused), or as a **same-name dual-generation document** when the old and new keys don't collide: the CP emits one document valid for both generations, and this side carries a consumed-and-ignored tombstone for the old selector — `McpAccess::legacy_mode` is the template: `#[serde(default, rename = "...", skip_serializing)]` + `#[schemars(skip)]` so the strict write path still rejects it, with a `COMPAT-SINCE:` marker naming the retirement condition (see the next section). A new enum value in an existing field cannot be made safe DP-side at all (lenient parsing keeps enums closed — it row-kills every older DP), so the paired CP PR must gate it behind `dpCompatGate` until the fleet minimum reads it. Whenever new semantics are invisible to the old release, verify the old default direction there: fail-closed or no-op is required; if it is fail-open, the CP must project an old-shape tombstone at the most restrictive value.
+- **Never change a projected field's shape or value domain in place — the previous release must keep loading the row.** The supported upgrade order is control plane first, then data planes, with a window that is minutes long in practice but must still be survived; a released DP is immutable, so whatever the new CP projects must still parse one release back. Unknown *fields* are tolerated by design at every depth, nested config objects included, and reported through the partial-compat channel — but only from the release that opened the read schema all the way down (#1014); a 0.10.0 or older binary drops the whole row for a field added inside a guardrail or exporter config block, and always will, because a released binary cannot be taught anything. A malformed *known* field — a reshape, a lost default, a new enum value — fails the row and the loader skips it whole on every release (the blast radius of the rule above). Two constraints keep the tolerance real on this side: unknown-field strictness lives in the strict schema (`models/schema.rs`), never in a nested `#[serde(deny_unknown_fields)]` — the loader deserializes the same types and cannot opt out of a type-level guard; and `serde_ignored` never fires inside serde-buffered content (a `#[serde(flatten)]`ed tagged enum, an untagged variant), so those resources take their report from `unknown_field_paths` instead of loading silently. A reshape therefore ships under a NEW field name (the old one is never reused), or as a **same-name dual-generation document** when the old and new keys don't collide: the CP emits one document valid for both generations, and this side carries a consumed-and-ignored tombstone for the old selector — `McpAccess::legacy_mode` is the template: `#[serde(default, rename = "...", skip_serializing)]` + `#[schemars(skip)]` so the strict write path still rejects it, with a `COMPAT-SINCE:` marker naming the retirement condition (see the next section). A new enum value in an existing field cannot be made safe DP-side at all (lenient parsing keeps enums closed — it row-kills every older DP), so the paired CP PR must gate it behind `dpCompatGate` until the fleet minimum reads it. Whenever new semantics are invisible to the old release, verify the old default direction there: fail-closed or no-op is required; if it is fail-open, the CP must project an old-shape tombstone at the most restrictive value.
+- **Size the mitigation to what the window costs.** The window is minutes, not a season — a control plane and its data planes are upgraded back to back — and anything that merely degrades in it self-heals on the next DP build: a knob not yet honoured, a field reported instead of read, a resource serving its previous semantics. That does not earn a dual-generation document, a tombstone, or holding a finished feature back a release; ship it and let the CP warn at save time (`dpCompatGate`). Three things do not self-heal, and keep the heavy technique regardless of how short the window is: a run-once migration, which rewrites stored data permanently; a security control the old binary cannot see, where the exposure has already happened however briefly (fail-closed or an old-shape tombstone, never bare degradation); and a **core carrier row** — `api_keys`, models, provider keys — where a skipped row is not a degradation but an outage, since that key authenticates nothing at all until the DP is upgraded. If a change is none of the three, the proportionate mitigation is the report.
 - **`ensemble` is an experimental surface.** Its known parity gaps — member `allowed_cidrs`/guardrail/cooldown/health consumption, Prometheus token+spend attribution, response caching, parent-level generic knobs — are deliberate TODOs under a single future design pass. Do NOT piecemeal-fix one gap ahead of that pass, and do NOT re-audit them as fresh findings. (The one exception is a marshal-family or shared-chokepoint change where covering ensemble is a one-line parallel edit, e.g. projecting an entry-level field the DP already enforces.)
 - Adding a NEW kind = sweeping every existing model-keyed mechanism against it (grep the kind predicates in `models/model.rs`; every hit re-answers the questions above).
 

crates/aisix-core/src/bin/dump-schema.rs

@@ -49,23 +49,7 @@ fn main() {
     // `ensemble`/`rate_limit`/`routing` have no standalone validator (they
     // are nested struct types) so they dump straight from the struct via
     // `schema_for!`, closed the same way.
-    for resource in [
-        "api_key",
-        "cache_policy",
-        "model",
-        "rate_limit_policy",
-        "provider_key",
-        "observability_exporter",
-        "guardrail",
-        "guardrail_attachment",
-        "mcp_server",
-        "mcp_policy",
-        "a2a_agent",
-        "oidc_provider",
-        "claim_mapping",
-        "passthrough_route",
-        "mcp_auth_settings",
-    ] {
+    for resource in schema::RESOURCES {
         dump_value(
             &out_dir,
             resource,

crates/aisix-core/src/models/guardrail.rs

@@ -78,7 +78,6 @@ pub enum KeywordPattern {
 
 /// Config block for `kind: "keyword"`.
 #[derive(Debug, Clone, Default, Serialize, Deserialize, schemars::JsonSchema, PartialEq, Eq)]
-#[serde(deny_unknown_fields)]
 pub struct KeywordConfig {
     /// Blocklist patterns. An empty list is valid and allows every
     /// request, equivalent to `enabled: false`.
@@ -122,7 +121,6 @@ pub enum BedrockLatencyMode {
 /// projection time so the DP always holds plaintext in memory. The
 /// key is never logged.
 #[derive(Debug, Clone, Serialize, Deserialize, schemars::JsonSchema, PartialEq, Eq)]
-#[serde(deny_unknown_fields)]
 pub struct AzureContentSafetyConfig {
     /// Azure Cognitive Services resource endpoint, e.g.
     /// `https://my-resource.cognitiveservices.azure.com`.
@@ -158,7 +156,6 @@ fn default_acs_timeout_ms() -> u32 {
 /// default. Only `api_key` is a secret (decrypted by cp-api before kine
 /// projection). Every other field travels in the clear.
 #[derive(Debug, Clone, Serialize, Deserialize, schemars::JsonSchema, PartialEq, Eq)]
-#[serde(deny_unknown_fields)]
 pub struct AzureContentSafetyTextModerationConfig {
     /// Azure Cognitive Services resource endpoint. The gateway appends
     /// `/contentsafety/text:analyze?api-version=2024-09-01`.
@@ -282,7 +279,6 @@ fn default_acs_on_buffer_exceeded() -> String {
 /// projection. It is plaintext in DP memory only and never logged). Every other
 /// field travels in the clear.
 #[derive(Debug, Clone, Serialize, Deserialize, schemars::JsonSchema, PartialEq, Eq)]
-#[serde(deny_unknown_fields)]
 pub struct AliyunTextModerationConfig {
     /// Aliyun region the guardrail lives in, e.g. `cn-shanghai`. The gateway
     /// builds the endpoint `https://green-cip.<region>.aliyuncs.com`.
@@ -361,7 +357,6 @@ fn default_aliyun_risk_level_threshold() -> String {
 /// projection. It is plaintext in DP memory only and never logged). Every
 /// other field travels in the clear.
 #[derive(Debug, Clone, Serialize, Deserialize, schemars::JsonSchema, PartialEq, Eq)]
-#[serde(deny_unknown_fields)]
 pub struct AliyunAiGuardrailConfig {
     /// Aliyun region the guardrail lives in, e.g. `cn-shanghai`. The gateway
     /// builds the endpoint `https://green-cip.<region>.aliyuncs.com`.
@@ -423,7 +418,6 @@ fn default_aliyun_aig_service_level() -> String {
 /// detector to enable; `action` optionally overrides the guardrail-level
 /// `default_action` for this detector only.
 #[derive(Debug, Clone, Serialize, Deserialize, schemars::JsonSchema, PartialEq, Eq)]
-#[serde(deny_unknown_fields)]
 pub struct PiiDetectorConfig {
     /// Built-in detector to enable for this PII guardrail entry.
     #[serde(rename = "type")]
@@ -437,7 +431,6 @@ pub struct PiiDetectorConfig {
 
 /// One operator-supplied regex detector for `kind: "pii"`.
 #[derive(Debug, Clone, Serialize, Deserialize, schemars::JsonSchema, PartialEq, Eq)]
-#[serde(deny_unknown_fields)]
 pub struct PiiCustomPattern {
     /// Detector name surfaced in the mask token (`[<NAME>_REDACTED]`),
     /// telemetry counts, and block reasons. Never the matched value.
@@ -478,7 +471,6 @@ pub struct PiiCustomPattern {
 /// content-filter envelope. Matched values never appear in logs, telemetry,
 /// or error envelopes — only detector names and match counts do.
 #[derive(Debug, Clone, Serialize, Deserialize, schemars::JsonSchema, PartialEq, Eq)]
-#[serde(deny_unknown_fields)]
 pub struct PiiConfig {
     /// Built-in detectors to enable. The resource schema rejects unknown
     /// detector ids, so a typo cannot silently disable the policy.
@@ -521,7 +513,6 @@ fn default_aliyun_window_overlap_size() -> u32 {
 /// `aisix-guardrails::BedrockGuardrail` from this and dispatches
 /// `ApplyGuardrail` on every governed request.
 #[derive(Debug, Clone, Serialize, Deserialize, schemars::JsonSchema, PartialEq, Eq)]
-#[serde(deny_unknown_fields)]
 pub struct BedrockConfig {
     /// Guardrail identifier issued by the AWS console.
     #[schemars(length(min = 1, max = 64))]
@@ -553,7 +544,6 @@ pub struct BedrockConfig {
 /// configuration is applied; the plaintext is held in memory only and is
 /// never logged.
 #[derive(Debug, Clone, Serialize, Deserialize, schemars::JsonSchema, PartialEq, Eq)]
-#[serde(deny_unknown_fields)]
 pub struct LakeraConfig {
     /// Lakera API key sent as a `Authorization: Bearer` header. Decrypted
     /// before projection. Plaintext is held in memory only and is not logged.
@@ -604,7 +594,6 @@ pub struct LakeraConfig {
 /// configuration is applied; the plaintext is held in memory only and is
 /// never logged.
 #[derive(Debug, Clone, Serialize, Deserialize, schemars::JsonSchema, PartialEq)]
-#[serde(deny_unknown_fields)]
 pub struct OpenaiModerationConfig {
     /// OpenAI API key sent as a `Authorization: Bearer` header. Decrypted
     /// before projection. Plaintext is held in memory only and is not logged.
@@ -649,7 +638,6 @@ fn default_openai_moderation_model() -> String {
 /// `action` optionally overrides the guardrail-level `default_action` for
 /// this entity only — the same per-detector shape as `kind: "pii"`.
 #[derive(Debug, Clone, Serialize, Deserialize, schemars::JsonSchema, PartialEq, Eq)]
-#[serde(deny_unknown_fields)]
 pub struct PresidioEntityConfig {
     /// Presidio entity type, e.g. `EMAIL_ADDRESS`, `PERSON`, `US_SSN`.
     #[serde(rename = "type")]
@@ -671,7 +659,6 @@ pub struct PresidioEntityConfig {
 /// compliance posture, and selectable anonymize operators. No vendor secret:
 /// both URLs point at customer-run containers.
 #[derive(Debug, Clone, Serialize, Deserialize, schemars::JsonSchema, PartialEq)]
-#[serde(deny_unknown_fields)]
 pub struct PresidioConfig {
     /// Presidio analyzer base URL, e.g. `http://presidio-analyzer:3000`.
     /// The gateway appends `/analyze`.
@@ -737,6 +724,14 @@ fn default_presidio_language() -> String {
 
 /// Provider discriminator. The kind drives which `*_config` block is
 /// expected. Serde's `tag = "kind"` keeps us honest at parse time.
+///
+/// The per-kind configs deliberately carry no `#[serde(deny_unknown_fields)]`:
+/// `Guardrail` flattens this enum, so a type-level guard here also applies to
+/// the etcd loader, which deserialises the same types — and a guardrail row
+/// dropped for one unknown field is a content policy that silently stops
+/// enforcing. Unknown-field strictness belongs to the write path and lives in
+/// the branch and definition closures in
+/// [`guardrail_root_schema`](crate::models::schema::guardrail_root_schema).
 #[derive(Debug, Clone, Serialize, Deserialize, schemars::JsonSchema, PartialEq)]
 #[serde(tag = "kind", rename_all = "snake_case")]
 pub enum GuardrailKind {
@@ -1120,30 +1115,34 @@ mod tests {
     }
 
     #[test]
-    fn unknown_field_rejected_by_inner_kind_struct() {
-        // The outer Guardrail can't use deny_unknown_fields (see its
-        // doc comment), but the inner KeywordConfig does — and serde
-        // surfaces unknown fields from the flattened inner type at
-        // the top level. Net effect: typos are still caught.
+    fn unknown_field_is_rejected_on_write_and_kept_on_read() {
+        // The typo guard used to be `deny_unknown_fields` on the config
+        // structs, which sat in the TYPE — so it also applied to the etcd
+        // loader, which deserialises the same types, and killed the whole
+        // row. It now lives in the strict schema: `aisix validate` and the
+        // file source still refuse the typo, while a stored document keeps
+        // loading with the field ignored.
         let v = json!({
             "name": "g",
             "kind": "keyword",
             "patterns": [],
             "extra": "nope"
         });
-        let r: Result<Guardrail, _> = serde_json::from_value(v);
-        assert!(r.is_err());
+        assert!(crate::models::validate_guardrail(&v).is_err());
+        assert!(crate::models::validate_guardrail_lenient(&v).is_ok());
+        let g: Guardrail = serde_json::from_value(v).expect("the read path keeps the row");
+        assert_eq!(g.name, "g");
     }
 
     #[test]
-    fn p0c_fields_dont_trip_keyword_config_deny_unknown_fields() {
-        // `KeywordConfig` has `deny_unknown_fields`. The P0c fields
-        // (`enforcement_mode`, `mandatory`, `direction`) are declared on
-        // the outer `Guardrail` struct with `#[serde(default)]`, so serde
-        // absorbs them at the outer level before the flattened inner sees
-        // the remaining fields. This test pins that routing: if any of
-        // these fields accidentally reached `KeywordConfig`, the parse
-        // would return an unknown-field error.
+    fn p0c_fields_travel_past_the_flattened_kind() {
+        // The P0c fields (`enforcement_mode`, `mandatory`, `direction`) are
+        // declared on the outer `Guardrail` struct with `#[serde(default)]`,
+        // so serde absorbs them at the outer level before the flattened
+        // inner kind sees the remaining fields. The strict schema owes them
+        // the same passage from the other direction: a closed kind branch
+        // lists only its own kind's fields, so the root properties are
+        // copied in (`guardrail_root_schema`) or every real document fails.
         let v = json!({
             "name": "g",
             "kind": "keyword",
@@ -1152,8 +1151,9 @@ mod tests {
             "mandatory": true,
             "direction": "input"
         });
-        let g: Guardrail = serde_json::from_value(v)
-            .expect("P0c fields must not trip KeywordConfig deny_unknown_fields");
+        assert!(crate::models::validate_guardrail(&v).is_ok());
+        let g: Guardrail =
+            serde_json::from_value(v).expect("outer fields must not reach the inner kind");
         assert_eq!(g.enforcement_mode, "monitor");
         assert!(g.mandatory);
         assert_eq!(g.direction, "input");