feat(semantic): field catalog — types from ClickHouse + authored roles (#2208)

docs/domain/semantic-layer/specs/DESIGN.md

@@ -282,6 +282,10 @@ Definitions live in application-owned storage so they can be authored, versioned
 
 Every dataset publishes a typed, role-annotated catalog of its fields — the editor's palette, the compiler's validation universe, and the discovery API's vocabulary at once. It is what makes capability a projection of definitions rather than data.
 
+**Implementation status (#2208, Phase 1 slice 2):** the build-time catalog is shipped in `domain/field_catalog/` (Rust). Because the dbt `schema.yml` files carry no `meta:` type/role annotations, the source is **hybrid**: column **types** come from a committed ClickHouse type snapshot (`types.snapshot.json`, generated from `system.columns` and kept honest by an ignored drift test against the live warehouse), and **roles** are authored in `roles.yaml`. The generator joins the two and rejects any role that names an absent column or an unmodeled type, so the embedded catalog is a consistent validation universe at build time (offline tests) with no live warehouse needed in CI. Inspect via `analytics field-catalog`.
+
+Coverage is the source families with a clean, self-contained entity and tenant at silver — **git, ai, collab, wiki** (12 datasets). Deferred to the extraction slice (slice 5), where roles are verified against real measures: **task** (`class_task_worklogs`/`comments` carry no tenant column and key on `author_id`, not an email — needs a tenant join and identity resolution), the reference/join relations with no direct entity (repository/branch, git PR comment/commit/reviewer, task statuses/projects/sprints/field-metadata, wiki engagement), and derived populations (active-day, focus). Two cross-family facts the catalog already absorbs: the tenant column name is **not** uniform (`tenant_id` for git/collab/wiki, `insight_tenant_id` for ai), named per dataset in `roles.yaml`; and `Decimal(p,s)` amount columns are a first-class field type distinct from `Float`.
+
 ##### Responsibility scope
 
 - Generated from dataset schemas (a Rust build-time generator sharing the backend's definition parsers, sourced from dbt `schema.yml` `meta:` blocks), never hand-maintained. Roles: entity, dimension, measurable, event time.

src/backend/services/analytics/src/domain/field_catalog/live_tests.rs

@@ -0,0 +1,91 @@
+//! Drift test: the committed type snapshot (via the built catalog) must still
+//! agree with the live ClickHouse schema. For every catalogued field, the
+//! warehouse's column type — normalized — must equal the field's type in the
+//! catalog. A mismatch means `types.snapshot.json` is stale and must be
+//! regenerated from ClickHouse.
+//!
+//! Ignored by default; requires a live ClickHouse. Enable with
+//! `INTEGRATION_TESTS_CLICKHOUSE_URL=<url> cargo test -p analytics -- --ignored
+//! field_catalog`.
+
+use std::collections::BTreeMap;
+
+use clickhouse::Row;
+use serde::Deserialize;
+
+use crate::domain::field_catalog::field_catalog;
+use crate::domain::field_catalog::model::FieldType;
+
+const CH_ENV: &str = "INTEGRATION_TESTS_CLICKHOUSE_URL";
+const CH_USER_ENV: &str = "INTEGRATION_TESTS_CLICKHOUSE_USER";
+const CH_PASSWORD_ENV: &str = "INTEGRATION_TESTS_CLICKHOUSE_PASSWORD";
+
+#[derive(Debug, Row, Deserialize)]
+struct ColumnRow {
+    name: String,
+    #[serde(rename = "type")]
+    ch_type: String,
+}
+
+#[tokio::test]
+#[ignore = "requires live ClickHouse; set INTEGRATION_TESTS_CLICKHOUSE_URL to enable"]
+async fn types_snapshot_matches_clickhouse() {
+    let Ok(url) = std::env::var(CH_ENV) else {
+        eprintln!("skipping: {CH_ENV} not set");
+        return;
+    };
+    let mut config = insight_clickhouse::Config::new(&url, "default");
+    if let (Ok(user), Ok(password)) = (std::env::var(CH_USER_ENV), std::env::var(CH_PASSWORD_ENV)) {
+        config = config.with_auth(user, password);
+    }
+    let client = insight_clickhouse::Client::new(config);
+
+    let mut drift: Vec<String> = Vec::new();
+
+    for dataset in &field_catalog().datasets {
+        let rows = client
+            .query("SELECT name, type FROM system.columns WHERE database = ? AND table = ?")
+            .bind(dataset.database.as_str())
+            .bind(dataset.table.as_str())
+            .fetch_all::<ColumnRow>()
+            .await
+            .unwrap_or_else(|e| panic!("querying columns of {}: {e}", dataset.relation()));
+
+        let live: BTreeMap<String, String> =
+            rows.into_iter().map(|r| (r.name, r.ch_type)).collect();
+
+        if live.is_empty() {
+            drift.push(format!(
+                "{}: relation absent from ClickHouse",
+                dataset.relation()
+            ));
+            continue;
+        }
+
+        for field in &dataset.fields {
+            match live.get(&field.name) {
+                None => drift.push(format!(
+                    "{}.{}: column absent from ClickHouse",
+                    dataset.relation(),
+                    field.name
+                )),
+                Some(ch_type) => match FieldType::normalize(ch_type) {
+                    Some((ty, nullable)) if ty == field.ty && nullable == field.nullable => {}
+                    other => drift.push(format!(
+                        "{}.{}: catalog has {:?} (nullable={}), ClickHouse has {ch_type} -> {other:?}",
+                        dataset.relation(),
+                        field.name,
+                        field.ty,
+                        field.nullable
+                    )),
+                },
+            }
+        }
+    }
+
+    assert!(
+        drift.is_empty(),
+        "field-catalog type snapshot is stale — regenerate types.snapshot.json from ClickHouse:\n{}",
+        drift.join("\n")
+    );
+}