feat(connector): add on-prem Active Directory (LDAP) HR connector

docs/domain/connector/HR_IDENTITY_CONNECTOR_GUIDE.md

@@ -0,0 +1,113 @@
+# HR / Identity Connector Authoring Guide (agent-facing)
+
+Companion to the canonical spec [`specs/DESIGN.md`](specs/DESIGN.md). DESIGN.md §4 is
+the **general** connector guide (manifest structure, auth, pagination, AddFields,
+schema rules, deployment). This file is the **HR / identity directory** specialization:
+the exact, repeatable recipe for a connector whose job is "pull a user directory and
+feed the Identity Manager". Read DESIGN.md first; this narrows it.
+
+> Scope guard for agents: an HR/identity connector lives under
+> `src/ingestion/connectors/hr-directory/{name}/`. Before building a new one, check
+> the three references below — the pattern is fixed; you are filling in a template,
+> not inventing structure.
+
+## Reference connectors (copy these, do not start blank)
+
+| Connector | Transport | Type | Use as template when… |
+|-----------|-----------|------|------------------------|
+| [`hr-directory/bamboohr`](../../../src/ingestion/connectors/hr-directory/bamboohr/) | REST + API key | **nocode** | Source is a REST/HTTP HR system. Adds `leave_requests`/`working_hours`/`hr_events`. |
+| [`hr-directory/ms-entra`](../../../src/ingestion/connectors/hr-directory/ms-entra/) | MS Graph REST + OAuth2 | **nocode** | Source is a cloud directory over HTTP. Minimal users-only identity connector. |
+| [`hr-directory/active-directory`](../../../src/ingestion/connectors/hr-directory/active-directory/) | LDAP/LDAPS | **cdk** | Source is **not HTTP** (LDAP, SOAP, file, DB). Plain `Stream` + custom transport. |
+
+`bamboohr` and `ms-entra` are nocode (declarative `connector.yaml`). `active-directory`
+is the canonical example of a **CDK connector for a non-HTTP source** — see below.
+
+## The HR/identity Silver contract (this is the point)
+
+Every HR/identity connector, regardless of transport, MUST emit the **same** Silver
+surface so the Identity Manager and the unified person registry work across sources:
+
+```text
+bronze_{name}.users (or .employees)        ← extraction writes here
+        │  (connector AddFields / parse: tenant_id, source_id, unique_key)
+        ▼
+{name}__bronze_promoted.sql                 ← promote_bronze_to_rmt(table, order_by='unique_key')
+        ▼
+{name}__users_snapshot.sql                  ← snapshot(...) SCD2 append-only
+        ▼
+{name}__users_fields_history.sql            ← fields_history(...) one row per field change
+        ├──────────────► {name}__identity_inputs.sql   ← identity_inputs_from_history(...)  → Identity Manager
+        └──────────────► {name}__to_class_people.sql   ← Silver Step 1 → silver.class_people
+```
+
+Shared dbt macros (already exist; do not reimplement): `promote_bronze_to_rmt`,
+`snapshot`, `fields_history`, `identity_inputs_from_history`.
+
+### `to_class_people` column contract (must match exactly)
+
+`tenant_id, source_id, unique_key, workspace_id, person_id, valid_from, valid_to,
+source, source_person_id, employee_number, display_name, first_name, last_name, email,
+job_title, department_name, org_unit_id, manager_person_id, status, employment_type,
+hire_date, termination_date, location, country, fte, custom_str_attrs (Map(String,String)),
+custom_num_attrs (Map(String,Float64)), ingested_at`.
+
+Rules learned from the references:
+- `source_person_id` = the source's **stable** id (Entra `oid`, AD `objectGUID`, BambooHR employee id). Survives renames.
+- `unique_key` in `to_class_people` = Bronze `unique_key` + `-` + `valid_from` (SCD2 grain — single ORDER BY column).
+- `email` = `coalesce(mail, userPrincipalName)` for directories.
+- `status` ∈ `active | on_leave | terminated`. Map the source's enabled/disabled or status flag.
+- Types must match `silver.class_people` exactly — `cpt-dataflow-constraint-staging-class-column-types-match`. `CAST(NULL AS Nullable(...))` for unknowns.
+
+### `identity_inputs` value types
+
+`identity_inputs_from_history(...)` emits rows with `value_type` ∈
+`id | email | employee_id | display_name | sam_account` (extend the model's
+`accepted_values` test if you add more). The **`sam_account`** signal is what reconciles
+on-prem AD (`sAMAccountName`) with cloud Entra (`onPremisesSamAccountName`) and with
+self-hosted Git/GitLab usernames — always emit it when the source has a SAM/legacy login.
+
+## CDK-for-non-HTTP recipe (the new bit `active-directory` demonstrates)
+
+DESIGN.md §4.9 says "use CDK when the declarative manifest can't express it." A non-HTTP
+source (LDAP, SOAP, JDBC, file) is the clearest such case. The CDK's `HttpStream`
+**also** assumes HTTP — so do **not** subclass it. Subclass the plain
+`airbyte_cdk.sources.streams.Stream` and drive the transport yourself:
+
+```text
+source_{name}/
+  __init__.py
+  spec.json                 # insight_tenant_id + insight_source_id required; prefix others ({name}_*)
+  source.py                 # AbstractSource: spec(), check_connection(), streams()
+  {transport}_client.py      # transport helper: connect(config), attribute allowlist, value coercion
+  streams/
+    __init__.py
+    users.py                # class X(Stream): name, primary_key, get_json_schema(), read_records()
+```
+
+Key contract points (all enforced by the references):
+- `check_connection` returns `(False, "actionable message")` — never raise. Validate
+  `insight_source_id` non-empty FIRST (empty → silent dedup collisions), then connect, then a size-limited probe.
+- Every record carries `tenant_id`, `source_id`, `unique_key` (`{tenant}-{source}-{natural_key}`), injected in the stream — there is no AddFields in CDK.
+- `get_json_schema()` per stream with `additionalProperties: true`; `unique_key`+stable-id `required`. Match the sibling nocode connector's field names so the dbt models stay parallel.
+- **Privacy by default**: fetch an explicit attribute allowlist, never the whole object. See `active_directory/ldap_client.py:USER_ATTRIBUTES`.
+- `pyproject.toml` declares the transport lib as a dependency (`ldap3` for AD). `[project.scripts]` maps `source-{name}` → `source_{name}.source:main`. Dockerfile `ENTRYPOINT ["source-{name}"]`.
+- `descriptor.yaml` `type: cdk` + `images.cdk` block (ADR-0016); `images.cdk.image: ""` until first CI build patches it.
+
+## File-by-file checklist for a new HR/identity connector
+
+1. `descriptor.yaml` — `name`, `type` (nocode|cdk), `version`, `schedule`, `workflow: sync`, `dbt_select: tag:{name}+`, `connection.namespace: bronze_{name_with_underscores}`, `secret.required_fields`. CDK adds `images.cdk`.
+2. Extraction — nocode `connector.yaml` (DeclarativeSource + AddFields) **or** CDK `source_{name}/` (AbstractSource + Stream).
+3. `dbt/{name}__bronze_promoted.sql`, `__users_snapshot.sql`, `__users_fields_history.sql`, `__identity_inputs.sql`, `__to_class_people.sql`, `schema.yml` (source + freshness + model tests).
+4. `README.md` — prerequisites, K8s Secret block + field table, streams, Silver targets.
+5. `src/ingestion/secrets/connectors/{name}.yaml.example` — Secret template (gitignored real file alongside).
+
+## Tooling note (docs vs reality)
+
+DESIGN.md §4 and the connector SKILL.md reference an `airbyte-toolkit/` directory and
+scripts like `register.sh` / `build-connector.sh`. The live tooling is under
+[`src/ingestion/reconcile-connectors/`](../../../src/ingestion/reconcile-connectors/)
+(`main.sh`, `lib/cdk-build.sh`, `lib/discover.sh`) plus
+[`src/ingestion/tools/declarative-connector/source.sh`](../../../src/ingestion/tools/declarative-connector/source.sh)
+for local nocode runs. Prefer `reconcile-connectors/main.sh` for build+register+connect
+and `run-sync.sh {name} <tenant>` for e2e. The connector-spec scripts names are stale;
+the workflow steps still apply.

src/ingestion/connectors/hr-directory/active-directory/Dockerfile

@@ -0,0 +1,10 @@
+FROM airbyte/python-connector-base:2.0.0
+
+COPY . /airbyte/integration_code
+RUN pip install /airbyte/integration_code
+
+RUN useradd --create-home --no-log-init --shell /bin/bash airbyte_user
+USER airbyte_user
+
+ENV AIRBYTE_ENTRYPOINT="source-active-directory"
+ENTRYPOINT ["source-active-directory"]

src/ingestion/connectors/hr-directory/active-directory/README.md

@@ -0,0 +1,113 @@
+# Active Directory (LDAP) Connector
+
+On-prem **Active Directory user directory** via LDAP/LDAPS (`ldap3`). Pulls the
+canonical user list — objectGUID, sAMAccountName, UPN, mail, employeeId — and feeds
+the Identity Manager so users authenticated against AD can be resolved to their
+accounts in other services (GitHub, Slack, Jira, BambooHR, …).
+
+This is a **CDK (Python)** connector, not nocode: Active Directory is queried over
+LDAP, not HTTP, so the declarative manifest framework cannot express it.
+
+**Sibling of [`../ms-entra`](../ms-entra/) (cloud Microsoft Entra ID).** Both emit the
+same `class_people` / `identity_inputs` Silver contract; only the transport differs
+(LDAP vs Microsoft Graph). Run **both** when on-prem AD is synced to Entra — the
+shared `sam_account` identity signal (`sAMAccountName` here ==
+`onPremisesSamAccountName` in ms-entra) lets the Identity Manager reconcile the same
+person across the on-prem and cloud directories.
+
+## Prerequisites
+
+1. A **read-only service account** in AD (e.g. `svc-insight`). It needs nothing beyond
+   default authenticated-user read on the user objects in scope.
+2. Network reachability from the cluster to a **domain controller** on the LDAP port
+   (636 for LDAPS, 389 for plaintext). **Use LDAPS** — a plaintext simple bind sends
+   the password in the clear.
+3. The **base DN** to search (forest root `DC=corp,DC=example,DC=com`, or a narrower OU).
+
+## K8s Secret
+
+```yaml
+apiVersion: v1
+kind: Secret
+metadata:
+  name: insight-active-directory-main          # convention: insight-{connector}-{source-id}
+  labels:
+    app.kubernetes.io/part-of: insight
+  annotations:
+    insight.cyberfabric.com/connector: active-directory   # must match descriptor.yaml name
+    insight.cyberfabric.com/source-id: active-directory-main  # passed as insight_source_id
+type: Opaque
+stringData:
+  ad_server_host: ""        # DC hostname/IP, no scheme/port (e.g. "dc1.corp.example.com")
+  ad_bind_dn: ""            # service-account DN or UPN (e.g. "svc-insight@corp.example.com")
+  ad_bind_password: ""      # service-account password (sensitive)
+  ad_search_base: ""        # base DN (e.g. "DC=corp,DC=example,DC=com")
+  # Optional:
+  # ad_port: "636"          # default 636 (LDAPS) / 389 (plaintext)
+  # ad_use_ssl: "true"      # default true
+  # ad_user_filter: "(&(objectClass=user)(objectCategory=person)(!(sAMAccountName=krbtgt)))"
+  # ad_page_size: "500"
+```
+
+### Fields
+
+| Field | Required | Description |
+|-------|----------|-------------|
+| `ad_server_host` | Yes | Domain controller hostname/IP (no scheme, no port) |
+| `ad_bind_dn` | Yes | Bind identity — full DN or UPN of a read-only account |
+| `ad_bind_password` | Yes | Bind account password (sensitive) |
+| `ad_search_base` | Yes | Base DN for the user search |
+| `ad_port` | No | LDAP port (default 636 LDAPS / 389 plaintext) |
+| `ad_use_ssl` | No | Use LDAPS/TLS (default `true`) |
+| `ad_user_filter` | No | LDAP filter for user objects (default excludes computers/contacts/krbtgt) |
+| `ad_page_size` | No | Paged-search page size (default 500; AD `MaxPageSize` is 1000) |
+
+### Automatically injected
+
+Set by `reconcile-connectors` / `connect.sh`, must NOT be in the Secret:
+
+| Field | Source |
+|-------|--------|
+| `insight_tenant_id` | `tenant_id` from tenant YAML |
+| `insight_source_id` | `insight.cyberfabric.com/source-id` annotation |
+
+### Local development
+
+```bash
+cp src/ingestion/secrets/connectors/active-directory.yaml.example src/ingestion/secrets/connectors/active-directory.yaml
+# Fill in real values, then apply:
+kubectl apply -f src/ingestion/secrets/connectors/active-directory.yaml
+```
+
+## Streams
+
+| Stream | Description | Sync Mode |
+|--------|-------------|-----------|
+| `users` | AD user accounts (person objects) via LDAP paged search | Full refresh |
+
+### Privacy
+
+The connector fetches an **explicit attribute allowlist** (see
+`source_active_directory/ldap_client.py:USER_ATTRIBUTES`) — only identity-resolution
+fields such as email, name, employee ID, manager, and distinguished name. It never
+reads photos, postal addresses, phone numbers, or other non-required fields even
+when the bind account could. Mirrors the ms-entra `$select` allowlist.
+
+## Silver Targets
+
+- `class_people` — unified person registry (via `active_directory__to_class_people`)
+- `identity_inputs` — identity signals for the Identity Manager (via `active_directory__identity_inputs`)
+
+## Build & deploy (CDK)
+
+```bash
+cd src/ingestion
+# Build image + register Airbyte source definition (reconcile-connectors / cdk-build.sh)
+./reconcile-connectors/main.sh                       # discovers, builds, registers, connects
+# Or per the connector skill:  /connector build hr-directory/active-directory
+./run-sync.sh active-directory <tenant>              # e2e: Airbyte sync → dbt (Bronze → Silver)
+./logs.sh -f latest
+```
+
+> objectGUID is AD's stable identity anchor: it survives renames, OU moves, and UPN
+> changes, so `source_person_id` / `unique_key` stay constant for a person's lifetime.

src/ingestion/connectors/hr-directory/active-directory/dbt/active_directory__bronze_promoted.sql

@@ -0,0 +1,22 @@
+{# -------------------------------------------------------------------------
+   Bootstrap model for Active Directory bronze → RMT promotion.
+
+   Counterpart of `ms_entra__bronze_promoted`. See ADR-0002 for the reasoning;
+   the macro `promote_bronze_to_rmt` is idempotent — already-RMT tables are
+   detected and skipped on subsequent runs.
+
+   The `users` Bronze table carries a `unique_key` column added by the connector
+   (`{tenant}-{source}-{objectGUID}`), so `order_by='unique_key'` is equivalent
+   to the natural key (AD objectGUID).
+   ------------------------------------------------------------------------- #}
+
+-- @cpt-principle:cpt-dataflow-principle-promote-bronze:p1
+{{ config(
+    materialized='view',
+    schema='staging',
+    tags=['active-directory']
+) }}
+
+{% do promote_bronze_to_rmt(table='bronze_active_directory.users', order_by='unique_key') %}
+
+SELECT 1 AS promoted

src/ingestion/connectors/hr-directory/active-directory/dbt/active_directory__identity_inputs.sql

@@ -0,0 +1,37 @@
+{{ config(
+    materialized='incremental',
+    incremental_strategy='append',
+    schema='staging',
+    tags=['active-directory', 'silver', 'silver:identity_inputs']
+) }}
+
+{# Emit identity signals from the on-prem Active Directory user directory.
+
+   `userPrincipalName` and `mail` both yield value_type='email' so the Identity
+   Manager can match a person across services regardless of which email form a
+   downstream connector recorded.
+
+   `sAMAccountName` carries the legacy AD/SAM login — the SAME value that
+   ms-entra surfaces as `onPremisesSamAccountName`. Emitting it as value_type
+   'sam_account' lets the Identity Manager reconcile this on-prem connector with
+   the cloud ms-entra connector AND with self-hosted services (Bitbucket Server,
+   GitLab self-hosted) where SAM is often the username.
+
+   `proxyAddresses` is array-valued in Bronze; the macro operates on scalar
+   history rows, so alternate addresses are added in a follow-up
+   (REC-IR-07: array-valued identity inputs) — same limitation as ms-entra.
+#}
+
+{{ identity_inputs_from_history(
+    fields_history_ref=ref('active_directory__users_fields_history'),
+    source_type='active-directory',
+    identity_fields=[
+        {'field': 'mail',              'value_type': 'email',        'value_field_name': 'bronze_active_directory.users.mail'},
+        {'field': 'userPrincipalName', 'value_type': 'email',        'value_field_name': 'bronze_active_directory.users.userPrincipalName'},
+        {'field': 'employeeId',        'value_type': 'employee_id',  'value_field_name': 'bronze_active_directory.users.employeeId'},
+        {'field': 'displayName',       'value_type': 'display_name', 'value_field_name': 'bronze_active_directory.users.displayName'},
+        {'field': 'sAMAccountName',    'value_type': 'sam_account',  'value_field_name': 'bronze_active_directory.users.sAMAccountName'},
+        {'field': 'status',            'value_type': 'status',       'value_field_name': 'bronze_active_directory.users.status'},
+    ],
+    deactivation_condition="field_name = 'status' AND new_value = 'Terminated'"
+) }}