feat(ingestion): add Figma connector (ui-design, bronze-only) - #1306
Merged
Conversation
…te-schema.sh Both scripts live in tools/declarative-connector/ but resolved INGESTION_DIR as SCRIPT_DIR/.., which points at tools/ and breaks every invocation (./tools/tools/declarative-connector/source.sh not found). Resolve two levels up instead. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: Roman Mitasov <Roman.Mitasov@constructor.tech>
Nocode declarative connector for the Figma REST API v1. Five raw
streams into bronze_figma: design_projects, design_files (client-side
incremental on last_modified), design_file_meta, design_file_versions
(start-date bounded), design_file_comments. dbt step is RMT promotion
only — Silver (class_design_activity) is a follow-up.
API facts verified against figma/rest-api-spec and encoded in the
manifest:
- no team or member enumeration endpoints (team IDs are required
config; User objects carry id/handle only, never email — identity
resolution deferred to dbt via handle matching or Enterprise SCIM)
- per-endpoint rate-limit tiers bound to the token owner's seat type;
the Tier 1 full-document endpoint is deliberately not used
- 429 honours Retry-After (600s cap); file-level 403/404 are IGNOREd
(invite-only projects, deleted files), team-level ones FAIL the run
Manifest is fully inlined (no $refs except schemas) and passes both
validate-strict and validate. Live-tested against a real workspace:
check/discover/per-stream read green, mandatory fields present,
incremental cursor advances on resume; empty versions/comments
confirmed as genuine API responses (30-day version retention on
Starter plan).
Docs: ui-design domain spec and figma.md corrected to v1.1 — the
March draft assumed a nonexistent GET /v1/teams/{id}/members endpoint
and user emails in version/comment payloads.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: Roman Mitasov <Roman.Mitasov@constructor.tech>
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (3)
🚧 Files skipped from review as they are similar to previous changes (3)
📝 WalkthroughWalkthroughThis PR adds a complete Figma UI design connector to the Insight platform. It defines five bronze-only streams (projects, files, metadata, versions, comments) extracted via Figma REST API v1, with identity resolution deferred to Silver and dbt-driven activity aggregation. Includes verified API specifications, declarative stream manifests with pagination and incremental sync, dbt promotion models, deployment templates, and build script fixes. ChangesFigma Connector Implementation
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~25 minutes Suggested reviewers
Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 4
🧹 Nitpick comments (2)
src/ingestion/connectors/ui-design/figma/README.md (2)
23-56: 💤 Low valueClarify that "automatically injected" fields are not user-provided.
The "Automatically injected" table (lines 51-56) documents fields that the platform provides, but their placement after the Secret template (lines 23-40) might suggest users should include them. Consider adding a brief note that these fields are NOT part of the user-provided Secret — they're injected by Insight at runtime. Alternatively, reorganize to present user-provided fields first, then platform-provided fields afterward.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/ingestion/connectors/ui-design/figma/README.md` around lines 23 - 56, The "Automatically injected" table in README.md may be mistaken for user-provided Secret fields; update the README.md (the K8s Secret example and the tables) so it's explicit these fields are platform-injected and not part of the Secret a user must create: either move the "Automatically injected" table away from the Secret YAML and add a short note above it stating that insight_tenant_id and insight_source_id are injected by Insight at runtime (not to be included in the Secret), or add a one-line parenthetical under the table header clarifying "NOT user-provided — injected by Insight (tenant_id and insight.cyberfabric.com/source-id)". Ensure the symbols insight_tenant_id and insight_source_id are mentioned exactly to help locate the content.
44-49: Align README docs with connector schema forfigma_page_size
connector.yamlincludesfigma_page_size(integer, default 50; used viaconfig.get('figma_page_size', 50)), so the README fields-table entry is accurate.- The K8s Secret example omits
figma_page_sizebecause it’s optional (default applies when not set); optional improvement: add it as a commented line for discoverability.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/ingestion/connectors/ui-design/figma/README.md` around lines 44 - 49, The README's fields-table is correct about figma_page_size but the K8s Secret example omits it; update README.md to explicitly note that figma_page_size is optional (default 50) and add a commented example line showing how to set figma_page_size in the Secret for discoverability, referencing the connector schema entry and the code usage (config.get('figma_page_size', 50)) so readers see the source of the default.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@docs/components/connectors/ui-design/figma/figma.md`:
- Around line 159-163: Add a language hint to the fenced code block that starts
with the lines "design_file_versions.author_id / author_handle → Identity
Manager name-matching (handle ≈ display name in HR/git sources) → person_id":
change the opening triple-backtick to include a language (e.g., ```text) so the
block is recognized by the linter and MD040 is resolved; update the opening
fence only (do not alter the block contents) where that snippet appears in
figma.md.
In `@docs/components/connectors/ui-design/README.md`:
- Line 60: The README currently conflicts on ownership of design_file_activity:
update the design_file_activity section and any "Populated at collection
time"/"At collection time" references to reflect that design_file_activity is a
dbt-derived (Silver layer) target model (same contract as design_users note), so
replace those collection-time statements with wording that it is produced by the
dbt pipeline/Silver derivation and mention any upstream collection holds raw
events used by dbt; ensure you update the headings and sentence(s) referring to
design_file_activity to consistently state "dbt-derived (Silver layer)" and keep
the separate note that design_users cannot be populated from the Figma REST API.
In `@src/ingestion/connectors/ui-design/figma/connector.yaml`:
- Around line 1035-1041: The figma_team_ids string allows empty partitions
(e.g., "123," or "123,,456") which leads to malformed team requests; update the
schema for the figma_team_ids field to validate input (either change its type to
an array of strings with items pattern /^\d+$/ and minItems: 1, or add a string
pattern like /^\d+(,\d+)*$/) so trailing commas and empty segments are rejected;
also ensure any consumer that currently does a split on figma_team_ids (the code
that parses this field) trims and filters empty values to avoid generating
"/v1/teams//projects" requests.
In `@src/ingestion/connectors/ui-design/figma/descriptor.yaml`:
- Around line 17-20: The descriptor.yaml incorrectly lists figma_start_date
under required_fields causing a mismatch with connector.yaml where
figma_start_date is optional with a default; remove figma_start_date from the
required_fields array (so only figma_token and figma_team_ids remain) to align
the descriptor with connector.yaml and allow deployments to use the defaulted
optional start date.
---
Nitpick comments:
In `@src/ingestion/connectors/ui-design/figma/README.md`:
- Around line 23-56: The "Automatically injected" table in README.md may be
mistaken for user-provided Secret fields; update the README.md (the K8s Secret
example and the tables) so it's explicit these fields are platform-injected and
not part of the Secret a user must create: either move the "Automatically
injected" table away from the Secret YAML and add a short note above it stating
that insight_tenant_id and insight_source_id are injected by Insight at runtime
(not to be included in the Secret), or add a one-line parenthetical under the
table header clarifying "NOT user-provided — injected by Insight (tenant_id and
insight.cyberfabric.com/source-id)". Ensure the symbols insight_tenant_id and
insight_source_id are mentioned exactly to help locate the content.
- Around line 44-49: The README's fields-table is correct about figma_page_size
but the K8s Secret example omits it; update README.md to explicitly note that
figma_page_size is optional (default 50) and add a commented example line
showing how to set figma_page_size in the Secret for discoverability,
referencing the connector schema entry and the code usage
(config.get('figma_page_size', 50)) so readers see the source of the default.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: efb62f66-7360-4119-b8fc-cd685cdc1721
📒 Files selected for processing (10)
docs/components/connectors/ui-design/README.mddocs/components/connectors/ui-design/figma/figma.mdsrc/ingestion/connectors/ui-design/figma/README.mdsrc/ingestion/connectors/ui-design/figma/connector.yamlsrc/ingestion/connectors/ui-design/figma/dbt/figma__bronze_promoted.sqlsrc/ingestion/connectors/ui-design/figma/dbt/schema.ymlsrc/ingestion/connectors/ui-design/figma/descriptor.yamlsrc/ingestion/secrets/connectors/figma.yaml.examplesrc/ingestion/tools/declarative-connector/generate-catalog.shsrc/ingestion/tools/declarative-connector/generate-schema.sh
Comment on lines
+17
to
+20
| required_fields: | ||
| - figma_token | ||
| - figma_team_ids | ||
| - figma_start_date |
There was a problem hiding this comment.
figma_start_date is incorrectly required in descriptor secret fields.
Line 20 conflicts with connector.yaml where figma_start_date is optional and defaulted. This mismatch can block deployments/configs that rely on defaults.
Suggested fix
secret:
required_fields:
- figma_token
- figma_team_ids
- - figma_start_date📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| required_fields: | |
| - figma_token | |
| - figma_team_ids | |
| - figma_start_date | |
| required_fields: | |
| - figma_token | |
| - figma_team_ids |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@src/ingestion/connectors/ui-design/figma/descriptor.yaml` around lines 17 -
20, The descriptor.yaml incorrectly lists figma_start_date under required_fields
causing a mismatch with connector.yaml where figma_start_date is optional with a
default; remove figma_start_date from the required_fields array (so only
figma_token and figma_team_ids remain) to align the descriptor with
connector.yaml and allow deployments to use the defaulted optional start date.
- validate figma_team_ids format in the spec (minLength + pattern) so trailing/double commas fail at check time instead of producing /v1/teams//projects requests - align design_file_activity derivation wording in the ui-design domain spec with the dbt/Silver contract (was: "at collection time") - add language hint to a fenced block in figma.md (MD040) figma_start_date stays in descriptor secret.required_fields by design — confluence and zoom require their start_date the same way (explicit deployment config over silent defaults). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: Roman Mitasov <Roman.Mitasov@constructor.tech>
cyberantonz
approved these changes
Jun 15, 2026
mitasovr
pushed a commit
to mitasovr/insight
that referenced
this pull request
Jun 15, 2026
The publish-chart job both patch-bumps the umbrella `version` and rewrites `ingestion.toolboxImage` in charts/insight/values.yaml from the tree checked out at this run's trigger SHA. When two version-bumping PRs merge back-to-back — especially two that touch src/ingestion/** and so both rebuild the toolbox — the second run's trigger SHA does not yet contain the first run's `chore(release)` commit, so both runs: * compute the SAME next umbrella version, and * rewrite the SAME `ingestion.toolboxImage` line to their own build tag. The first run pushes its release commit; the second run's commit-back is then rejected (non-fast-forward). The retry's `git pull --rebase` hits a conflict in charts/insight/values.yaml — specifically on the `ingestion.toolboxImage` line (the version line auto-merges because both runs wrote the same value) — and aborts -> exit 1. A plain job re-run repeats this forever because `checkout` re-pins the same stale SHA. Secondary symptom: both runs `helm push` the same chart version with different contents, so the chart published to GHCR diverges from what main records as that version. Fix: re-anchor the working tree to the live branch tip (`git fetch` + `git reset --hard origin/$GITHUB_REF_NAME`) before any value is computed, so version-compute, the toolboxImage ref and the commit-back are all consistent with the branch tip — and a re-run recomputes against the refreshed tip instead of replaying the stale SHA. Combined with the existing per-ref `concurrency` serialisation the final push fast-forwards, so the fragile rebase-retry is replaced with a fail-loud guard. Reproduced by run 27535189783 (constructorfabric#1306 Figma then constructorfabric#1305 Workday merged back-to-back, both touching src/ingestion/**). Note: bump-descriptors shares the same rebase-retry pattern; left untouched here to keep this fix scoped to the reproduced failure. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
mitasovr
added a commit
that referenced
this pull request
Jun 15, 2026
The publish-chart job both patch-bumps the umbrella `version` and rewrites `ingestion.toolboxImage` in charts/insight/values.yaml from the tree checked out at this run's trigger SHA. When two version-bumping PRs merge back-to-back — especially two that touch src/ingestion/** and so both rebuild the toolbox — the second run's trigger SHA does not yet contain the first run's `chore(release)` commit, so both runs: * compute the SAME next umbrella version, and * rewrite the SAME `ingestion.toolboxImage` line to their own build tag. The first run pushes its release commit; the second run's commit-back is then rejected (non-fast-forward). The retry's `git pull --rebase` hits a conflict in charts/insight/values.yaml — specifically on the `ingestion.toolboxImage` line (the version line auto-merges because both runs wrote the same value) — and aborts -> exit 1. A plain job re-run repeats this forever because `checkout` re-pins the same stale SHA. Secondary symptom: both runs `helm push` the same chart version with different contents, so the chart published to GHCR diverges from what main records as that version. Fix: re-anchor the working tree to the live branch tip (`git fetch` + `git reset --hard origin/$GITHUB_REF_NAME`) before any value is computed, so version-compute, the toolboxImage ref and the commit-back are all consistent with the branch tip — and a re-run recomputes against the refreshed tip instead of replaying the stale SHA. Combined with the existing per-ref `concurrency` serialisation the final push fast-forwards, so the fragile rebase-retry is replaced with a fail-loud guard. Reproduced by run 27535189783 (#1306 Figma then #1305 Workday merged back-to-back, both touching src/ingestion/**). Note: bump-descriptors shares the same rebase-retry pattern; left untouched here to keep this fix scoped to the reproduced failure. Co-authored-by: Roman Mitasov <Roman.Mitasov@constructor.tech> Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
New nocode declarative connector for the Figma REST API v1 — first connector in the
ui-designcategory. Bronze-only by design: five raw streams land inbronze_figma, the dbt step is RMT promotion only (figma__bronze_promoted). Silver (class_design_activity) is a deliberate follow-up.design_projectsGET /v1/teams/{id}/projects(per configured team)design_filesGET /v1/projects/{id}/fileslast_modifieddesign_file_metaGET /v1/files/{key}/meta(creator, last_touched_by, editor_type)design_file_versionsGET /v1/files/{key}/versions(paginated, bounded byfigma_start_date)design_file_commentsGET /v1/files/{key}/comments(replies viaparent_comment_id)API constraints encoded in the manifest
Verified against the official OpenAPI spec (figma/rest-api-spec, June 2026):
figma_team_idsis required config (taken from team URLs);Userobjects carry onlyid/handle, never email. Identity resolution is deferred to dbt (handle matching via Identity Manager, or Enterprise SCIM later).GET /v1/files/{key}, 6 req/month on viewer seats) is deliberately not used — design content never leaves Figma.Retry-After(600 s cap, confluence pattern); file-level 403/404 are IGNOREd (invite-only projects, deleted files are routine), team-level ones FAIL the run (bad token/team id).$refsexcept schemas — inline substream parents, inline auth) and passes bothvalidate-strictandvalidate, including the stricter post-drift:latestCDK image.Docs
docs/components/connectors/ui-design/README.mdandfigma/figma.mdcorrected to v1.1: the March draft assumed a nonexistentGET /v1/teams/{id}/membersendpoint and user emails in version/comment payloads. The two large PRDs underui-design/*/specs/still carry the stale assumption — flagged for a separate pass.Tooling fix (first commit)
generate-catalog.shandgenerate-schema.shresolvedINGESTION_DIRasSCRIPT_DIR/..while living intools/declarative-connector/— every invocation failed withtools/tools/...path. Fixed to../...Test plan
source.sh validate-strict ui-design/figma— Builder-UI compatiblesource.sh validate ui-design/figma— CDK runtime validcheckagainst a live workspace — credentials/team id OKdiscover— 5 streams,design_filesincremental withlast_modifiedcursorread— 0 errors, every record carriestenant_id/source_id/unique_key;extra_fieldsdenormalization (project_name,team_id) confirmed workinglast_modified); boundary record re-emission deduped by RMTversions/commentscross-checked against the raw API (genuine — 30-day version retention on Starter plan, untouched file)🤖 Generated with Claude Code
Summary by CodeRabbit
New Features
Documentation
Chores