Skip to content

fix(ingestion): read airbyte-auth-secrets from the Airbyte namespace - #1913

Closed
cyberdima wants to merge 2 commits into
mainfrom
fix/1906-reconcile-airbyte-namespace
Closed

fix(ingestion): read airbyte-auth-secrets from the Airbyte namespace#1913
cyberdima wants to merge 2 commits into
mainfrom
fix/1906-reconcile-airbyte-namespace

Conversation

@cyberdima

@cyberdima cyberdima commented Jul 24, 2026

Copy link
Copy Markdown
Contributor

What

ab_get_token read airbyte-auth-secrets from $INSIGHT_NAMESPACE, assuming Airbyte shares the app namespace. When Airbyte runs in its own namespace the Secret is absent there, the token fetch fails, and the whole reconcile run aborts before any connector is created or synced.

Change

  • airbyte.sh: new AIRBYTE_NAMESPACE env (defaults to INSIGHT_NAMESPACE, so single-namespace installs are unaffected); ab_get_token reads the Secret from it.
  • chart: new airbyte.namespace value ("" = .Release.Namespace), insight.airbyte.namespace helper, AIRBYTE_NAMESPACE rendered on the reconcile CronWorkflow, and a conditional Role/RoleBinding granting the reconcile ServiceAccount get on the Secret in the Airbyte namespace — rendered only when it differs from the release namespace (single-namespace installs render nothing extra).

Scope

bash -n clean; helm lint passes.

Refs #1906

@cyberdima
cyberdima requested a review from a team as a code owner July 24, 2026 23:47
@coderabbitai

coderabbitai Bot commented Jul 24, 2026

Copy link
Copy Markdown

Warning

Review limit reached

@cyberdima, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 54 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: a2aa4180-62ce-471e-875b-d29d8b9162b4

📥 Commits

Reviewing files that changed from the base of the PR and between 638ec02 and 63a553c.

📒 Files selected for processing (5)
  • charts/insight/templates/_helpers.tpl
  • charts/insight/templates/ingestion/reconcile-cron.yaml
  • charts/insight/templates/ingestion/reconcile-rbac.yaml
  • charts/insight/values.yaml
  • src/ingestion/reconcile-connectors/lib/airbyte.sh
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/1906-reconcile-airbyte-namespace

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

ab_get_token read airbyte-auth-secrets from $INSIGHT_NAMESPACE, assuming
Airbyte shares the app namespace. When Airbyte runs in its own namespace
the Secret is absent there, the token fetch fails, and the whole reconcile
run aborts before any connector is created or synced.

- airbyte.sh: new AIRBYTE_NAMESPACE env (defaults to INSIGHT_NAMESPACE, so
  single-namespace installs are unaffected); ab_get_token reads the Secret
  from it.
- chart: new airbyte.namespace value ("" = .Release.Namespace),
  insight.airbyte.namespace helper, AIRBYTE_NAMESPACE rendered on the
  reconcile CronWorkflow, and a conditional Role/RoleBinding granting the
  reconcile ServiceAccount `get` on the Secret in the Airbyte namespace —
  rendered only when it differs from the release namespace.

Refs #1906

Signed-off-by: Dmitry.Saukh <cyberdima@gmail.com>

Signed-off-by: Dmitry Saukh <38005371+cyberdima@users.noreply.github.com>
@cyberdima
cyberdima force-pushed the fix/1906-reconcile-airbyte-namespace branch from 95b315c to 63a553c Compare July 27, 2026 09:00
@github-actions

Copy link
Copy Markdown
Contributor

⚠️ Regenerate the connectors-ddl snapshot

This PR changes src/ingestion/**. If your change affects any
bronze / silver / gold schema, regenerate the committed DDL snapshot
and include it in this PR:

cd src/ingestion/scripts/bootstrap-db
set -a; source pins.env; source .env; set +a
./bootstrap-db.sh connectors-config.yaml   # fresh ClickHouse 25.7.5
./dump-ddl.sh                              # writes scripts/connectors-ddl/*.sql

Commit the resulting scripts/connectors-ddl/*.sql diff. If nothing
changed, no snapshot update is needed. (Regeneration is manual for now.)

My earlier conflict resolution left unresolved merge markers (broken
bash) and duplicated logic that #1935 already merged to main under
the same issue (#1885/#1906). Restoring these 5 files to main's
content leaves no diff — PR #1913 is superseded, closing it.

Signed-off-by: Dmitry Saukh <38005371+cyberdima@users.noreply.github.com>
@cyberdima

Copy link
Copy Markdown
Contributor Author

Closing as superseded by #1935, which already merged the same fix (cross-namespace airbyte-auth-secrets read) to main under issue #1885, via a more complete implementation (AIRBYTE_AUTH_SECRET_NAME/AIRBYTE_AUTH_SECRET_NAMESPACE/AIRBYTE_CLIENT_ID_KEY/AIRBYTE_CLIENT_SECRET_KEY, dedicated airbyte-auth-rbac.yaml, and binds the argo-workflow fallback SA in addition to the reconcile SA).

Rebased this branch onto main to confirm: after resolving conflicts in favor of main's version, the diff across all 5 touched files (_helpers.tpl, reconcile-cron.yaml, reconcile-rbac.yaml, values.yaml, airbyte.sh) is zero. Nothing left to merge.

@cyberdima cyberdima closed this Jul 27, 2026
auto-merge was automatically disabled July 27, 2026 09:07

Pull request was closed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Reconcile token helper reads the Airbyte auth Secret from the app namespace instead of the Airbyte namespace

1 participant