Skip to content

fix(ingestion): zoom participants substream deadlock — bump default_concurrency to 4 - #1488

Merged
mitasovr merged 3 commits into
constructorfabric:mainfrom
mitasovr:fix/zoom-substream-concurrency
Jun 25, 2026
Merged

fix(ingestion): zoom participants substream deadlock — bump default_concurrency to 4#1488
mitasovr merged 3 commits into
constructorfabric:mainfrom
mitasovr:fix/zoom-substream-concurrency

Conversation

@mitasovr

@mitasovr mitasovr commented Jun 25, 2026

Copy link
Copy Markdown
Contributor

Problem

The zoom participants stream is a SubstreamPartitionRouter over meetings (~30k partitions on a multi-month backfill). At concurrency_level.default_concurrency: 1 the concurrent CDK self-deadlocks once the substream generates ≥10k partitions: the single worker throttles in generate_partitions waiting on partition-read futures that no other worker can execute.

Observed on dev-vhc during first connector sync:

  • users (446) and meetings (29,935) flush to bronze_zoom.* within minutes ✅
  • then participants sits RUNNING emitting zero records / zero checkpoints for ~50 min while the source container spins ~12% CPU forever ❌

This is the same root cause and fix as the earlier jira/confluence substream hang (concurrent-CDK self-deadlock at default_concurrency: 1, confirmed there via py-spy).

Fix

  • connector.yaml: default_concurrency 1 → 4.
  • descriptor.yaml: version 2026.05.04 → 1.0.0 (strict semver per ADR-0015) so the reconciler detects the drift and republishes the manifest to the existing Airbyte source definition. classify_bump("1.0.0", "2026.05.04")migration (re-discover catalog, no full-refresh).

Verification plan (post-deploy)

After the toolbox image ships and dev redeploys, the reconcile loop updates the zoom source manifest; a re-sync should show participants checkpoints/records flowing and bronze_zoom.participants row count climbing (mirrors the jira/confluence recovery: 0 → thousands/min once concurrency hit 4).

🤖 Generated with Claude Code

Summary by CodeRabbit

  • New Features
    • Increased the default concurrency for the Zoom connector, which may improve sync throughput.
  • Chores
    • Updated the Zoom connector version to 1.0.0 for clearer release tracking.

…oncurrency to 4

The zoom `participants` stream is a SubstreamPartitionRouter over `meetings`
(~30k partitions on a multi-month backfill). At `default_concurrency: 1` the
concurrent CDK self-deadlocks once the substream generates >=10k partitions:
the single worker throttles in `generate_partitions` waiting on partition-read
futures that no other worker can execute. Symptom observed on dev-vhc: `users`
(446) and `meetings` (29935) flush to bronze_zoom within minutes, then
`participants` sits RUNNING emitting zero records for ~50 min while the source
spins ~12% CPU forever.

Same root cause and fix as the prior jira/confluence substream hang: bump
`default_concurrency` 1 -> 4.

Also bump `descriptor.version` 2026.05.04 -> 1.0.0 (strict semver per ADR-0015)
so the reconciler detects the drift and republishes the manifest to the
existing Airbyte source definition.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Signed-off-by: Roman Mitasov <Roman.Mitasov@constructor.tech>
@mitasovr
mitasovr requested a review from a team as a code owner June 25, 2026 14:01
@coderabbitai

coderabbitai Bot commented Jun 25, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: d388bd85-a821-4044-8fc1-f7fe7555e5c9

📥 Commits

Reviewing files that changed from the base of the PR and between 509e241 and cc14529.

📒 Files selected for processing (2)
  • src/ingestion/connectors/collaboration/zoom/connector.yaml
  • src/ingestion/connectors/collaboration/zoom/descriptor.yaml

📝 Walkthrough

Walkthrough

Updated the Zoom connector descriptor version to 1.0.0 and changed the connector’s default concurrency from 1 to 4.

Changes

Zoom connector settings

Layer / File(s) Summary
Descriptor version and concurrency default
src/ingestion/connectors/collaboration/zoom/descriptor.yaml, src/ingestion/connectors/collaboration/zoom/connector.yaml
The Zoom connector descriptor version is set to 1.0.0, and the connector’s default_concurrency is set to 4.

Estimated code review effort

🎯 1 (Trivial) | ⏱️ ~2 minutes

Possibly related PRs

Suggested reviewers

  • cyberantonz
  • mozhaev-dev

Poem

Hop hop, I tuned the Zoom,
Four little lanes now share the room.
Version one now stands so neat,
Rabbit ears wiggle to the beat. 🐇

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly matches the main change: fixing the Zoom participants substream deadlock by increasing default concurrency to 4.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@mitasovr
mitasovr requested a review from aleksdotbar June 25, 2026 14:07
@mitasovr
mitasovr merged commit 0adeeff into constructorfabric:main Jun 25, 2026
18 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants