[jonahgabriel]

Summary

• Defaults WORKER_REPLICAS from 2 to 0 in docker-compose to stop Kafka consumer rebalance storm
• Workers (runtime-worker-1, runtime-worker-2) join the same per-topic consumer groups as omninode-runtime on 1-partition topics
• With 3 members competing for 1 partition across 100+ topics, the broker triggers constant rebalance cycles that block the build loop and other consumers

Root cause

The runtime kernel wires ALL consumers regardless of RUNTIME_PROFILE. Workers and the main runtime both subscribe to every topic (delegation, build-loop, routing, intelligence, etc.). Since each topic has 1 partition, only 1 member per group can own it — the other 2 trigger rebalances trying to join.

Fix

Default WORKER_REPLICAS=0 so workers don't start. Workers can be re-enabled via WORKER_REPLICAS=N once the kernel supports profile-based subscription filtering (workers only subscribing to their designated topic subset).

Evidence

Logs from all 3 containers show continuous rebalance churn:

[WARNING] Heartbeat failed for group ...build-loop-start.v1 because it is rebalancing
[INFO] Revoking previously assigned partitions ... for group ...routing-decision.v1
[INFO] Setting newly assigned partitions set() for group ...quality-gate-result.v1

Test plan

• Verify infra-up-runtime starts without worker containers
• Verify build loop topic can be consumed without rebalance noise
• Verify WORKER_REPLICAS=2 env override still works to re-enable workers

Summary by CodeRabbit

• Chores
  • Default infrastructure worker scaling changed so worker services start with zero instances by default (remains configurable via environment).
• Tests
  • Test expectations updated to allow zero worker replicas (range expanded to include 0), reflecting the new default behavior.

[coderabbitai]

📝 Walkthrough

Walkthrough

The default runtime-worker Docker Compose replica count is changed to start at 0 when WORKER_REPLICAS is unset; accompanying test expectations were relaxed to allow 0 replicas. Inline comments referencing OMN-7569 were added to the compose file.

Changes

Cohort / File(s)	Summary
Docker Compose Configuration docker/docker-compose.infra.yml	Changed services.runtime-worker.deploy.replicas fallback from ${WORKER_REPLICAS:-2} to ${WORKER_REPLICAS:-0} and added inline comments referencing OMN-7569 and guidance to keep workers scaled to 0 until profile-based filtering exists.
Unit Tests tests/unit/docker/test_docker_performance.py	Updated test_worker_replicas_reasonable assertion range to allow 0..10 replicas (previously 1..10) to reflect the new default of 0 replicas.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

Poem

🐰 I nudged the workers into gentle rest,
Zero now waits where two once pressed,
Profiles will call them back one day—
Till then I hop and munch away. 🥕

🚥 Pre-merge checks | ✅ 3

✅ Passed checks (3 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title accurately describes the main change: defaulting runtime worker replicas to 0 to address the Kafka rebalance storm issue (OMN-7569). It is specific, concise, and directly reflects the primary objective of the pull request.
Docstring Coverage	✅ Passed	Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch jonah/fix-kafka-rebalance-storm

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 1

🧹 Nitpick comments (1)

docker/docker-compose.infra.yml (1)

656-658: Add a concrete re-enable command next to the new default rationale.

The explanation is good, but adding one explicit override example at Lines 656-658 would make operator behavior unambiguous.

Suggested doc tweak

   # OMN-7569: Default replicas=0 — workers join the same per-topic consumer groups
   # as the main runtime on 1-partition topics, causing a constant rebalance storm.
   # Scale to 0 until the kernel supports profile-based subscription filtering.
+  # Re-enable workers explicitly when needed, e.g.:
+  #   WORKER_REPLICAS=2 docker compose -f docker/docker-compose.infra.yml --profile runtime up -d

🤖 Prompt for AI Agents

Verify each finding against the current code and only fix it if needed.

In `@docker/docker-compose.infra.yml` around lines 656 - 658, Add a concrete
re-enable command next to the OMN-7569 rationale that documents the temporary
default replicas=0; specifically, show an explicit override example such as how
to scale the service back to 1 (e.g. using docker compose --scale <service>=1 or
docker-compose up --scale <service>=1), referencing the replicas=0 decision so
operators know exactly how to revert it. Ensure the example names the affected
service (replace <service> with the actual service name used in the compose
file) and is placed immediately after the existing comment block about OMN-7569.

🤖 Prompt for all review comments with AI agents

Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@docker/docker-compose.infra.yml`:
- Line 691: Add explicit documentation and defaults to avoid the silent breaking
change from setting replicas to 0 by default: add
WORKER_REPLICAS=<default_value> (e.g., 2) with a brief comment to
docker/.env.example, update CHANGELOG.md and docker/README.md with a
breaking-change entry and migration steps explaining that WORKER_REPLICAS now
defaults to 0 and must be set to restore previous behavior, and add an inline
comment in deploy-runtime.sh near the environment sourcing logic
(scripts/deploy-runtime.sh) calling out the new default and instructing
operators to set WORKER_REPLICAS in their ~/.omnibase/.env or CI pipelines.

---

Nitpick comments:
In `@docker/docker-compose.infra.yml`:
- Around line 656-658: Add a concrete re-enable command next to the OMN-7569
rationale that documents the temporary default replicas=0; specifically, show an
explicit override example such as how to scale the service back to 1 (e.g. using
docker compose --scale <service>=1 or docker-compose up --scale <service>=1),
referencing the replicas=0 decision so operators know exactly how to revert it.
Ensure the example names the affected service (replace <service> with the actual
service name used in the compose file) and is placed immediately after the
existing comment block about OMN-7569.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 15ae4e03-ae5f-43b1-9cd5-181d31566cfe

📥 Commits

Reviewing files that changed from the base of the PR and between 48e0326 and 8f5dd52.

📒 Files selected for processing (1)

• docker/docker-compose.infra.yml

[coderabbitai]

🧹 Nitpick comments (1)

tests/unit/docker/test_docker_performance.py (1)

126-126: Update the inline example to avoid stale guidance.

Line 126 still shows "${WORKER_REPLICAS:-2}"; updating it to "${WORKER_REPLICAS:-0}" will keep the comment consistent with current behavior.

Suggested diff

-                # Handle environment variable substitution (e.g., "${WORKER_REPLICAS:-2}")
+                # Handle environment variable substitution (e.g., "${WORKER_REPLICAS:-0}")

🤖 Prompt for AI Agents

Verify each finding against the current code and only fix it if needed.

In `@tests/unit/docker/test_docker_performance.py` at line 126, Update the inline
comment example that demonstrates environment variable substitution by replacing
the obsolete default value "${WORKER_REPLICAS:-2}" with the current behavior
value "${WORKER_REPLICAS:-0}", so the comment in
tests/unit/docker/test_docker_performance.py reflects the actual default used.

🤖 Prompt for all review comments with AI agents

Verify each finding against the current code and only fix it if needed.

Nitpick comments:
In `@tests/unit/docker/test_docker_performance.py`:
- Line 126: Update the inline comment example that demonstrates environment
variable substitution by replacing the obsolete default value
"${WORKER_REPLICAS:-2}" with the current behavior value "${WORKER_REPLICAS:-0}",
so the comment in tests/unit/docker/test_docker_performance.py reflects the
actual default used.

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 87517903-5bc4-4530-983e-f813744e229b

📥 Commits

Reviewing files that changed from the base of the PR and between 8f5dd52 and ec7037f.

📒 Files selected for processing (1)

• tests/unit/docker/test_docker_performance.py