Skip to content

fix: copy repo-root fixtures into web-console test image - #1151

Merged
sakibsadmanshajib merged 2 commits into
mainfrom
fix/web-console-test-fixtures
Aug 25, 2026
Merged

sakibsadmanshajib merged 2 commits into
mainfrom
fix/web-console-test-fixtures

Conversation

@sakibsadmanshajib

@sakibsadmanshajib sakibsadmanshajib commented Aug 25, 2026 •

Copy link
Copy Markdown
Owner

Summary

Two apps/web-console unit test files failed to load on every clean-tree run of the mandated docker compose run --rm --build web-console npm run test:unit command, and a third test silently skipped, all for the same reason: deploy/docker/Dockerfile.web-console builds with the repo root as its build context (context: ../../ in docker-compose.yml) but only COPYs apps/web-console/. Anything a test reads from outside that subtree does not exist in the image.

  • tests/unit/control-plane-host.test.ts reads .env.example and the whole deploy/ tree (to cross-check every hostname under deploy/ against deploy/cloudflare/tunnel-ingress.json) at module load time. Missing .env.example threw synchronously, so the entire file failed to load.
  • tests/unit/chat-coverage-lib.test.ts reads docs/proof/chat-interaction-coverage-2026-08-10/coverage.run.json inside the "is never below what any recorded live run enumerated" test, which reproduced the reported ENOENT.
  • components/catalog/model-catalog-table.test.ts guards its last assertion with it.skipIf(constraintValues.length === 0), and constraintValues comes from existsSync(supabase/migrations). That directory is also outside the copied subtree, so the test silently skipped instead of failing loud. Same root-cause class as the two ENOENTs above, just surfaced as a quiet skip; the test file itself already documents this ("Skipped only where supabase/migrations is not on disk, which is the deploy/docker web-console image").

Fix

Added three narrow COPY lines to Dockerfile.web-console, after the existing COPY apps/web-console/ ./, landing each fixture at the same repo-root-relative path the tests already resolve against (/app/..., matching WORKDIR /app/apps/web-console's /app root):

  • .env.example (64K)
  • deploy/ (2.5M, needed whole since the test recursively scans it for hostnames)
  • docs/proof/chat-interaction-coverage-2026-08-10/coverage.run.json (76K, the one file the test reads, not all of docs/proof/ which is 17M)
  • supabase/migrations/ (652K)

None of these paths are excluded by the repo's .dockerignore. Total added context: about 3.2M, chosen to keep the image copy narrow rather than pulling in the rest of the repo (docs/proof/ alone is 17M, of which the tests need one 76K file).

Deliberately did not make either test skip when its fixture is missing, and did not touch the assertions themselves: this repo has an explicit rule that a loud failure is preferable to a quiet absence, and the fix here is to make the fixture actually present rather than to soften the check.

Investigation of the reported "1 failed" test

Re-running the mandated command against origin/main before this fix, three times, produced 0 failures beyond the two ENOENT load failures and the one skip described above; the module-level reads in control-plane-host.test.ts throw at import time (whole-file load failure), while chat-coverage-lib.test.ts's ENOENT throws lazily inside a single it() body, so depending on how a given tool renders vitest's collect-error vs test-failure distinction, that single assertion can show up as either "file failed to load" or "1 failed test" in a summary. After the fix, all three (both ENOENTs and the skip) resolve together; no separate, unrelated failing test was found across repeated runs.

Required-check fix: deploy-demo-box.yml paths filter

The first push failed the required Repo policy lints (tenant + audit) check, specifically node .github/ci/lint-deploy-paths-filter.mjs. Read the real failure (gh run view --log-failed) rather than guessing: the lint is genuinely working as designed. It walks every COPY/ADD source in every deploy/docker/Dockerfile.* and requires a matching entry in deploy-demo-box.yml's on.push.paths, because that filter has twice before silently swallowed a real change (apps/web-console via #786, vendor/open-webui via #971) with no failure at all, just a merge that never triggered a deploy. My three new COPY sources (.env.example, deploy/, the one docs/proof/... file) were exactly the next instance of that gap. supabase/migrations/ was already covered by an existing supabase/migrations/** entry, so the lint did not flag it.

Fixed by adding three entries to the paths filter (with a comment matching this file's existing convention, referencing this PR): .env.example, deploy/**, docs/proof/chat-interaction-coverage-2026-08-10/coverage.run.json. deploy/** is deliberately the whole tree, not narrowed to the pre-existing deploy/docker/**/deploy/litellm/** entries, for the same reason the Dockerfile copies the whole tree (see next section). Verified locally: npm install at repo root, then node .github/ci/lint-deploy-paths-filter.mjs reports Deploy path-filter coverage OK: every COPY/ADD source across 15 Dockerfiles under deploy/docker/ is covered. Also re-ran the sibling lint-workflow-check-names.mjs in the same job to confirm it is unaffected.

Security review of the COPY deploy/ /app/deploy/ scope

Answering the three questions raised on this PR directly:

1. Does anything under deploy/ that now enters the image carry a credential, token, or internal hostname that should not be baked into a built image? No. Grepped the whole deploy/ tree for AWS-style keys, PEM private-key headers, sk-... style tokens, and literal password:/secret: values with a non-os.environ right-hand side: zero hits. Read deploy/litellm/config.yaml specifically since it was named directly: every api_key: line in it is os.environ/OPENROUTER_API_KEY, os.environ/GROQ_API_KEY, or the literal "none" (for a route that takes no key); there is no hardcoded credential anywhere in that file. It does carry internal hostnames and routing/pricing commentary, which is operational detail rather than a secret, and no different from what already ships in the image today via deploy/docker/**/deploy/litellm/**, both already COPYed by other Dockerfiles in this same directory before this PR.

2. Is hive-web-console:ci (the image this Dockerfile builds) genuinely CI/test-only, or does it also serve the console on the demo box? Confirmed CI/test-only, and distinct from what ships. docker-compose.yml defines two separate services: web-console (image hive-web-console:ci, this Dockerfile, gated behind profiles: [dev], runs plain next dev) and web-console-prod (image hive-web-console-prod:ci, a different file, Dockerfile.web-console.prod, sitting behind its own Caddy origin, caddy-console). The header comment on the paths-filter's existing apps/web-console/** entry says this explicitly: "console-hive.scubed.co is web-console-prod... built... by Dockerfile.web-console.prod." deploy-demo-box.yml never references the web-console service or the :ci tag at all, only apps/web-console as a source path (because web-console-prod is built from that same source tree). docker-bake.hcl's web-console target tags the image hive-web-console:ci and is never pushed to any registry in ci.yml or deploy-demo-box.yml (grepped both; no docker push/registry step references this tag). So the image this PR's COPY deploy/ lands in never leaves the CI runner or a developer's machine, and never reaches the demo box.

3. Would a narrower COPY satisfy the test? No, not without defeating the test's purpose, and this is deliberate, not laziness. control-plane-host.test.ts's "deploy configuration hostnames" suite exists specifically to recursively scan every text file under all of deploy/ (excluding deploy/cloudflare/ itself, which is the registry it checks against, and node_modules) for any *.scubed.co hostname not declared in deploy/cloudflare/tunnel-ingress.json. That is the whole point of the suite: catch a stray or retired hostname in any file under deploy/, including ones that do not exist yet. Copying only today's known files would make the test blind to the next Caddyfile or compose fragment added later, which is exactly the silent-gap failure mode this repo's own rules warn against. supabase/migrations/ is the same shape (whole directory, needed because the constraint-value scan unions across every migration file). The two single-file copies (.env.example, the one coverage.run.json) are already as narrow as the tests need.

Test plan

  • cd deploy/docker && docker compose run --rm --build web-console npm run test:unit run to completion against the fixed image, with a clean/idle host: Test Files 60 passed (60), Tests 653 passed (653), 0 failed, 0 skipped.
  • Confirmed via .dockerignore review that none of the four newly-copied paths are excluded from the build context.
  • node .github/ci/lint-deploy-paths-filter.mjs passes locally after the paths-filter fix.
  • Grepped deploy/ for hardcoded secrets (AWS keys, PEM headers, sk-... tokens, literal password/secret values); none found. deploy/litellm/config.yaml confirmed to use os.environ/... indirection for every API key.
  • Confirmed via docker-compose.yml, docker-bake.hcl, and both ci.yml/deploy-demo-box.yml that hive-web-console:ci is CI/dev-only, never pushed to a registry, and not the image serving the demo box (web-console-prod/Dockerfile.web-console.prod is).

Note on later local reruns: after rebasing onto a newer main, repeated non---build and --build runs on this same dev box intermittently showed 2-5 unrelated test failures (analytics-billing-page-wiring.test.tsx, members-page-rbac.test.tsx, others), each a different set each run, with no code in this diff touching those files. The box was running 20+ other containers from concurrent sessions at the time (docker ps showed multiple hiveverify-*, composerfix-* stacks, 4.2G swapped) and the same run's own "environment" setup phase ballooned from ~150s to ~360s between attempts, consistent with resource-contention flakiness in waitFor()-based React tests rather than a regression from this PR's Dockerfile/workflow-only diff. GitHub's isolated CI runner ran the Web console (type + unit + build) job clean (60/60) on the pre-fix commit of this same PR, which is the authoritative signal for this class of test.

Buglog entry

{"date":"2026-08-25","error_message":"ENOENT open '/app/.env.example' and ENOENT open '/app/docs/proof/chat-interaction-coverage-2026-08-10/coverage.run.json' during `docker compose run --build web-console npm run test:unit`; separately, components/catalog/model-catalog-table.test.ts silently skips its last assertion via it.skipIf","root_cause":"Dockerfile.web-console uses the repo root as build context but only COPYs apps/web-console/, so any test reading a repo-root path (.env.example, deploy/, docs/proof/..., supabase/migrations/) gets ENOENT or a false existsSync inside the image, even though those files are present on disk in every other run context","fix":"Added narrow COPY lines for .env.example, deploy/, the one coverage.run.json fixture under docs/proof/, and supabase/migrations/ into Dockerfile.web-console, landing each at the same /app-relative path the tests already resolve against; added matching entries to deploy-demo-box.yml's push.paths filter so a change to any of these still triggers a demo-box deploy; left the tests themselves untouched so a genuinely stale/missing fixture still fails loudly","tags":["web-console","docker","test-fixtures","vitest","dockerfile","ci-paths-filter"]}

@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

@coderabbitai

coderabbitai Bot commented Aug 25, 2026 •

Copy link
Copy Markdown

Warning

Review limit reached

Next included review available in 32 minutes.

View limit details

Limit details: You’ve used the included review currently available.

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Learn how review limits work.

Review configuration:

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 3cfb254d-785c-47d8-968b-bdf47cfffa84

📥 Commits

Reviewing files that changed from the base of the PR and between d369b3c and b2b7826.

📒 Files selected for processing (2)
  • .github/workflows/deploy-demo-box.yml
  • deploy/docker/Dockerfile.web-console

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@sakibsadmanshajib
sakibsadmanshajib force-pushed the fix/web-console-test-fixtures branch from 91a389e to b2b7826 Compare August 25, 2026 12:03
@sakibsadmanshajib
sakibsadmanshajib merged commit 9ced2a8 into main Aug 25, 2026
26 checks passed
@sakibsadmanshajib
sakibsadmanshajib deleted the fix/web-console-test-fixtures branch August 25, 2026 12:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant