Skip to content

#130: Snowflake estimate_query_bytes — EXPLAIN-based estimation (plan) - #132

Merged
wjduenow merged 23 commits into
devfrom
feature/130-snowflake-estimate-explain
May 27, 2026
Merged

#130: Snowflake estimate_query_bytes — EXPLAIN-based estimation (plan)#132
wjduenow merged 23 commits into
devfrom
feature/130-snowflake-estimate-explain

Conversation

@wjduenow

@wjduenow wjduenow commented May 26, 2026

Copy link
Copy Markdown
Owner

Summary

Super plan for #130 — replace the Snowflake --estimate degrade (#123) with a real EXPLAIN USING JSON-based estimate.

Phase: detailing (awaiting approval)
Stories: 6 implementation stories + Quality Gate + Patterns & Memory
Decisions: 9 (DEC-001 … DEC-009)

Key findings

Plan document

See plans/super/130-snowflake-estimate-explain.md.

Next steps

  • Review the plan in this PR
  • Approve in Claude Code, then say "devolve" to create beads

🤖 Generated with Claude Code

Summary by CodeRabbit

  • New Features

    • Snowflake adapters now support query byte estimation using EXPLAIN USING JSON.
    • CLI now displays the source of warehouse estimates (e.g., "Snowflake EXPLAIN" or "BigQuery dryRun").
  • Improvements

    • New error handling for estimation failures when query plans lack parseable byte figures.

Review Change Stack

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented May 26, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

🗂️ Base branches to auto review (4)
  • feature/.*
  • bug/.*
  • hotfix/.*
  • feat/.*

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 42e689df-5327-4d53-85b9-1488ef1ed215

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

This PR implements Snowflake's estimate_query_bytes using EXPLAIN USING JSON to extract byte counts from GlobalStats.bytesAssigned, introduces a new EstimateUnavailableError for parsing failures, and integrates the feature through CLI reporting, exit codes, and comprehensive test coverage including live certification support.

Changes

Snowflake EXPLAIN-based estimate_query_bytes implementation

Layer / File(s) Summary
Error type definition and wiring
src/signalforge/warehouse/errors.py, src/signalforge/warehouse/__init__.py, src/signalforge/cli/_helpers.py, tests/warehouse/test_errors.py, tests/cli/test_exit_codes.py
New EstimateUnavailableError is defined as a WarehouseError subclass with a detail field and locked remediation message. The error is exported through signalforge.warehouse, mapped to exit-code tier 3 in the CLI exception handler, and covered by unit tests asserting inheritance, rendering, and exit-code mapping.
Core adapter implementation: EXPLAIN parser and estimate_query_bytes
src/signalforge/warehouse/adapters/snowflake.py
New _parse_explain_json_bytes(cell) helper extracts and validates GlobalStats.bytesAssigned from JSON EXPLAIN output, raising EstimateUnavailableError for missing/invalid fields or non-integer values. New _execute_scalar(sql) helper runs SQL and returns the first result cell, routing Snowflake SDK errors through the exception mapper. estimate_query_bytes(sql) validates input, executes EXPLAIN USING JSON <sql>, and parses the result or raises EstimateUnavailableError on empty results.
Unit tests for parser and adapter method
tests/warehouse/test_snowflake_estimate.py
Comprehensive unit tests for _parse_explain_json_bytes covering happy paths (string JSON, dict inputs, minimal structures, zero bytes), failure modes (missing GlobalStats, non-mapping types, missing/invalid bytesAssigned field, malformed JSON, bool/float/negative values), and SnowflakeAdapter.estimate_query_bytes covering happy path, SQL validation, SQL embedding, exception mapping, empty results, and unmapped exception passthrough.
CLI estimate reporting and source labeling
src/signalforge/cli/_estimate.py, tests/cli/test_estimate_render.py
New warehouse_estimate_source: str | None field added to EstimateReport to track which adapter produced the byte estimate. Populated on successful estimation using an internal adapter-class-to-label mapping (e.g., "Snowflake EXPLAIN", "BigQuery dryRun"). Render output includes the source label in the bytes-per-row line instead of hardcoding "BigQuery dryRun".
Estimate engine tests for Snowflake happy and degrade paths
tests/cli/test_estimate_engine.py
Fixture-loaded EXPLAIN JSON samples and tests for estimate() with SnowflakeAdapter on FakeSnowflakeConnection. Happy-path test verifies real warehouse bytes with correct "Snowflake EXPLAIN" source label. Degrade-path test verifies missing GlobalStats reports EstimateUnavailableError, warehouse bytes become None, but LLM cost computation succeeds.
Test fixtures, README, and Snowflake stub tests
tests/fixtures/warehouse/snowflake/*.json, tests/fixtures/warehouse/snowflake/README.md, tests/warehouse/test_snowflake_stub.py
Hand-crafted EXPLAIN USING JSON fixtures (with and without GlobalStats) pinned for deterministic test behavior. README documents fixture maintenance and gated live-test procedure. Snowflake stub test suite replaces prior EstimateNotSupportedError expectation with real EXPLAIN-based byte extraction tests and graceful degradation on missing stats.
Integration tests: FakeSnowflakeConnection and live certification
tests/warehouse/test_snowflake_estimate_live.py, tests/cli/test_generate_estimate.py
Gated live Snowflake tests (@pytest.mark.snowflake) run only when env vars are set, exercising real EXPLAIN USING JSON execution and validating result shape against committed fixtures. End-to-end CLI tests verify generate --estimate produces real warehouse bytes and graceful degradation with correct error output.
Rules, operations docs, and planning documentation
.claude/rules/warehouse-adapters.md, docs/warehouse-adapter-ops.md, plans/super/130-snowflake-estimate-explain.md, pyproject.toml
Updated warehouse-adapters.md to document Snowflake skeleton dialect contracts and graduation to EXPLAIN-based estimation. Expanded warehouse-adapter-ops.md to document BigQuery and Snowflake estimate_query_bytes behavior, error conditions, remediation, and migration paths. Comprehensive planning document (plans/super/130) details implementation strategy, testing approach, and graduation recipe. Pytest marker updated to reflect gated live EXPLAIN-estimate certification.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Possibly related issues

  • #130: Snowflake estimate_query_bytes(EXPLAIN-based estimation) — the exact feature implemented in this PR (SnowflakeAdapter.estimate_query_bytes, _parse_explain_json_bytes, EstimateUnavailableError, and comprehensive test/docs coverage).
  • #118: Snowflake adapter epic — this PR advances the v0.2 skeleton by implementing the #ESTIMATE follow-up surface and corresponding test/documentation parity.

Possibly related PRs

  • wjduenow/SignalForge#128: Replaces Snowflake's earlier degrade-first EstimateNotSupportedError behavior (from PR #123) with the new EXPLAIN-based implementation and updates CLI/engine/stub test expectations accordingly.
  • wjduenow/SignalForge#125: Introduces the SnowflakeAdapter skeleton (#119) that left estimate_query_bytes to the ABC typed "not supported" degrade; this PR implements that method.

Poem

🐰 A rabbit hops through EXPLAIN trees,

Extracting GlobalStats with ease,

When bytes can't parse, no sorrow—

A graceful degrade saves the day!

Snowflake soars in every way. ❄️

🚥 Pre-merge checks | ✅ 3 | ❌ 2

❌ Failed checks (2 warnings)

Check name Status Explanation Resolution
Linked Issues check ⚠️ Warning The PR implements issue #130 (Snowflake estimate_query_bytes via EXPLAIN) comprehensively, but the linked issue #122 (Snowflake deterministic sampling) is unrelated—this PR contains no sampling implementation whatsoever. The PR fully addresses #130 but does not implement #122 sampling surfaces (sample_rows, materialise_sample). Clarify whether #122 is a dependency (should be listed as 'depends on' rather than 'linked') or remove it from linked issues.
Docstring Coverage ⚠️ Warning Docstring coverage is 75.93% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (3 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and concisely identifies the main change: implementing EXPLAIN-based estimation for Snowflake's estimate_query_bytes method (issue #130), accurately reflecting the PR's primary objective.
Out of Scope Changes check ✅ Passed All changes directly support issue #130: new EstimateUnavailableError exception, Snowflake EXPLAIN-based estimation implementation, fixture-based tests, documentation, and CLI integration. No unrelated refactoring or scope creep detected.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.


Comment @coderabbitai help to get the list of available commands and usage tips.

@codecov-commenter

codecov-commenter commented May 26, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

wjduenow and others added 19 commits May 26, 2026 16:14
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…error + tier-3 registration

Add EstimateUnavailableError(WarehouseError) for the "estimation seam ran
but produced no usable figure for THIS query" case (DEC-003), distinct from
EstimateNotSupportedError ("adapter does no estimation at all"). Keyword-only
`detail` rendered repr-safe via _format_value; locked-verbatim
default_remediation pointing at the price-only-preview degrade. Exported from
warehouse/__init__.py and registered in cli/_helpers._EXCEPTION_TO_EXIT_CODE
at tier 3 (external-dep). Tests: subclass check, __str__ renders message +
remediation, locked-verbatim remediation, distinct-from-not-supported,
map_exception_to_exit_code == 3; scan-7 green with the new concrete.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Strip per-issue genealogy (DEC suffixes, issue/PR citations, dated status
notes) and multi-sentence retellings of how decisions evolved from the four
largest rules files; that history lives in plans/super/ and CHANGELOG.md.
Every load-bearing invariant, locked verbatim string, taxonomy, config-field
list, function/seam name, and AST-scan / grep-gate / drift-detector contract
is preserved. ~287 lines removed across warehouse-adapters, prune-engine,
cli-layer, and diff-renderer.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The 14 .claude/rules/*.md files were auto-loaded into every Claude Code
session (~48K tokens of project instructions), dwarfing CLAUDE.md and
triggering the large-context warning even after CLAUDE.md was trimmed. They
are reference contracts meant to be read per-layer on demand, not held in
context globally.

Move them to docs/rules/ (out of the auto-loaded .claude/ tree) and add
rules/ to mkdocs exclude_docs so they stay off the published site (mirrors
research/). Update the ~52 live docstring/comment pointers in src/, tests/,
and docs/ to the new path, and correct CLAUDE.md (drop the now-false
"auto-loaded into context" claim; the architecture-map table is the
layer -> file lookup). plans/super/ ADR pointers are left as historical
record. No runtime path dependency existed; all references are prose.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…parser + fixtures

Add module-level pure function _parse_explain_json_bytes(cell) -> int in
warehouse/adapters/snowflake.py: navigates GlobalStats.bytesAssigned from an
EXPLAIN USING JSON result cell (str JSON or pre-parsed dict), returning a
non-negative int. Raises typed EstimateUnavailableError (imported from
warehouse.errors, US-001) on unparseable JSON, non-object document, missing/
non-mapping GlobalStats, missing bytesAssigned, non-int/bool/negative value —
never fabricates a number, never returns 0 for a missing field (DEC-002).

Pure: no connection, no logging. Wiring into estimate_query_bytes is US-003.

Ship hand-crafted fixtures under tests/fixtures/warehouse/snowflake/
(explain_using_json_sample.json @ 104857600 bytes + explain_using_json_no_stats.json)
plus a README documenting they are hand-crafted (workers can't reach live
Snowflake) and the maintainer regen command. Engineered determinism: the parse
test asserts the int EQUALS the fixture's known bytesAssigned.

DEC-001/002/006.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…_bytes EXPLAIN override

Override estimate_query_bytes(sql)->int: validate_test_sql first, then
EXPLAIN USING JSON <validated-sql> via a new _execute_scalar cursor helper
(no TableRef in scope), parse GlobalStats.bytesAssigned via the existing
_parse_explain_json_bytes pure fn. SDK exceptions route through
map_snowflake_exception (DEC-005); empty result -> EstimateUnavailableError.
No snowflake.connector import in the adapter. Docstrings updated (estimate
now implemented, no longer inherits ABC degrade).

DEC-001/004/005/008.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…timate test + regen note

Add tests/warehouse/test_snowflake_estimate_live.py: a @pytest.mark.snowflake-
gated live test that drives a real SnowflakeAdapter through estimate_query_bytes
against a live warehouse (EXPLAIN USING JSON), certifying the committed fixture's
GlobalStats.bytesAssigned shape. Belt-and-suspenders gating: marker (deselected by
default addopts) + runtime _skip_reason naming each missing prerequisite
(SF_RUN_SNOWFLAKE=1 + SNOWFLAKE_ACCOUNT/USER/PASSWORD/WAREHOUSE). Asserts shape +
non-negativity, never an exact planner value (DEC-006).

Extend the fixture README regen note with the exact maintainer run command and a
pointer to the live test module.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…real EXPLAIN estimate

#130 US-003 overrides SnowflakeAdapter.estimate_query_bytes with a real
EXPLAIN-based implementation, making the three #123 tests that asserted
EstimateNotSupportedError RED. DEC-007: rewrite (not delete) each into a
happy path (real EXPLAIN bytes via an injected FakeSnowflakeConnection) plus
a degrade path (no-stat EXPLAIN -> EstimateUnavailableError, keyed on the
specific class name per the #123 rule note).

- test_snowflake_stub.py: replace test_estimate_query_bytes_raises_not_supported
  with test_estimate_query_bytes_returns_explain_bytes +
  test_estimate_query_bytes_degrades_on_missing_stat; refresh the stale
  module docstring claiming the ABC-default inheritance.
- test_estimate_engine.py: split the Snowflake degrade test into
  test_estimate_reports_real_bytes_for_snowflake_explain (no degrade) and
  test_estimate_degrades_on_snowflake_explain_missing_stat.
- test_generate_estimate.py: split the CLI Snowflake test into
  ..._reports_real_bytes (real estimate) and ..._degrades_to_exit_zero
  (EstimateUnavailableError); refresh the stale helper docstring.

Tests only; no src/ changes. Full suite green (2368 passed).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…lake EXPLAIN estimate

Correct the now-stale "Snowflake inherits the estimate degrade" claims
across the two doc surfaces that carried them (DEC-009 of #130):

- docs/warehouse-adapter-ops.md § Query-bytes estimation: Snowflake is
  now a real EXPLAIN USING JSON estimate (GlobalStats.bytesAssigned)
  alongside BigQuery's dry_run; documents the EstimateUnavailableError
  degrade (EXPLAIN ran but no parseable figure) and the planner-estimate
  accuracy caveat. New error-reference row for EstimateUnavailableError;
  EstimateNotSupportedError row narrowed to the Postgres stub.
- .claude/rules/warehouse-adapters.md: #123 note corrected — Snowflake's
  estimate_query_bytes graduated from the ABC degrade to a real EXPLAIN
  override in #130 (was deferred-to-#130); the #119/#122 historical notes
  reframed as past-state with explicit "graduated in #130" pointers.

CLAUDE.md (slim version on disk) carries no estimate / public-API
enumeration, so no stale claim to correct there.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…source label + marker description

Code-review pass 3 caught the --estimate renderer hardcoding "(BigQuery dryRun)"
on the bytes-per-row line; now that SnowflakeAdapter returns a real EXPLAIN
estimate it reached that branch and mislabelled the source. Add a
warehouse_estimate_source field derived from the adapter class, render it, and
pin "Snowflake EXPLAIN" in the happy-path test. Also broaden the stale
`snowflake` pytest-marker description to cover the gated live EXPLAIN test, and
soften the live-test opt-in docstring.

Quality Gate: 4 code-review passes + CodeRabbit (no findings in the #130
changeset; CodeRabbit's 2 findings were in an untracked docs/temp scratch dir
left out of this branch).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…-degrade graduation recipe

Distil the reusable per-adapter graduation pattern into warehouse-adapters.md
(pure-fn parse + hand-crafted fixture + gated live test; rewrite the prior
phase's degrade tests rather than deleting them) so the next adapter (Postgres
EXPLAIN) inherits it. Memory note added for the orchestrator git-add-all gotcha.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@wjduenow
wjduenow marked this pull request as ready for review May 27, 2026 00:09
snowflake.py:836 (the `mapped is exc` bare-raise arm of _execute_scalar) was
the one #130-introduced line missing patch coverage. Add a test driving
estimate_query_bytes with a fake whose EXPLAIN raises a non-connector
RuntimeError: map_snowflake_exception returns it unchanged, so the original
propagates as-is. snowflake.py now 100%.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@wjduenow

Copy link
Copy Markdown
Owner Author

@CodeRabbit review

@coderabbitai

coderabbitai Bot commented May 27, 2026

Copy link
Copy Markdown
✅ Actions performed

Review triggered.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR implements Snowflake query-bytes estimation for signalforge generate --estimate by overriding SnowflakeAdapter.estimate_query_bytes to run EXPLAIN USING JSON and parse GlobalStats.bytesAssigned, replacing the prior “not supported” degrade behavior. It also introduces a new typed EstimateUnavailableError for the “supported but no parseable stat” case, updates CLI labeling for the warehouse estimate source, and adds fixture-based + gated-live tests to pin behavior without requiring CI Snowflake access.

Changes:

  • Add SnowflakeAdapter.estimate_query_bytes implementation + pure JSON-plan parser.
  • Introduce EstimateUnavailableError (tier-3) and update engine/CLI/reporting to degrade gracefully when EXPLAIN lacks a byte stat.
  • Add offline fixtures + unit tests + @pytest.mark.snowflake gated live certification; update docs/rules/plan accordingly.

Reviewed changes

Copilot reviewed 20 out of 20 changed files in this pull request and generated 5 comments.

Show a summary per file
File Description
src/signalforge/warehouse/adapters/snowflake.py Adds EXPLAIN-based estimation, scalar-exec helper, and pure bytesAssigned parser.
src/signalforge/warehouse/errors.py Adds EstimateUnavailableError and exports it via __all__.
src/signalforge/warehouse/__init__.py Re-exports EstimateUnavailableError at package level.
src/signalforge/cli/_helpers.py Registers EstimateUnavailableError in exit-code mapping (tier 3).
src/signalforge/cli/_estimate.py Adds warehouse_estimate_source to report + renders estimate source label (BQ vs Snowflake).
tests/warehouse/test_snowflake_stub.py Updates Snowflake stub contract tests to cover real EXPLAIN estimation + missing-stat degrade.
tests/warehouse/test_snowflake_estimate.py New: pins pure parser behavior and adapter override behavior using fixtures + fakes.
tests/warehouse/test_snowflake_estimate_live.py New: gated live Snowflake tests validating EXPLAIN shape and non-negative parsed int.
tests/warehouse/test_errors.py Adds coverage for EstimateUnavailableError inheritance, rendering, remediation lock, and distinction from NotSupported.
tests/cli/test_estimate_engine.py Updates estimate-engine tests for Snowflake happy-path bytes + missing-stat degrade and source labeling.
tests/cli/test_generate_estimate.py Updates CLI --estimate tests for Snowflake happy-path bytes + missing-stat degrade.
tests/cli/test_exit_codes.py Adds probe construction and explicit mapping assertion for EstimateUnavailableError.
tests/cli/test_estimate_render.py Updates renderer test report setup to include the new source label field.
tests/fixtures/warehouse/snowflake/README.md New: documents fixture purpose and maintainer regeneration instructions.
tests/fixtures/warehouse/snowflake/explain_using_json_sample.json New: sample EXPLAIN JSON fixture containing GlobalStats.bytesAssigned.
tests/fixtures/warehouse/snowflake/explain_using_json_no_stats.json New: EXPLAIN JSON fixture missing GlobalStats for degrade-path tests.
docs/warehouse-adapter-ops.md Documents Snowflake EXPLAIN estimation + accuracy caveat + new degrade error.
.claude/rules/warehouse-adapters.md Updates adapter conventions/rules to reflect Snowflake estimation “graduation” pattern.
pyproject.toml Updates snowflake marker description to include the gated live certification.
plans/super/130-snowflake-estimate-explain.md Adds/updates the “super plan” document for #130.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread src/signalforge/warehouse/adapters/snowflake.py Outdated
Comment thread src/signalforge/warehouse/adapters/snowflake.py Outdated
Comment thread tests/warehouse/test_snowflake_estimate.py Outdated
Comment thread tests/fixtures/warehouse/snowflake/README.md Outdated
Comment thread plans/super/130-snowflake-estimate-explain.md

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 10

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
tests/warehouse/test_errors.py (1)

14-35: ⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

Use warehouse public exports in tests, not private errors module paths.

At Line 14 and Line 407, import from signalforge.warehouse instead of signalforge.warehouse.errors to keep tests aligned with the package contract.

Proposed fix
-from signalforge.warehouse.errors import (
+from signalforge.warehouse import (
     BytesBilledExceededError,
     ColumnNotFoundError,
     EstimateUnavailableError,
@@
     WarehouseError,
 )
@@
-    from signalforge.warehouse.errors import EstimateNotSupportedError
+    from signalforge.warehouse import EstimateNotSupportedError

As per coding guidelines, "Package imports: import from the public API surface (re-exported names from subpackage __init__.py files) rather than private submodule paths."

Also applies to: 407-407

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tests/warehouse/test_errors.py` around lines 14 - 35, Replace direct imports
from the private submodule signalforge.warehouse.errors with imports from the
package public surface signalforge.warehouse; specifically update the import
statement that currently brings in BytesBilledExceededError,
ColumnNotFoundError, EstimateUnavailableError, IncompleteProfileError,
InvalidIdentifierError, ManifestProjectNotFoundError,
ManifestSchemaNotFoundError, MaterialisationFailedError,
MaterialisationNotSupportedError, ProfileNotFoundError,
ProfileTargetNotFoundError, QuerySyntaxError, SamplingError,
SamplingRequiresPartitionFilterError, TableNotFoundError, UnknownTableSizeError,
UnsupportedAuthMethodError, UnsupportedProfileTypeError, WarehouseAuthError, and
WarehouseError so they are imported from signalforge.warehouse (do the same
replacement for the second occurrence that imports the same error symbols).
🧹 Nitpick comments (1)
tests/warehouse/test_snowflake_estimate.py (1)

159-163: ⚡ Quick win

Type _record explicitly instead of suppressing with type: ignore.

This helper can be fully typed and avoid no-untyped-def suppression, which keeps pyright strictness meaningful.

As per coding guidelines, "Type annotations: Use full type annotations on all function signatures, class attributes, and module-level variables... Use # pyright: ignore sparingly with an inline comment explaining the exception."

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tests/warehouse/test_snowflake_estimate.py` around lines 159 - 163, Add
explicit typing to the helper: change def _record(sql: str):  # type:
ignore[no-untyped-def] to include a return annotation (e.g., def _record(sql:
str) -> Any:) and import Any from typing, and remove the no-untyped-def
suppression; then assign the function to fake._consume_execute without
suppressing method-assign by using a proper Callable type or cast (e.g.,
annotate/declare fake._consume_execute as Callable[[str], Any] or use
typing.cast when doing fake._consume_execute = _record) so the assignment and
function are fully typed; references: _record, original, fake._consume_execute.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@docs/warehouse-adapter-ops.md`:
- Around line 370-374: The document contains conflicting statements about
adapter overrides: the paragraph describing Snowflake running `EXPLAIN USING
JSON` and parsing `GlobalStats.bytesAssigned` (and inheriting
`EstimateNotSupportedError`) contradicts the later sentence that "v0.2 ships the
BigQuery override only"; update the latter to reflect the Snowflake behavior (or
vice versa) so both places are consistent—specifically edit the sentence
mentioning "v0.2 ships the BigQuery override only" to either include Snowflake
as also shipping an override or change the earlier Snowflake description to
indicate it is not shipped in v0.2; ensure references to "EXPLAIN USING JSON",
"GlobalStats.bytesAssigned", and "EstimateNotSupportedError" remain accurate
after the change.
- Around line 401-404: The markdown block in the Snowflake section contains
inline code spans with extra inner/trailing spaces (e.g., the literal `EXPLAIN
USING JSON`) that trigger MD038; edit the paragraph referencing
`_sql_safety.validate_test_sql` and the Snowflake override mechanism to remove
any spaces inside backticks so inline code spans are `EXPLAIN USING JSON` (no
leading/trailing spaces) and similarly fix any other inline code spans in that
block to have no inner/trailing spaces.

In `@src/signalforge/warehouse/adapters/snowflake.py`:
- Around line 840-841: In _execute_scalar, the current return logic returns a
mapping row object as-is; update it to normalize mapping rows by extracting and
returning the first cell value (not the full row). Specifically, after assigning
first = rows[0], if first is a Mapping (dict-like), return its first value
(e.g., the first value from first.values()); otherwise keep the existing
behavior that returns first[0] for sequence rows or first for scalar rows.
Ensure you reference the existing symbols rows and first inside _execute_scalar.

In `@tests/cli/test_estimate_engine.py`:
- Line 36: Replace the direct submodule import of SnowflakeAdapter with the
public package re-export: update the import statement that currently imports
SnowflakeAdapter from signalforge.warehouse.adapters.snowflake to import
SnowflakeAdapter from signalforge.warehouse so the test uses the package's
public API surface.

In `@tests/cli/test_exit_codes.py`:
- Line 589: The test imports EstimateUnavailableError from a private submodule;
change the import to use the package public surface by replacing the current
import "from signalforge.warehouse.errors import EstimateUnavailableError" with
"from signalforge.warehouse import EstimateUnavailableError" so the test
references the re-exported symbol from the warehouse package (update the import
in tests/cli/test_exit_codes.py where EstimateUnavailableError is referenced).

In `@tests/cli/test_generate_estimate.py`:
- Line 371: The test uses SnowflakeAdapter via a private import; update the test
to import SnowflakeAdapter from the package's public API (the re-export on
signalforge.warehouse) and keep the call site unchanged
(adapter=SnowflakeAdapter(connection=fake_conn)) so _install_estimate_patches
still receives the adapter instance; also update the other occurrence around the
second mention (line ~410) to use the public import as well.
- Line 40: Add an explicit module-level type annotation for the fixture path by
changing the declaration of _SNOWFLAKE_FIXTURES to include its type (e.g.,
_SNOWFLAKE_FIXTURES: Path = Path(__file__).resolve().parents[1] / "fixtures" /
"warehouse" / "snowflake"); ensure Path is imported from pathlib at the top of
the module if not already.

In `@tests/warehouse/test_snowflake_estimate_live.py`:
- Around line 66-69: The test imports private symbols directly (SnowflakeAdapter
and _parse_explain_json_bytes) from the snowflake submodule; change the imports
to use the public package re-exports instead (import SnowflakeAdapter from the
warehouse package public API) and remove/replace any usage of the private helper
_parse_explain_json_bytes with public-facing behavior or a public helper
exported by the package so the test asserts against the adapter's public API
rather than internal module symbols.

In `@tests/warehouse/test_snowflake_estimate.py`:
- Around line 143-147: The test currently uses pytest.raises(Exception) which is
too broad; change it to assert the specific validation exception thrown by the
SQL-safety check (e.g., pytest.raises(ValidationError) or the concrete
QueryValidationError used in your codebase) when calling
adapter.estimate_query_bytes("SELECT 1; DROP TABLE x"), import that exception at
top of the test, keep the assert that "unexpected query" not in
str(excinfo.value), and leave fake.assert_all_expectations_met() unchanged so
the test only passes for the intended validation reject path.

In `@tests/warehouse/test_snowflake_stub.py`:
- Around line 34-37: Replace direct imports from internal modules by importing
re-exported symbols from the package public API: stop importing SnowflakeAdapter
from signalforge.warehouse.adapters.snowflake and EstimateUnavailableError from
signalforge.warehouse.errors; instead import SnowflakeAdapter and
EstimateUnavailableError from signalforge.warehouse while leaving other imports
(e.g., WarehouseAdapter, SNOWFLAKE_DIALECT, Dialect, TableRef) unchanged or also
moved to the public surface if available; update the import statements in
tests/warehouse/test_snowflake_stub.py to reference SnowflakeAdapter and
EstimateUnavailableError via signalforge.warehouse.

---

Outside diff comments:
In `@tests/warehouse/test_errors.py`:
- Around line 14-35: Replace direct imports from the private submodule
signalforge.warehouse.errors with imports from the package public surface
signalforge.warehouse; specifically update the import statement that currently
brings in BytesBilledExceededError, ColumnNotFoundError,
EstimateUnavailableError, IncompleteProfileError, InvalidIdentifierError,
ManifestProjectNotFoundError, ManifestSchemaNotFoundError,
MaterialisationFailedError, MaterialisationNotSupportedError,
ProfileNotFoundError, ProfileTargetNotFoundError, QuerySyntaxError,
SamplingError, SamplingRequiresPartitionFilterError, TableNotFoundError,
UnknownTableSizeError, UnsupportedAuthMethodError, UnsupportedProfileTypeError,
WarehouseAuthError, and WarehouseError so they are imported from
signalforge.warehouse (do the same replacement for the second occurrence that
imports the same error symbols).

---

Nitpick comments:
In `@tests/warehouse/test_snowflake_estimate.py`:
- Around line 159-163: Add explicit typing to the helper: change def
_record(sql: str):  # type: ignore[no-untyped-def] to include a return
annotation (e.g., def _record(sql: str) -> Any:) and import Any from typing, and
remove the no-untyped-def suppression; then assign the function to
fake._consume_execute without suppressing method-assign by using a proper
Callable type or cast (e.g., annotate/declare fake._consume_execute as
Callable[[str], Any] or use typing.cast when doing fake._consume_execute =
_record) so the assignment and function are fully typed; references: _record,
original, fake._consume_execute.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 3882ca6d-300b-4f62-bb27-5304c6194726

📥 Commits

Reviewing files that changed from the base of the PR and between f903570 and a1688b7.

📒 Files selected for processing (20)
  • .claude/rules/warehouse-adapters.md
  • docs/warehouse-adapter-ops.md
  • plans/super/130-snowflake-estimate-explain.md
  • pyproject.toml
  • src/signalforge/cli/_estimate.py
  • src/signalforge/cli/_helpers.py
  • src/signalforge/warehouse/__init__.py
  • src/signalforge/warehouse/adapters/snowflake.py
  • src/signalforge/warehouse/errors.py
  • tests/cli/test_estimate_engine.py
  • tests/cli/test_estimate_render.py
  • tests/cli/test_exit_codes.py
  • tests/cli/test_generate_estimate.py
  • tests/fixtures/warehouse/snowflake/README.md
  • tests/fixtures/warehouse/snowflake/explain_using_json_no_stats.json
  • tests/fixtures/warehouse/snowflake/explain_using_json_sample.json
  • tests/warehouse/test_errors.py
  • tests/warehouse/test_snowflake_estimate.py
  • tests/warehouse/test_snowflake_estimate_live.py
  • tests/warehouse/test_snowflake_stub.py

Comment thread docs/warehouse-adapter-ops.md
Comment thread docs/warehouse-adapter-ops.md Outdated
Comment thread src/signalforge/warehouse/adapters/snowflake.py
Comment thread tests/cli/test_estimate_engine.py Outdated
Comment thread tests/cli/test_exit_codes.py Outdated
Comment thread tests/cli/test_generate_estimate.py Outdated
Comment thread tests/cli/test_generate_estimate.py
Comment thread tests/warehouse/test_snowflake_estimate_live.py Outdated
Comment thread tests/warehouse/test_snowflake_estimate.py Outdated
Comment thread tests/warehouse/test_snowflake_stub.py Outdated
- _execute_scalar: close the cursor in a finally so repeated estimate calls
  don't leak server-side cursors (the cursor I introduced in this PR).
- _parse_explain_json_bytes docstring: clarify an explicit bytesAssigned=0 is a
  valid estimate; the "never 0" rule only forbids fabricating a 0 fallback when
  the stat is missing/unparseable (was internally inconsistent with the tests).
- semicolon-reject test: narrow pytest.raises(Exception) -> QuerySyntaxError so
  it can't pass on an unrelated exception.
- fixtures README: keep the fully-qualified parser path on one line so the
  Markdown inline code span renders.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@wjduenow

Copy link
Copy Markdown
Owner Author

PR Review Summary

All 5 review comments addressed (commit d02b050 + plan/PR-body alignment). No false positives — every comment was a real fix.

Fixed (5 items)

File Line Issue Resolution
src/signalforge/warehouse/adapters/snowflake.py 837 _execute_scalar cursor never closed Close cursor in a finally (success + both raise paths)
src/signalforge/warehouse/adapters/snowflake.py 145 Docstring "NEVER 0" inconsistent with tests treating bytesAssigned=0 as valid Reworded: explicit 0 is valid; "never 0" forbids only a fabricated fallback
tests/warehouse/test_snowflake_estimate.py 143 pytest.raises(Exception) too broad Narrowed to QuerySyntaxError (what validate_test_sql raises on ;)
tests/fixtures/warehouse/snowflake/README.md 6 Inline code span split across newline broke Markdown Fully-qualified parser path kept on one line
plans/super/130-...md / PR body 9 PR body said "detailing (awaiting approval)" vs plan "devolved" PR body updated to "implemented / devolved" to match

False Positives (0 items)

None.

All review threads resolved. Validation green: 2369 passed, snowflake.py 100%.

- Promote SnowflakeAdapter to the public signalforge.warehouse surface (mirrors
  BigQueryAdapter; CLAUDE.md already lists it as public) + __all__ sorted;
  migrate all 8 Snowflake test files to the public import.
- _execute_scalar: normalise a DictCursor-style mapping row to its first cell
  (was returning the whole row dict → false degrade) + test.
- Switch flagged EstimateUnavailableError imports to the package surface.
- test_generate_estimate: annotate module-level _SNOWFLAKE_FIXTURES: Path.
- live test: keep the genuinely-private _parse_explain_json_bytes white-box
  import (no public seam) with a justifying comment; SnowflakeAdapter public.
- docs: resolve the "v0.2 ships BigQuery override only" contradiction (Snowflake
  overrides too; Postgres still inherits) + fix MD038 trailing space in code span.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@wjduenow

Copy link
Copy Markdown
Owner Author

PR Review Summary (CodeRabbit batch)

All 10 CodeRabbit threads addressed in commit e80c1a1. No false positives.

Fixed (10 items)

File Issue Resolution
src/.../adapters/snowflake.py _execute_scalar returned a whole DictCursor row instead of the first cell Normalise mapping rows to next(iter(first.values())) + regression test
tests/cli/test_estimate_engine.py SnowflakeAdapter imported from private submodule Promoted SnowflakeAdapter to the public signalforge.warehouse surface (matches BigQueryAdapter); import migrated
tests/cli/test_generate_estimate.py (import) same migrated to public surface
tests/cli/test_generate_estimate.py:40 module-level var lacked annotation _SNOWFLAKE_FIXTURES: Path
tests/cli/test_exit_codes.py:589 EstimateUnavailableError from private submodule from signalforge.warehouse
tests/warehouse/test_snowflake_stub.py SnowflakeAdapter + EstimateUnavailableError private imports public surface
tests/warehouse/test_snowflake_estimate_live.py SnowflakeAdapter private import public; _parse_explain_json_bytes kept as a documented white-box import (no public seam exists for a private pure helper)
tests/warehouse/test_snowflake_estimate.py over-broad pytest.raises(Exception) already narrowed to QuerySyntaxError in d02b050
docs/warehouse-adapter-ops.md (override status) "v0.2 ships the BigQuery override only" contradicted the Snowflake-override paragraph reworded: BigQuery + Snowflake override, Postgres inherits
docs/warehouse-adapter-ops.md (MD038) trailing space inside `EXPLAIN USING JSON ` code span removed in-span space; trailing space described in prose

Consistency bonus: migrated all 8 Snowflake test files (incl. pre-existing #119/#122 ones) to the public SnowflakeAdapter import so the convention is uniform.

False Positives (0)

None.

Validation green: 2370 passed, snowflake.py 100%, ruff/pyright/mkdocs clean.

@wjduenow
wjduenow merged commit eb663e5 into dev May 27, 2026
6 checks passed
@wjduenow
wjduenow deleted the feature/130-snowflake-estimate-explain branch May 27, 2026 03:20
@coderabbitai coderabbitai Bot mentioned this pull request May 28, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants