Skip to content

test(silver): enumerate handlers + per-handler raise check#154

Merged
Artic0din merged 5 commits into
devfrom
fix/silver-test-tighter
May 24, 2026
Merged

test(silver): enumerate handlers + per-handler raise check#154
Artic0din merged 5 commits into
devfrom
fix/silver-test-tighter

Conversation

@Artic0din

Copy link
Copy Markdown
Owner

Hardens test_handlers_raise_home_assistant_error_on_missing_coordinator after your question — the old hard-coded >= 3 count threshold let handle_reset_today slip through silently. New test enumerates handlers via regex, checks each body for the raise, reports specific missing handlers.

Test-only change — no manifest bump, no HACS deploy. Will wait for gemini review before merging.

The prior assertion src.count("raise HomeAssistantError(") >= 3
hard-coded the handler count at the time of writing. When
handle_reset_today was added in beta.8 without a raise, the test stayed
green because the count was still ≥ 3 — the new handler just didn't
add to it. Gemini caught the compliance gap on PR #152 review.

Test now enumerates handlers via regex on async def handle_<name>,
checks each handler's body contains raise HomeAssistantError(, and
reports specific missing handlers in the assertion message. Threshold
auto-scales: adding a new handler without the raise breaks the test.

Also asserts >= 4 handlers exist so a future refactor that accidentally
removes one trips the test.

No manifest bump — test-only change, no HACS deploy needed.

Full test suite: 1140 passing.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request updates the test suite to dynamically verify that all service handlers in __init__.py raise HomeAssistantError on unrecoverable conditions, replacing a static count check. The review feedback correctly identifies that the regex-based approach for parsing function bodies is fragile and prone to false positives, particularly for the last handler in the file. It is recommended to use the ast module for more robust code inspection as suggested.

Comment thread tests/test_silver_checklist.py Outdated
Comment on lines +179 to +217
import re as _re
src = (
REPO / "custom_components" / "pricehawk" / "__init__.py"
).read_text()
# At least one raise per handler — count must match the three handlers.
assert src.count("raise HomeAssistantError(") >= 3

# Each handler is an ``async def handle_<name>(call: object) -> None``
# nested inside async_setup_entry. Split the file at handler defs
# and check the body of each contains at least one raise.
handler_starts = [
m for m in _re.finditer(
r"^ async def handle_(\w+)\(call: object\) -> None:",
src, _re.MULTILINE,
)
]
assert len(handler_starts) >= 4, (
f"Expected at least 4 service handlers in __init__.py, "
f"found {len(handler_starts)}. Update this test if handlers "
f"were intentionally removed."
)

missing_raises: list[str] = []
for i, start in enumerate(handler_starts):
handler_name = start.group(1)
body_start = start.end()
body_end = (
handler_starts[i + 1].start()
if i + 1 < len(handler_starts)
else len(src)
)
handler_body = src[body_start:body_end]
if "raise HomeAssistantError(" not in handler_body:
missing_raises.append(handler_name)

assert not missing_raises, (
f"Silver action-exceptions: these handlers don't raise "
f"HomeAssistantError anywhere in their body: {missing_raises}. "
f"Every service handler must raise on unrecoverable conditions "
f"(missing coordinator, no entries, etc)."
)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The current regex-based approach is fragile and contains a logic error in the body_end calculation. For the last handler, body_end defaults to len(src), which includes the entire remainder of the file (e.g., async_unload_entry). If any subsequent code raises HomeAssistantError, the test will produce a false positive for the last handler. Additionally, the regex is highly sensitive to exact whitespace and type hints.

Using the ast module provides a much more robust way to inspect function bodies and correctly bounds the search to each handler's scope.

        import ast
        src = (REPO / "custom_components" / "pricehawk" / "__init__.py").read_text()
        tree = ast.parse(src)

        # Find the function where handlers are defined
        reg_func = next(
            (n for n in tree.body if isinstance(n, ast.FunctionDef) and n.name == "_register_services_once"),
            None
        )
        assert reg_func, "_register_services_once not found in __init__.py"

        # Find all nested async functions starting with 'handle_'
        handlers = [
            n for n in reg_func.body
            if isinstance(n, ast.AsyncFunctionDef) and n.name.startswith("handle_")
        ]

        assert len(handlers) >= 4, f"Expected >= 4 handlers, found {len(handlers)}"

        missing_raises = []
        for handler in handlers:
            # Check if HomeAssistantError is raised in the handler's scope
            has_raise = any(
                isinstance(node, ast.Raise) and node.exc and (
                    (isinstance(node.exc, ast.Call) and getattr(node.exc.func, "id", None) == "HomeAssistantError") or
                    (getattr(node.exc, "id", None) == "HomeAssistantError")
                )
                for node in ast.walk(handler)
            )
            if not has_raise:
                missing_raises.append(handler.name)

        assert not missing_raises, (
            f"Silver action-exceptions: these handlers don't raise "
            f"HomeAssistantError anywhere in their body: {missing_raises}. "
            f"Every service handler must raise on unrecoverable conditions "
            f"(missing coordinator, no entries, etc)."
        )

…view)

Gemini flagged the regex-based approach as fragile: the last handler's
sliced 'body' extended to EOF, swallowing post-handler text (service
registrations, future module-level code) into false positives.

Rewrote with ast: parse the file, walk to AsyncFunctionDef nodes named
handle_*, then per-handler walk its body checking for
raise HomeAssistantError(...). Nested AsyncFunctionDef nodes are
skipped (they're their own scope, validated when they're top-level).

Self-test: ast finds 4 handlers (analyze_csv, backfill,
rank_alternatives, reset_today) — same as ground truth.
@Artic0din

Copy link
Copy Markdown
Owner Author

/gemini review

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request updates the test_silver_checklist.py file to replace a fragile string-based check with a more robust AST-based validation. The new test ensures that every service handler in __init__.py raises a HomeAssistantError for unrecoverable conditions. Feedback was provided regarding the implementation of the AST traversal; specifically, the current use of ast.walk fails to correctly prune nested function definitions, which could lead to false positives. A code suggestion was offered to implement a manual traversal that properly excludes nested scopes.

Comment thread tests/test_silver_checklist.py Outdated
Comment on lines +201 to +212
for child in ast.walk(stmt):
if isinstance(child, ast.AsyncFunctionDef):
# Don't descend into nested defs — they're their
# own scope and validated separately.
continue
if not isinstance(child, ast.Raise):
continue
exc = child.exc
if isinstance(exc, ast.Call) and isinstance(
exc.func, ast.Name
) and exc.func.id == "HomeAssistantError":
return True

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The current implementation of _body_has_raise_hae does not correctly prune nested function definitions. ast.walk is a flat generator that yields all nodes in the subtree; calling continue when an AsyncFunctionDef is encountered only skips the logic for that specific node, but ast.walk will still yield all of its children (including any raise statements inside the nested function) in subsequent iterations. This leads to false positives where a handler is credited for a raise statement that actually belongs to a nested helper function.

To correctly restrict the search to the current function's scope, you should use a traversal that allows skipping subtrees, such as a manual stack-based BFS/DFS or a custom ast.NodeVisitor that does not descend into function nodes. Additionally, you should skip ast.FunctionDef (synchronous functions) as well to be consistent with the intent of excluding nested scopes.

Suggested change
for child in ast.walk(stmt):
if isinstance(child, ast.AsyncFunctionDef):
# Don't descend into nested defs — they're their
# own scope and validated separately.
continue
if not isinstance(child, ast.Raise):
continue
exc = child.exc
if isinstance(exc, ast.Call) and isinstance(
exc.func, ast.Name
) and exc.func.id == "HomeAssistantError":
return True
todo = [stmt]
while todo:
node = todo.pop()
if isinstance(node, (ast.FunctionDef, ast.AsyncFunctionDef)):
# Don't descend into nested defs — they're their
# own scope and validated separately.
continue
if isinstance(node, ast.Raise):
exc = node.exc
if isinstance(exc, ast.Call) and isinstance(
exc.func, ast.Name
) and exc.func.id == "HomeAssistantError":
return True
todo.extend(ast.iter_child_nodes(node))

ast.walk is a flat generator — yields all descendants. My continue
on encountered AsyncFunctionDef only skipped that node, not its
subtree, so a raise HomeAssistantError inside a nested helper would
still satisfy the check for the outer handler.

Replaced with manual stack-based DFS using ast.iter_child_nodes.
Encountering a FunctionDef/AsyncFunctionDef now skips the entire
subtree, not just the def node.

Self-verified: synthesised a handler with nested helper that raises
HAE, check correctly returns False (nested doesn't count for outer).

Per gemini suggestion on PR #154 inline review.
@Artic0din

Copy link
Copy Markdown
Owner Author

/gemini review

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request refactors the test_every_service_handler_raises_home_assistant_error in tests/test_silver_checklist.py to use an AST-based approach instead of simple string counting. This improvement allows for more precise validation that every service handler correctly raises HomeAssistantError within its own scope. Feedback suggests extending the check to support synchronous function definitions, including ServiceValidationError as a valid exception, and ensuring nested class definitions are correctly pruned during the AST walk.

Comment thread tests/test_silver_checklist.py Outdated
Comment on lines +190 to +197
def _iter_async_funcs(node):
"""Yield AsyncFunctionDef nodes recursively (handlers may be
nested inside async_setup_entry, etc)."""
for child in ast.walk(node):
if isinstance(child, ast.AsyncFunctionDef):
yield child

def _body_has_raise_hae(func: ast.AsyncFunctionDef) -> bool:

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Service handlers in Home Assistant can be implemented as either synchronous (def) or asynchronous (async def) functions. To ensure full coverage of the Silver action-exceptions rule, the test should enumerate both types. I've also updated the helper name and type hint to reflect this broader scope.

Suggested change
def _iter_async_funcs(node):
"""Yield AsyncFunctionDef nodes recursively (handlers may be
nested inside async_setup_entry, etc)."""
for child in ast.walk(node):
if isinstance(child, ast.AsyncFunctionDef):
yield child
def _body_has_raise_hae(func: ast.AsyncFunctionDef) -> bool:
def _iter_funcs(node):
"""Yield FunctionDef and AsyncFunctionDef nodes recursively (handlers may be
nested inside async_setup_entry, etc)."""
for child in ast.walk(node):
if isinstance(child, (ast.FunctionDef, ast.AsyncFunctionDef)):
yield child
def _body_has_raise_hae(func: ast.FunctionDef | ast.AsyncFunctionDef) -> bool:

Comment thread tests/test_silver_checklist.py Outdated
todo: list[ast.AST] = [stmt]
while todo:
node = todo.pop()
if isinstance(node, (ast.FunctionDef, ast.AsyncFunctionDef)):

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

To correctly prune all nested scopes during the AST walk, ast.ClassDef should be included in the skip list. While unlikely in a service handler, a raise statement directly within a nested class body (executed at definition time) would currently be incorrectly attributed to the outer handler function.

Suggested change
if isinstance(node, (ast.FunctionDef, ast.AsyncFunctionDef)):
if isinstance(node, (ast.FunctionDef, ast.AsyncFunctionDef, ast.ClassDef)):

Comment thread tests/test_silver_checklist.py Outdated
if (
isinstance(exc, ast.Call)
and isinstance(exc.func, ast.Name)
and exc.func.id == "HomeAssistantError"

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The Silver "action-exceptions" rule requires raising HomeAssistantError or one of its subclasses. ServiceValidationError is a common subclass used for input validation in this integration. Including it in this check prevents false negatives for handlers that might only perform input validation (though currently all handlers in this integration also perform a coordinator check).

Suggested change
and exc.func.id == "HomeAssistantError"
and exc.func.id in ("HomeAssistantError", "ServiceValidationError")

Comment thread tests/test_silver_checklist.py Outdated
return False

handlers = [
f for f in _iter_async_funcs(tree)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Update the call to use the renamed helper function that includes both sync and async function definitions.

Suggested change
f for f in _iter_async_funcs(tree)
f for f in _iter_funcs(tree)

…efs (gemini #154 review-3)

Gemini's third pass on PR #154 flagged three remaining gaps:

1. Silver action-exceptions accepts EITHER HomeAssistantError OR
   ServiceValidationError per HA docs. The check accepted only HAE,
   so a future handler that legitimately raises only SVE for user-
   input validation would fail the test. Now accepts both via a
   frozenset.

2. ClassDef wasn't in the prune list. A handler with an inline
   helper class whose method raises HAE/SVE would wrongly satisfy
   the check for the outer handler. Added to the skip set.

3. Sync function handlers (def handle_x, not async def) were missed
   entirely. The iter helper only yielded AsyncFunctionDef. Now
   yields both — a future non-async handler still gets validated.

Self-verified each: nested-class raise no longer counts toward the
outer handler (negative test in the commit msg below).

Negative test snippet:
  async def handle_x(call):
    class Helper:
      def fail(self): raise HomeAssistantError('inner')
    pass
DFS result: False ✓ (correctly does NOT count the class method's raise)
@Artic0din

Copy link
Copy Markdown
Owner Author

/gemini review

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request replaces a fragile string-based test with a robust AST-based validation to ensure all service handlers in __init__.py raise required Home Assistant exceptions. The new implementation correctly handles nested scopes and scales with the number of handlers. A review comment suggests enhancing the detection logic to include exceptions raised without parentheses, which would make the test more robust against different coding styles.

Comment on lines +234 to +239
if (
isinstance(exc, ast.Call)
and isinstance(exc.func, ast.Name)
and exc.func.id in _ACTION_EXCEPTIONS
):
return True

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The current logic only detects exceptions raised as calls (e.g., raise HomeAssistantError(...)). It will miss cases where the exception class is raised directly without parentheses (e.g., raise HomeAssistantError), which is valid Python and occasionally used in Home Assistant integrations. Expanding the check to handle both ast.Call and ast.Name makes the test more robust against different coding styles.

                        if isinstance(exc, ast.Call):
                            exc = exc.func
                        if isinstance(exc, ast.Name) and exc.id in _ACTION_EXCEPTIONS:
                            return True

raise HomeAssistantError (no parens) is valid Python and used in some
HA integrations. Prior code only matched ast.Call wrapping ast.Name —
the bare form was an ast.Name directly on node.exc.

Per gemini suggestion: unify both forms by unwrapping ast.Call to its
.func first, then doing the single ast.Name check at the end.

Self-verified: bare raise HomeAssistantError correctly detected.
@Artic0din

Copy link
Copy Markdown
Owner Author

/gemini review

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request refactors the service handler validation test in tests/test_silver_checklist.py. The previous implementation, which relied on simple string counting, has been replaced with a more robust approach using the ast module to parse and inspect function bodies. This ensures that every service handler (prefixed with handle_) explicitly raises HomeAssistantError or ServiceValidationError while correctly ignoring nested scopes and supporting various raise syntax forms. I have no feedback to provide as there were no review comments.

@Artic0din Artic0din merged commit bfa9e2c into dev May 24, 2026
8 of 11 checks passed
@Artic0din Artic0din deleted the fix/silver-test-tighter branch May 24, 2026 06:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant