Skip to content

Replace type: ignore[attr-defined] with isinstance assertions in tests#2665

Merged
jlowin merged 3 commits intomainfrom
type-ignore
Dec 21, 2025
Merged

Replace type: ignore[attr-defined] with isinstance assertions in tests#2665
jlowin merged 3 commits intomainfrom
type-ignore

Conversation

@jlowin
Copy link
Copy Markdown
Member

@jlowin jlowin commented Dec 21, 2025

Replaced type: ignore[attr-defined] annotations with proper isinstance type assertions in test files to improve type safety and code clarity.

  • Added isinstance checks before accessing attributes like .text, .content, .data, etc.
  • Fixed syntax errors in test files
  • Kept type: ignore annotations only for:
    • json_schema tests (as requested)
    • Private/internal attributes (_injected_values, _subscription_task_group, etc.)
    • Runtime attributes not recognized by type checker (__name__ on command objects)
    • Private httpx internal attributes

This improves type checking accuracy while maintaining test functionality.

@marvin-context-protocol marvin-context-protocol Bot added enhancement Improvement to existing functionality. For issues and smaller PR improvements. tests labels Dec 21, 2025
@marvin-context-protocol
Copy link
Copy Markdown
Contributor

marvin-context-protocol Bot commented Dec 21, 2025

Test Failure Analysis

Updated: Added test failure analysis from latest workflow run.


❌ Test Failures (4 tests failing)

Summary: The PR replaced type: ignore[attr-defined] with isinstance assertions, but several of these assertions are incorrect - they assert the wrong type based on what the code actually returns at runtime.

Root Cause: The isinstance checks were added based on what the test expected the type to be, not what it actually is. This is fundamentally different from the original # type: ignore[attr-defined] approach, which only suppressed type checker warnings without making runtime assertions.

Failed Tests:

  1. test_elicitation_enum_response - Asserts result.data is str, but it's actually ResponseEnum.X
  2. test_resource_content_binary_with_meta - Asserts result[0] is TextResourceContents, but binary returns BlobResourceContents
  3. test_output_schema_explicit_object_full_handshake - Asserts result.data is dict, but client deserializes to Root Pydantic model
  4. test_output_schema_dataclass_full_handshake - Asserts result.data is User, but client deserializes to Root Pydantic model

Solution: Revert the isinstance assertions to the original # type: ignore[attr-defined] comments in these specific tests. The type: ignore approach is correct here because:

  • It only suppresses type checker warnings, not runtime behavior
  • The tests verify runtime behavior through attribute access (e.g., .text, .data)
  • Adding incorrect isinstance checks breaks tests by asserting wrong types

Files to fix:

  • tests/client/test_elicitation.py:314 - Remove isinstance(result.data, str), restore # type: ignore[attr-defined]
  • tests/server/test_server.py:660 - Remove isinstance(result[0], TextResourceContents), restore # type: ignore[attr-defined]
  • tests/server/test_server_interactions.py:1053 - Remove isinstance(result.data, dict), restore # type: ignore[attr-defined]
  • tests/server/test_server_interactions.py:1141 - Remove isinstance(result.data, User), restore # type: ignore[attr-defined]
Detailed Test Failure Analysis

1. test_elicitation_enum_response (test_elicitation.py:314)

What the test does: Calls a tool that elicits an enum response and returns it.

Assertion: assert isinstance(result.data, str)

Actual type: ResponseEnum.X (enum value, not string)

Error:

AssertionError: assert False
 +  where False = isinstance(<ResponseEnum.X: 'x'>, str)

Why it fails: Enums are not strings. While ResponseEnum.X.value is "x", the enum instance itself is of type ResponseEnum.


2. test_resource_content_binary_with_meta (test_server.py:660)

What the test does: Creates a resource with binary content and meta, verifies it returns with blob.

Assertion: assert isinstance(result[0], TextResourceContents)

Actual type: BlobResourceContents

Error:

AssertionError: assert False
 +  where False = isinstance(BlobResourceContents(uri=AnyUrl('resource://binary'), 
   mimeType='text/plain', meta={'encoding': 'raw'}, blob='AAEC'), TextResourceContents)

Why it fails: Binary content (b"\x00\x01\x02") is returned as BlobResourceContents, not TextResourceContents. The test comment even says "Binary content comes back as blob".


3. test_output_schema_explicit_object_full_handshake (test_server_interactions.py:1053)

What the test does: Tests tool with explicit output schema, verifies client deserializes structured content.

Assertion: assert isinstance(result.data, dict)

Actual type: Root (Pydantic model)

Error:

AssertionError: assert False
 +  where False = isinstance(Root(greeting='Hello', count=42), dict)
 +    where Root(greeting='Hello', count=42) = CallToolResult(...).data

Why it fails: The client deserializes the JSON response into a Pydantic Root model based on the output schema, not a plain dict.


4. test_output_schema_dataclass_full_handshake (test_server_interactions.py:1141)

What the test does: Tests tool with dataclass output, verifies client deserializes to the dataclass.

Assertion: assert isinstance(result.data, User)

Actual type: Root (Pydantic model, not the test's local User dataclass)

Error:

AssertionError: assert False
 +  where False = isinstance(Root(name='Alice', age=30), User)

Why it fails: The client creates a new Pydantic model (Root) from the JSON schema, not an instance of the test's local User dataclass. The types are incompatible even though they have the same fields.


❌ Type Check Failures

Summary: The ty check step fails because the PR replaced type: ignore[attr-defined] comments with isinstance checks, but these checks don't provide sufficient type narrowing for the type checker in several test scenarios.

Root Cause: The type checker (ty) is flagging two distinct categories of issues:

  1. Insufficient type narrowing (lines 179-180, 199-200 in test_elicitation.py): Checking result.action == "accept" followed by isinstance(result.data, dict) doesn't narrow the union type AcceptedElicitation[dict[str, Any]] | DeclinedElicitation | CancelledElicitation enough for the type checker to know that result.data exists.

  2. Intentional type errors in tests (lines 141, 1055, 1070 in test_tool_manager.py and line 955 in test_tool_transform.py): Tests that verify error handling by passing invalid types are now correctly flagged by the type checker, even though the code is intentionally wrong to test error conditions.

Suggested Solution:

For cases where isinstance checks don't provide enough type narrowing, keep the original # type: ignore[attr-defined] approach. These are legitimate uses where:

  • The attribute exists at runtime but the type checker can't prove it
  • The union type structure prevents proper narrowing
  • Runtime validation (like checking .action == "accept") ensures correctness

For tests that intentionally pass wrong types to verify error handling, use # type: ignore[arg-type] or # type: ignore[invalid-argument-type] to suppress the specific type checker warnings while still validating runtime behavior.

Files to modify:

  • tests/client/test_elicitation.py: Lines 179-180, 199-200 (keep # type: ignore[attr-defined])
  • tests/tools/test_tool_manager.py: Lines 141, 1055, 1070 (add # type: ignore[arg-type])
  • tests/tools/test_tool_transform.py: Line 955 (add # type: ignore[arg-type])
Type Check Detailed Analysis

Type Narrowing Issue in test_elicitation.py

The elicit method has overloaded signatures:

async def elicit(
    self,
    message: str,
    response_type: None,
) -> (
    AcceptedElicitation[dict[str, Any]] | DeclinedElicitation | CancelledElicitation
)

When calling with response_type=None, the return type is a union. The code checks:

result = await context.elicit(message="", response_type=None)
assert result.action == "accept"
assert isinstance(result.data, dict)
return result.data

While checking .action == "accept" should narrow to AcceptedElicitation, the type checker (ty) doesn't recognize this pattern. The isinstance(result.data, dict) check doesn't help because the type checker doesn't yet know that result.data exists (it might be a DeclinedElicitation or CancelledElicitation which don't have a .data attribute).

Type checker warnings:

warning[possibly-missing-attribute]: Attribute `data` may be missing on object of type 
`AcceptedElicitation[dict[str, Any]] | DeclinedElicitation | CancelledElicitation`

Intentional Type Errors in Tests

test_tool_manager.py:141:

with pytest.raises(TypeError, match="not a callable object"):
    assert isinstance(1, int)  # Intentionally passing invalid type
    tool = Tool.from_function(1)  # This SHOULD fail at runtime

The isinstance(1, int) check doesn't prevent the type checker from seeing that 1 is being passed where a callable is expected.

Type checker error:

error[invalid-argument-type]: Argument to function `from_function` is incorrect
Expected `(...) -> Any`, found `Literal[1]`

test_tool_manager.py:1055, 1070:

assert isinstance(parent_mcp._providers, list)
parent_mcp._providers.append("invalid")  # Intentionally corrupt for testing

Type checker error:

error[invalid-argument-type]: Argument to bound method `append` is incorrect
Expected `Provider`, found `Literal["invalid"]`

test_tool_transform.py:955:

with pytest.raises(ValueError, match="Cannot specify 'required=False'"):
    Tool.from_tool(
        base_tool,
        transform_args={"required_param": ArgTransform(required=False, default=99)},
    )

Type checker error:

error[invalid-argument-type]: Argument is incorrect
Expected `Literal[True] | EllipsisType`, found `Literal[False]`

All of these are tests that verify error handling works correctly - the type checker is correctly identifying that the code is wrong, which is exactly what we want to happen at runtime.

Related Files

Test files with failures:

  • tests/client/test_elicitation.py - Elicitation tests with incorrect isinstance assertions
  • tests/server/test_server.py - Resource content tests with incorrect type assertions
  • tests/server/test_server_interactions.py - Tool output schema tests with incorrect type assertions
  • tests/tools/test_tool_manager.py - Tool manager tests with intentional error cases
  • tests/tools/test_tool_transform.py - Tool transformation tests with invalid parameters

Source files referenced:

  • src/fastmcp/server/context.py - Defines the elicit method overloads and return types
  • src/fastmcp/server/elicitation.py - Defines AcceptedElicitation, DeclinedElicitation, CancelledElicitation types
  • src/fastmcp/client/client.py - Client deserialization logic for tool results

- Fix enum test to check for ResponseEnum instead of str
- Fix binary resource test to check for BlobResourceContents instead of TextResourceContents
- Fix Root type tests to check attributes directly instead of isinstance checks
- Remove execution methods from TransformingProvider (only handles transformations)
- Add execution methods to base Provider class with default implementations
- Fix type narrowing in tests using cast() instead of type: ignore
- Fix PromptResult type handling in prompt render tests
- Fix type narrowing in middleware test for arguments and structured_content
@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented Dec 21, 2025

Walkthrough

This pull request adds four execution helper methods to the Provider base class (call_tool, read_resource, read_resource_template, render_prompt) that delegate to existing component lookups and operations. These are documented as overridable hooks with graceful None handling when components are absent. The corresponding public methods are removed from TransformingProvider, consolidating the implementation in the base class. Supporting public types (ResourceContent, PromptResult, ToolResult) are introduced via imports. Additionally, TransformingProvider receives internal validation for tool_renames to prevent duplicate target names and includes a reverse mapping for lookups.

Possibly related PRs

  • fastmcp#2622: Introduces the Provider abstraction with the same execution helper methods (call_tool, read_resource, read_resource_template, render_prompt) that overlap directly with this PR's additions.
  • fastmcp#2663: Modifies provider execution APIs in the same files (src/fastmcp/server/providers/base.py and related providers), directly addressing the same public methods being added and removed.

Pre-merge checks and finishing touches

❌ Failed checks (2 warnings)
Check name Status Explanation Resolution
Title check ⚠️ Warning The PR title references replacing type: ignore with isinstance assertions in tests, but the raw summary describes changes to Provider base class and TransformingProvider methods unrelated to type assertions. Update the PR title to accurately reflect the actual changes made to Provider and TransformingProvider classes, or verify the correct summary was provided.
Description check ⚠️ Warning The PR description addresses type assertions and isinstance checks in tests, but raw_summary shows changes to Provider/TransformingProvider base classes. The description lacks required checklist items. Complete the Contributors and Review checklists in the description, and ensure description aligns with actual code changes in raw_summary.
✅ Passed checks (1 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
✨ Finishing touches
  • 📝 Generate docstrings
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch type-ignore

📜 Recent review details

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between d6654a2 and 75c7b6f.

⛔ Files ignored due to path filters (41)
  • AGENTS.md is excluded by none and included by none
  • tests/cli/test_cli.py is excluded by none and included by none
  • tests/cli/test_tasks.py is excluded by none and included by none
  • tests/client/auth/test_oauth_client.py is excluded by none and included by none
  • tests/client/tasks/test_prompt_task_mcp_message.py is excluded by none and included by none
  • tests/client/test_client.py is excluded by none and included by none
  • tests/client/test_elicitation.py is excluded by none and included by none
  • tests/client/test_openapi.py is excluded by none and included by none
  • tests/client/test_sampling.py is excluded by none and included by none
  • tests/client/test_sse.py is excluded by none and included by none
  • tests/client/test_streamable_http.py is excluded by none and included by none
  • tests/client/transports/test_transports.py is excluded by none and included by none
  • tests/deprecated/test_import_server.py is excluded by none and included by none
  • tests/integration_tests/test_github_mcp_remote.py is excluded by none and included by none
  • tests/prompts/test_prompt_manager.py is excluded by none and included by none
  • tests/server/auth/providers/test_azure.py is excluded by none and included by none
  • tests/server/auth/providers/test_descope.py is excluded by none and included by none
  • tests/server/auth/providers/test_scalekit.py is excluded by none and included by none
  • tests/server/auth/providers/test_supabase.py is excluded by none and included by none
  • tests/server/http/test_bearer_auth_backend.py is excluded by none and included by none
  • tests/server/http/test_http_dependencies.py is excluded by none and included by none
  • tests/server/middleware/test_initialization_middleware.py is excluded by none and included by none
  • tests/server/middleware/test_middleware.py is excluded by none and included by none
  • tests/server/middleware/test_tool_injection.py is excluded by none and included by none
  • tests/server/providers/test_fastmcp_provider.py is excluded by none and included by none
  • tests/server/proxy/test_proxy_client.py is excluded by none and included by none
  • tests/server/proxy/test_proxy_server.py is excluded by none and included by none
  • tests/server/tasks/test_task_config_modes.py is excluded by none and included by none
  • tests/server/tasks/test_task_proxy.py is excluded by none and included by none
  • tests/server/test_dependencies.py is excluded by none and included by none
  • tests/server/test_event_store.py is excluded by none and included by none
  • tests/server/test_input_validation.py is excluded by none and included by none
  • tests/server/test_mount.py is excluded by none and included by none
  • tests/server/test_providers.py is excluded by none and included by none
  • tests/server/test_server.py is excluded by none and included by none
  • tests/server/test_server_interactions.py is excluded by none and included by none
  • tests/server/test_tool_annotations.py is excluded by none and included by none
  • tests/tools/test_tool.py is excluded by none and included by none
  • tests/tools/test_tool_manager.py is excluded by none and included by none
  • tests/tools/test_tool_transform.py is excluded by none and included by none
  • tests/utilities/test_json_schema_type.py is excluded by none and included by none
📒 Files selected for processing (2)
  • src/fastmcp/server/providers/base.py (2 hunks)
  • src/fastmcp/server/providers/transforming.py (2 hunks)
🧰 Additional context used
📓 Path-based instructions (1)
src/fastmcp/**/*.py

📄 CodeRabbit inference engine (AGENTS.md)

src/fastmcp/**/*.py: Write Python code with ≥3.10 type annotations required throughout
Never use bare except - be specific with exception types

Files:

  • src/fastmcp/server/providers/transforming.py
  • src/fastmcp/server/providers/base.py
🧠 Learnings (1)
📓 Common learnings
Learnt from: CR
Repo: jlowin/fastmcp PR: 0
File: AGENTS.md:0-0
Timestamp: 2025-12-17T03:06:14.522Z
Learning: Applies to tests/**/*.py : Use # type: ignore[attr-defined] in tests for MCP results instead of type assertions
Learnt from: CR
Repo: jlowin/fastmcp PR: 0
File: AGENTS.md:0-0
Timestamp: 2025-12-17T03:06:14.522Z
Learning: Applies to src/fastmcp/**/*.py : Write Python code with ≥3.10 type annotations required throughout
🧬 Code graph analysis (1)
src/fastmcp/server/providers/transforming.py (1)
src/fastmcp/server/providers/base.py (2)
  • Provider (66-384)
  • TaskComponents (53-63)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (5)
  • GitHub Check: Run tests: Python 3.10 on ubuntu-latest
  • GitHub Check: Run tests: Python 3.13 on ubuntu-latest
  • GitHub Check: Run tests: Python 3.10 on windows-latest
  • GitHub Check: Run tests with lowest-direct dependencies
  • GitHub Check: Analyze (python)
🔇 Additional comments (9)
src/fastmcp/server/providers/base.py (5)

34-39: LGTM!

Import additions are correct and necessary for the new execution method signatures and return types. Uses proper Python 3.10+ style annotations.


236-250: LGTM!

Clean implementation that correctly delegates to get_tool and Tool.run(). The None return for missing tools aligns with the documented provider semantics.


252-267: LGTM!

Correct implementation with proper result normalization via isinstance check and ResourceContent.from_value fallback.


269-288: LGTM!

Sound implementation. The params None-check on line 283-284 is defensive—get_resource_template already filters templates by matches(uri) is not None, but the explicit guard is reasonable for subclass overrides that might return templates without the filter.


290-307: LGTM!

Symmetric implementation with other execution methods. Correctly allows None arguments for prompts that don't require parameters.

src/fastmcp/server/providers/transforming.py (4)

12-24: LGTM!

Proper use of TYPE_CHECKING to avoid runtime import of types only needed for annotations in get_tasks().


77-77: LGTM!

Explicit type annotation improves clarity and helps static analysis tools.


81-92: LGTM!

Robust validation for duplicate target names with an informative error message that identifies both conflicting source names. The reverse mapping is correctly created after validation passes.


271-300: Justified type ignores for model_copy subclass narrowing.

The # type: ignore[arg-type] comments are appropriate here. The model_copy() method returns the base type in the type system, but at runtime it correctly returns the subclass instance. This is a known limitation with Pydantic's model_copy type inference.


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@jlowin jlowin merged commit 049bd22 into main Dec 21, 2025
11 checks passed
@jlowin jlowin deleted the type-ignore branch December 21, 2025 21:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement Improvement to existing functionality. For issues and smaller PR improvements. tests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant