Skip to content
Closed
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
154 changes: 154 additions & 0 deletions tests/test_autofix_validation.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,154 @@
"""Test file with intentional issues to validate autofix system.

This file contains:
1. Black formatting violations
2. Ruff lint errors

Check failure on line 5 in tests/test_autofix_validation.py

View workflow job for this annotation

GitHub Actions / Python CI / lint-ruff

Ruff (W291)

tests/test_autofix_validation.py:5:20: W291 Trailing whitespace

Check failure on line 5 in tests/test_autofix_validation.py

View workflow job for this annotation

GitHub Actions / Python CI / lint-ruff

Ruff (W291)

tests/test_autofix_validation.py:5:20: W291 Trailing whitespace

Check failure on line 5 in tests/test_autofix_validation.py

View workflow job for this annotation

GitHub Actions / Python CI / lint-ruff

Ruff (W291)

tests/test_autofix_validation.py:5:20: W291 Trailing whitespace

Check failure on line 5 in tests/test_autofix_validation.py

View workflow job for this annotation

GitHub Actions / Python CI / lint-ruff

Ruff (W291)

tests/test_autofix_validation.py:5:20: W291 Trailing whitespace
3. Mypy type errors
4. Failing tests
5. Actual useful test coverage

Purpose: Validate the full autofix pipeline handles all failure modes.
"""

# --- BLACK VIOLATION: Bad formatting ---
import os,sys,time

Check failure on line 14 in tests/test_autofix_validation.py

View workflow job for this annotation

GitHub Actions / Python CI / lint-ruff

Ruff (F401)

tests/test_autofix_validation.py:14:15: F401 `time` imported but unused

Check failure on line 14 in tests/test_autofix_validation.py

View workflow job for this annotation

GitHub Actions / Python CI / lint-ruff

Ruff (F401)

tests/test_autofix_validation.py:14:11: F401 `sys` imported but unused

Check failure on line 14 in tests/test_autofix_validation.py

View workflow job for this annotation

GitHub Actions / Python CI / lint-ruff

Ruff (E401)

tests/test_autofix_validation.py:14:1: E401 Multiple imports on one line

Check failure on line 14 in tests/test_autofix_validation.py

View workflow job for this annotation

GitHub Actions / Python CI / lint-ruff

Ruff (F401)

tests/test_autofix_validation.py:14:15: F401 `time` imported but unused

Check failure on line 14 in tests/test_autofix_validation.py

View workflow job for this annotation

GitHub Actions / Python CI / lint-ruff

Ruff (F401)

tests/test_autofix_validation.py:14:11: F401 `sys` imported but unused

Check failure on line 14 in tests/test_autofix_validation.py

View workflow job for this annotation

GitHub Actions / Python CI / lint-ruff

Ruff (E401)

tests/test_autofix_validation.py:14:1: E401 Multiple imports on one line

Check failure on line 14 in tests/test_autofix_validation.py

View workflow job for this annotation

GitHub Actions / Python CI / lint-ruff

Ruff (F401)

tests/test_autofix_validation.py:14:15: F401 `time` imported but unused

Check failure on line 14 in tests/test_autofix_validation.py

View workflow job for this annotation

GitHub Actions / Python CI / lint-ruff

Ruff (F401)

tests/test_autofix_validation.py:14:11: F401 `sys` imported but unused

Check failure on line 14 in tests/test_autofix_validation.py

View workflow job for this annotation

GitHub Actions / Python CI / lint-ruff

Ruff (E401)

tests/test_autofix_validation.py:14:1: E401 Multiple imports on one line

Check failure on line 14 in tests/test_autofix_validation.py

View workflow job for this annotation

GitHub Actions / Python CI / lint-ruff

Ruff (F401)

tests/test_autofix_validation.py:14:15: F401 `time` imported but unused

Check failure on line 14 in tests/test_autofix_validation.py

View workflow job for this annotation

GitHub Actions / Python CI / lint-ruff

Ruff (F401)

tests/test_autofix_validation.py:14:11: F401 `sys` imported but unused

Check failure on line 14 in tests/test_autofix_validation.py

View workflow job for this annotation

GitHub Actions / Python CI / lint-ruff

Ruff (E401)

tests/test_autofix_validation.py:14:1: E401 Multiple imports on one line

Copilot AI Dec 29, 2025

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Import of 'sys' is not used.
Import of 'time' is not used.

Suggested change
import os,sys,time
import os

Copilot uses AI. Check for mistakes.
from typing import Dict,List,Optional,Any

Check failure on line 15 in tests/test_autofix_validation.py

View workflow job for this annotation

GitHub Actions / Python CI / lint-ruff

Ruff (F401)

tests/test_autofix_validation.py:15:30: F401 `typing.Optional` imported but unused

Check failure on line 15 in tests/test_autofix_validation.py

View workflow job for this annotation

GitHub Actions / Python CI / lint-ruff

Ruff (F401)

tests/test_autofix_validation.py:15:25: F401 `typing.List` imported but unused

Check failure on line 15 in tests/test_autofix_validation.py

View workflow job for this annotation

GitHub Actions / Python CI / lint-ruff

Ruff (F401)

tests/test_autofix_validation.py:15:20: F401 `typing.Dict` imported but unused

Check failure on line 15 in tests/test_autofix_validation.py

View workflow job for this annotation

GitHub Actions / Python CI / lint-ruff

Ruff (UP035)

tests/test_autofix_validation.py:15:1: UP035 `typing.List` is deprecated, use `list` instead

Check failure on line 15 in tests/test_autofix_validation.py

View workflow job for this annotation

GitHub Actions / Python CI / lint-ruff

Ruff (UP035)

tests/test_autofix_validation.py:15:1: UP035 `typing.Dict` is deprecated, use `dict` instead

Check failure on line 15 in tests/test_autofix_validation.py

View workflow job for this annotation

GitHub Actions / Python CI / lint-ruff

Ruff (F401)

tests/test_autofix_validation.py:15:30: F401 `typing.Optional` imported but unused

Check failure on line 15 in tests/test_autofix_validation.py

View workflow job for this annotation

GitHub Actions / Python CI / lint-ruff

Ruff (F401)

tests/test_autofix_validation.py:15:25: F401 `typing.List` imported but unused

Check failure on line 15 in tests/test_autofix_validation.py

View workflow job for this annotation

GitHub Actions / Python CI / lint-ruff

Ruff (F401)

tests/test_autofix_validation.py:15:20: F401 `typing.Dict` imported but unused

Check failure on line 15 in tests/test_autofix_validation.py

View workflow job for this annotation

GitHub Actions / Python CI / lint-ruff

Ruff (UP035)

tests/test_autofix_validation.py:15:1: UP035 `typing.List` is deprecated, use `list` instead

Check failure on line 15 in tests/test_autofix_validation.py

View workflow job for this annotation

GitHub Actions / Python CI / lint-ruff

Ruff (UP035)

tests/test_autofix_validation.py:15:1: UP035 `typing.Dict` is deprecated, use `dict` instead

Check failure on line 15 in tests/test_autofix_validation.py

View workflow job for this annotation

GitHub Actions / Python CI / lint-ruff

Ruff (F401)

tests/test_autofix_validation.py:15:30: F401 `typing.Optional` imported but unused

Check failure on line 15 in tests/test_autofix_validation.py

View workflow job for this annotation

GitHub Actions / Python CI / lint-ruff

Ruff (F401)

tests/test_autofix_validation.py:15:25: F401 `typing.List` imported but unused

Check failure on line 15 in tests/test_autofix_validation.py

View workflow job for this annotation

GitHub Actions / Python CI / lint-ruff

Ruff (F401)

tests/test_autofix_validation.py:15:20: F401 `typing.Dict` imported but unused

Check failure on line 15 in tests/test_autofix_validation.py

View workflow job for this annotation

GitHub Actions / Python CI / lint-ruff

Ruff (UP035)

tests/test_autofix_validation.py:15:1: UP035 `typing.List` is deprecated, use `list` instead

Check failure on line 15 in tests/test_autofix_validation.py

View workflow job for this annotation

GitHub Actions / Python CI / lint-ruff

Ruff (UP035)

tests/test_autofix_validation.py:15:1: UP035 `typing.Dict` is deprecated, use `dict` instead

Check failure on line 15 in tests/test_autofix_validation.py

View workflow job for this annotation

GitHub Actions / Python CI / lint-ruff

Ruff (F401)

tests/test_autofix_validation.py:15:30: F401 `typing.Optional` imported but unused

Check failure on line 15 in tests/test_autofix_validation.py

View workflow job for this annotation

GitHub Actions / Python CI / lint-ruff

Ruff (F401)

tests/test_autofix_validation.py:15:25: F401 `typing.List` imported but unused

Check failure on line 15 in tests/test_autofix_validation.py

View workflow job for this annotation

GitHub Actions / Python CI / lint-ruff

Ruff (F401)

tests/test_autofix_validation.py:15:20: F401 `typing.Dict` imported but unused

Check failure on line 15 in tests/test_autofix_validation.py

View workflow job for this annotation

GitHub Actions / Python CI / lint-ruff

Ruff (UP035)

tests/test_autofix_validation.py:15:1: UP035 `typing.List` is deprecated, use `list` instead

Check failure on line 15 in tests/test_autofix_validation.py

View workflow job for this annotation

GitHub Actions / Python CI / lint-ruff

Ruff (UP035)

tests/test_autofix_validation.py:15:1: UP035 `typing.Dict` is deprecated, use `dict` instead

Copilot AI Dec 29, 2025

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Import of 'Dict' is not used.
Import of 'List' is not used.
Import of 'Optional' is not used.
Import of 'Any' is not used.

Suggested change
from typing import Dict,List,Optional,Any

Copilot uses AI. Check for mistakes.
from adapters.base import connect_db,get_adapter,tracked_call

Copilot AI Dec 29, 2025

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All async tests in the codebase use the @pytest.mark.asyncio decorator (see tests/test_adapter_base.py, tests/test_edgar.py, etc.). However, this file doesn't import pytest or use any pytest markers. If these tests are intended to be run as part of the test suite, they should follow the established conventions for test structure, including proper imports and decorators where applicable.

Copilot uses AI. Check for mistakes.

Copilot AI Dec 29, 2025

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Import of 'tracked_call' is not used.

Suggested change
from adapters.base import connect_db,get_adapter,tracked_call
from adapters.base import connect_db,get_adapter

Copilot uses AI. Check for mistakes.

# --- RUFF VIOLATIONS ---
# F401: unused import
import json

Copilot AI Dec 29, 2025

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Import of 'json' is not used.

Copilot uses AI. Check for mistakes.
import re

Copilot AI Dec 29, 2025

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Import of 're' is not used.

Suggested change
import re

Copilot uses AI. Check for mistakes.
import collections

Check failure on line 22 in tests/test_autofix_validation.py

View workflow job for this annotation

GitHub Actions / Python CI / lint-ruff

Ruff (I001)

tests/test_autofix_validation.py:14:1: I001 Import block is un-sorted or un-formatted

Check failure on line 22 in tests/test_autofix_validation.py

View workflow job for this annotation

GitHub Actions / Python CI / lint-ruff

Ruff (I001)

tests/test_autofix_validation.py:14:1: I001 Import block is un-sorted or un-formatted

Check failure on line 22 in tests/test_autofix_validation.py

View workflow job for this annotation

GitHub Actions / Python CI / lint-ruff

Ruff (I001)

tests/test_autofix_validation.py:14:1: I001 Import block is un-sorted or un-formatted

Check failure on line 22 in tests/test_autofix_validation.py

View workflow job for this annotation

GitHub Actions / Python CI / lint-ruff

Ruff (I001)

tests/test_autofix_validation.py:14:1: I001 Import block is un-sorted or un-formatted

Copilot AI Dec 29, 2025

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Import of 'collections' is not used.

Suggested change
import collections

Copilot uses AI. Check for mistakes.

# E501: line too long
VERY_LONG_STRING_THAT_VIOLATES_LINE_LENGTH = "This is a very long string that definitely exceeds the maximum line length limit of 88 characters that ruff and black enforce by default"

Copilot AI Dec 29, 2025

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The PR description lists E501 (line too long) as an intentional Ruff violation that should be caught, but the project's ruff configuration in pyproject.toml explicitly ignores E501 with ignore = ["E501"]. This intentional violation will not trigger a lint error and therefore will not validate the autofix system's handling of E501 errors.

Copilot uses AI. Check for mistakes.

# --- MYPY TYPE ERROR ---
def bad_type_annotation(x: int) -> str:
return x # Returns int, claims str
Comment on lines +26 to +27

Copilot AI Dec 29, 2025

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The PR description lists Mypy type errors as intentional issues that should trigger Codex dispatch, but the project's mypy configuration in pyproject.toml excludes the tests directory and also has overrides that ignore errors in adapters. Since this test file imports from adapters.base, mypy type checking may not catch these errors as expected, preventing validation of the autofix system's handling of type errors.

Copilot uses AI. Check for mistakes.

def missing_return_type(value):
"""Function missing type annotations."""
return value * 2


# --- BLACK VIOLATION: Inconsistent spacing ---
def poorly_formatted_function( arg1,arg2, arg3 ):
"""This function has poor formatting."""
result=arg1+arg2+arg3
if result>10:
return result
else:
return result*2


class BadlyFormattedClass:
"""Class with formatting issues."""

def __init__(self,name:str,value:int):
self.name=name
self.value=value

def compute(self,multiplier:int)->int:
return self.value*multiplier


# --- ACTUAL USEFUL COVERAGE TESTS ---

def test_connect_db_sqlite_default():
"""Test connect_db returns valid SQLite connection."""
# Clear env vars to force SQLite path
old_url = os.environ.pop("DB_URL", None)
old_path = os.environ.pop("DB_PATH", None)
Comment on lines +62 to +63

Copilot AI Dec 29, 2025

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The test manipulates environment variables directly with os.environ.pop(), but other tests in the codebase use pytest's monkeypatch fixture for environment variable management (see tests/test_adapter_base.py:16, tests/test_embeddings.py:31). Using monkeypatch ensures automatic cleanup and prevents test pollution. Consider accepting monkeypatch as a parameter and using monkeypatch.delenv() or monkeypatch.setenv() instead.

Copilot uses AI. Check for mistakes.

try:
conn = connect_db(":memory:")
assert conn is not None
# Verify it's a working connection
cursor = conn.cursor()
cursor.execute("SELECT 1")
result = cursor.fetchone()
assert result == (1,)
conn.close()
finally:
if old_url:
os.environ["DB_URL"] = old_url
if old_path:
os.environ["DB_PATH"] = old_path


def test_connect_db_with_timeout():
"""Test connect_db accepts timeout parameter."""
conn = connect_db(":memory:", connect_timeout=5.0)
assert conn is not None
conn.close()


def test_get_adapter_edgar():
"""Test that edgar adapter can be loaded."""
try:
adapter = get_adapter("edgar")
assert adapter is not None
# Verify it has expected protocol methods
assert hasattr(adapter, "list_new_filings") or callable(getattr(adapter, "list_new_filings", None)) is False

Copilot AI Dec 29, 2025

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The logic in this assertion is incorrect. The expression callable(getattr(adapter, "list_new_filings", None)) is False will only be True if the attribute exists and is explicitly the False object (not just falsy). This should use not callable(...) instead, or simply check hasattr(adapter, "list_new_filings"). The current logic makes this assertion always pass when the method exists and is callable, which defeats the purpose of the test.

Suggested change
assert hasattr(adapter, "list_new_filings") or callable(getattr(adapter, "list_new_filings", None)) is False
assert hasattr(adapter, "list_new_filings") and callable(getattr(adapter, "list_new_filings", None))

Copilot uses AI. Check for mistakes.

Copilot AI Dec 29, 2025

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Based on the existing test in tests/test_adapter_registry.py (line 10-13), adapters should be tested as modules with the pattern isinstance(adapter, types.ModuleType) and direct attribute checks using hasattr(). The current test uses callable(getattr(...)) which is inconsistent with the established testing pattern for adapters in this codebase.

Copilot uses AI. Check for mistakes.
except ModuleNotFoundError:
# Adapter module may not exist yet
pass


def test_get_adapter_invalid():
"""Test get_adapter raises for unknown adapter."""
try:
get_adapter("nonexistent_adapter_xyz")
assert False, "Should have raised"
except (ModuleNotFoundError, ImportError):
pass # Expected


# --- INTENTIONALLY FAILING TESTS ---

def test_intentional_failure_assertion():
"""This test intentionally fails with an assertion error."""
expected = 42
actual = 41
assert actual == expected, f"Expected {expected} but got {actual}"
Comment on lines +115 to +119

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Remove unconditional failing tests blocking suite

The three “intentional failure” tests here all fail unconditionally (41 == 42, followed by a KeyError and a TypeError in the two tests immediately below), and they are not marked xfail/skip. As written, any pytest run will halt on this block, keeping CI permanently red and preventing the rest of the suite from running; these should be guarded or deleted if the goal is a stable test run.

Useful? React with 👍 / 👎.

Comment on lines +115 to +119

Copilot AI Dec 29, 2025

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This test intentionally fails by asserting 41 == 42. While this is documented as an intentional failure for validation purposes, when this test runs it will cause the test suite to fail. Consider whether this test should be marked with a custom pytest marker (like @pytest.mark.skip(reason="Intentional failure for autofix validation")) until the autofix system is validated, to avoid breaking the main test suite.

Copilot uses AI. Check for mistakes.


def test_intentional_failure_exception():
"""This test intentionally raises an exception."""
data = {"key": "value"}
# This will raise KeyError
result = data["nonexistent_key"]
Comment on lines +122 to +126

Copilot AI Dec 29, 2025

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This test intentionally raises a KeyError. While this is documented as an intentional failure, when executed it will cause the test suite to fail. The same concern applies as with the other intentionally failing tests - consider using pytest markers to skip these until the autofix system has been validated, to prevent disrupting the normal development workflow.

Copilot uses AI. Check for mistakes.

Copilot AI Dec 29, 2025

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Variable result is not used.

Suggested change
result = data["nonexistent_key"]
data["nonexistent_key"]

Copilot uses AI. Check for mistakes.


def test_intentional_failure_type_error():
"""This test intentionally causes a TypeError."""
value = "not a number"
# This will raise TypeError
result = value + 5
Comment on lines +129 to +133

Copilot AI Dec 29, 2025

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This test intentionally raises a TypeError. Like the other intentionally failing tests, this will cause test suite failures when executed. Consider using pytest skip markers for these validation tests to avoid disrupting normal CI/CD workflows until the autofix system has been properly validated.

Copilot uses AI. Check for mistakes.

Copilot AI Dec 29, 2025

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Variable result is not used.

Suggested change
result = value + 5
value + 5

Copilot uses AI. Check for mistakes.


# --- RUFF VIOLATIONS: More lint issues ---

# W293: whitespace on blank line
def function_with_trailing_whitespace():
"""Has trailing whitespace."""
x = 1

y = 2
return x + y


# E711: comparison to None
def bad_none_comparison(value):
if value == None:

Copilot AI Dec 29, 2025

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Testing for None should use the 'is' operator.

Suggested change
if value == None:
if value is None:

Copilot uses AI. Check for mistakes.
return "empty"
return "full"


# E712: comparison to True
def bad_bool_comparison(flag):
if flag == True:
return "yes"
return "no"
Loading