-
Notifications
You must be signed in to change notification settings - Fork 0
test: autofix system validation with intentional failures #86
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from 3 commits
0e2f7f7
2e405e4
afcaad0
ff74818
06efdcd
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change | ||||||||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| @@ -0,0 +1,162 @@ | ||||||||||||||||||||||||||||||||||||||
| """Test file with intentional issues to validate autofix system. | ||||||||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||||||||
| This file contains: | ||||||||||||||||||||||||||||||||||||||
| 1. Black formatting violations | ||||||||||||||||||||||||||||||||||||||
| 2. Ruff lint errors | ||||||||||||||||||||||||||||||||||||||
| 3. Mypy type errors | ||||||||||||||||||||||||||||||||||||||
| 4. Failing tests | ||||||||||||||||||||||||||||||||||||||
| 5. Actual useful test coverage | ||||||||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||||||||
| Purpose: Validate the full autofix pipeline handles all failure modes. | ||||||||||||||||||||||||||||||||||||||
| """ | ||||||||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||||||||
| # --- BLACK VIOLATION: Bad formatting --- | ||||||||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||||||||
| # --- RUFF VIOLATIONS --- | ||||||||||||||||||||||||||||||||||||||
| # F401: unused import | ||||||||||||||||||||||||||||||||||||||
| import os | ||||||||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||||||||
| from adapters.base import connect_db, get_adapter | ||||||||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||||||||
| # E501: line too long | ||||||||||||||||||||||||||||||||||||||
| VERY_LONG_STRING_THAT_VIOLATES_LINE_LENGTH = "This is a very long string that definitely exceeds the maximum line length limit of 88 characters that ruff and black enforce by default" | ||||||||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||||||||
| # --- MYPY TYPE ERROR --- | ||||||||||||||||||||||||||||||||||||||
| def bad_type_annotation(x: int) -> str: | ||||||||||||||||||||||||||||||||||||||
| return x # Returns int, claims str | ||||||||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||||||||||||
| # Intentional mypy error: function without parameter or return type annotations |
Copilot
AI
Dec 29, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The logic in this assertion is flawed. The condition hasattr(adapter, "list_new_filings") or callable(getattr(adapter, "list_new_filings", None)) is False will always evaluate to True if hasattr returns True, making the second part unreachable. The intended check appears to be verifying that the attribute exists AND is callable. Consider simplifying to assert hasattr(adapter, "list_new_filings") or checking assert callable(getattr(adapter, "list_new_filings", None)).
| assert ( | |
| hasattr(adapter, "list_new_filings") | |
| or callable(getattr(adapter, "list_new_filings", None)) is False | |
| assert hasattr(adapter, "list_new_filings") and callable( | |
| getattr(adapter, "list_new_filings", None) |
Check failure on line 107 in tests/test_autofix_validation.py
GitHub Actions / Python CI / lint-ruff
Ruff (B011)
tests/test_autofix_validation.py:107:16: B011 Do not `assert False` (`python -O` removes these calls), raise `AssertionError()`
Check failure on line 107 in tests/test_autofix_validation.py
GitHub Actions / Python CI / lint-ruff
Ruff (B011)
tests/test_autofix_validation.py:107:16: B011 Do not `assert False` (`python -O` removes these calls), raise `AssertionError()`
Check failure on line 107 in tests/test_autofix_validation.py
GitHub Actions / Python CI / lint-ruff
Ruff (B011)
tests/test_autofix_validation.py:107:16: B011 Do not `assert False` (`python -O` removes these calls), raise `AssertionError()`
Check failure on line 107 in tests/test_autofix_validation.py
GitHub Actions / Python CI / lint-ruff
Ruff (B011)
tests/test_autofix_validation.py:107:16: B011 Do not `assert False` (`python -O` removes these calls), raise `AssertionError()`
Check failure on line 107 in tests/test_autofix_validation.py
GitHub Actions / Python CI / lint-ruff
Ruff (B011)
tests/test_autofix_validation.py:107:16: B011 Do not `assert False` (`python -O` removes these calls), raise `AssertionError()`
Check failure on line 107 in tests/test_autofix_validation.py
GitHub Actions / Python CI / lint-ruff
Ruff (B011)
tests/test_autofix_validation.py:107:16: B011 Do not `assert False` (`python -O` removes these calls), raise `AssertionError()`
Check failure on line 107 in tests/test_autofix_validation.py
GitHub Actions / Python CI / lint-ruff
Ruff (B011)
tests/test_autofix_validation.py:107:16: B011 Do not `assert False` (`python -O` removes these calls), raise `AssertionError()`
Copilot
AI
Dec 29, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This test expects an exception to be raised, but if get_adapter doesn't raise an exception, the assertion on line 107 will fail with a confusing error message. The pattern used here doesn't actually verify that an exception was raised - it only passes if an exception occurs. Consider using pytest's raises() context manager for clearer intent: with pytest.raises((ModuleNotFoundError, ImportError)): get_adapter("nonexistent_adapter_xyz"). This would require importing pytest at the top of the file.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Check failure on line 126 in tests/test_autofix_validation.py
GitHub Actions / Python CI / lint-ruff
Ruff (F841)
tests/test_autofix_validation.py:126:5: F841 Local variable `result` is assigned to but never used
Check failure on line 126 in tests/test_autofix_validation.py
GitHub Actions / Python CI / lint-ruff
Ruff (F841)
tests/test_autofix_validation.py:126:5: F841 Local variable `result` is assigned to but never used
Check failure on line 126 in tests/test_autofix_validation.py
GitHub Actions / Python CI / lint-ruff
Ruff (F841)
tests/test_autofix_validation.py:126:5: F841 Local variable `result` is assigned to but never used
Check failure on line 126 in tests/test_autofix_validation.py
GitHub Actions / Python CI / lint-ruff
Ruff (F841)
tests/test_autofix_validation.py:126:5: F841 Local variable `result` is assigned to but never used
Check failure on line 126 in tests/test_autofix_validation.py
GitHub Actions / Python CI / lint-ruff
Ruff (F841)
tests/test_autofix_validation.py:126:5: F841 Local variable `result` is assigned to but never used
Check failure on line 126 in tests/test_autofix_validation.py
GitHub Actions / Python CI / lint-ruff
Ruff (F841)
tests/test_autofix_validation.py:126:5: F841 Local variable `result` is assigned to but never used
Check failure on line 126 in tests/test_autofix_validation.py
GitHub Actions / Python CI / lint-ruff
Ruff (F841)
tests/test_autofix_validation.py:126:5: F841 Local variable `result` is assigned to but never used
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Check failure on line 133 in tests/test_autofix_validation.py
GitHub Actions / Python CI / lint-ruff
Ruff (F841)
tests/test_autofix_validation.py:133:5: F841 Local variable `result` is assigned to but never used
Check failure on line 133 in tests/test_autofix_validation.py
GitHub Actions / Python CI / lint-ruff
Ruff (F841)
tests/test_autofix_validation.py:133:5: F841 Local variable `result` is assigned to but never used
Check failure on line 133 in tests/test_autofix_validation.py
GitHub Actions / Python CI / lint-ruff
Ruff (F841)
tests/test_autofix_validation.py:133:5: F841 Local variable `result` is assigned to but never used
Check failure on line 133 in tests/test_autofix_validation.py
GitHub Actions / Python CI / lint-ruff
Ruff (F841)
tests/test_autofix_validation.py:133:5: F841 Local variable `result` is assigned to but never used
Check failure on line 133 in tests/test_autofix_validation.py
GitHub Actions / Python CI / lint-ruff
Ruff (F841)
tests/test_autofix_validation.py:133:5: F841 Local variable `result` is assigned to but never used
Check failure on line 133 in tests/test_autofix_validation.py
GitHub Actions / Python CI / lint-ruff
Ruff (F841)
tests/test_autofix_validation.py:133:5: F841 Local variable `result` is assigned to but never used
Check failure on line 133 in tests/test_autofix_validation.py
GitHub Actions / Python CI / lint-ruff
Ruff (F841)
tests/test_autofix_validation.py:133:5: F841 Local variable `result` is assigned to but never used
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Copilot
AI
Dec 29, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The variable result is assigned but never used. While the test is intentionally designed to fail with a KeyError, the unused variable assignment will also trigger a linting warning (F841). Consider either removing the assignment or using it in an assertion if you want to test a different failure mode.
| result = data["nonexistent_key"] | |
| def test_intentional_failure_type_error(): | |
| """This test intentionally causes a TypeError.""" | |
| value = "not a number" | |
| # This will raise TypeError | |
| result = value + 5 | |
| _ = data["nonexistent_key"] | |
| def test_intentional_failure_type_error(): | |
| """This test intentionally causes a TypeError.""" | |
| value = "not a number" | |
| # This will raise TypeError | |
| _ = value + 5 |
Copilot
AI
Dec 29, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The variable result is assigned but never used. While the test is intentionally designed to fail with a TypeError, the unused variable assignment will also trigger a linting warning (F841). Consider either removing the assignment or using it in an assertion if you want to test a different failure mode.
| result = value + 5 | |
| value + 5 |
Check failure on line 150 in tests/test_autofix_validation.py
GitHub Actions / Python CI / lint-ruff
Ruff (E711)
tests/test_autofix_validation.py:150:17: E711 Comparison to `None` should be `cond is None`
Check failure on line 150 in tests/test_autofix_validation.py
GitHub Actions / Python CI / lint-ruff
Ruff (E711)
tests/test_autofix_validation.py:150:17: E711 Comparison to `None` should be `cond is None`
Check failure on line 150 in tests/test_autofix_validation.py
GitHub Actions / Python CI / lint-ruff
Ruff (E711)
tests/test_autofix_validation.py:150:17: E711 Comparison to `None` should be `cond is None`
Check failure on line 150 in tests/test_autofix_validation.py
GitHub Actions / Python CI / lint-ruff
Ruff (E711)
tests/test_autofix_validation.py:150:17: E711 Comparison to `None` should be `cond is None`
Check failure on line 150 in tests/test_autofix_validation.py
GitHub Actions / Python CI / lint-ruff
Ruff (E711)
tests/test_autofix_validation.py:150:17: E711 Comparison to `None` should be `cond is None`
Check failure on line 150 in tests/test_autofix_validation.py
GitHub Actions / Python CI / lint-ruff
Ruff (E711)
tests/test_autofix_validation.py:150:17: E711 Comparison to `None` should be `cond is None`
Check failure on line 150 in tests/test_autofix_validation.py
GitHub Actions / Python CI / lint-ruff
Ruff (E711)
tests/test_autofix_validation.py:150:17: E711 Comparison to `None` should be `cond is None`
Copilot
AI
Dec 29, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Testing for None should use the 'is' operator.
| if value == None: | |
| if value is None: |
Check failure on line 157 in tests/test_autofix_validation.py
GitHub Actions / Python CI / lint-ruff
Ruff (E712)
tests/test_autofix_validation.py:157:8: E712 Avoid equality comparisons to `True`; use `flag:` for truth checks
Check failure on line 157 in tests/test_autofix_validation.py
GitHub Actions / Python CI / lint-ruff
Ruff (E712)
tests/test_autofix_validation.py:157:8: E712 Avoid equality comparisons to `True`; use `flag:` for truth checks
Check failure on line 157 in tests/test_autofix_validation.py
GitHub Actions / Python CI / lint-ruff
Ruff (E712)
tests/test_autofix_validation.py:157:8: E712 Avoid equality comparisons to `True`; use `flag:` for truth checks
Check failure on line 157 in tests/test_autofix_validation.py
GitHub Actions / Python CI / lint-ruff
Ruff (E712)
tests/test_autofix_validation.py:157:8: E712 Avoid equality comparisons to `True`; use `flag:` for truth checks
Check failure on line 157 in tests/test_autofix_validation.py
GitHub Actions / Python CI / lint-ruff
Ruff (E712)
tests/test_autofix_validation.py:157:8: E712 Avoid equality comparisons to `True`; use `flag:` for truth checks
Check failure on line 157 in tests/test_autofix_validation.py
GitHub Actions / Python CI / lint-ruff
Ruff (E712)
tests/test_autofix_validation.py:157:8: E712 Avoid equality comparisons to `True`; use `flag:` for truth checks
Check failure on line 157 in tests/test_autofix_validation.py
GitHub Actions / Python CI / lint-ruff
Ruff (E712)
tests/test_autofix_validation.py:157:8: E712 Avoid equality comparisons to `True`; use `flag:` for truth checks
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The import
osis marked as unused (F401) in the comment on line 16, but it is actually used in the test functions below (lines 62-63, 76-78 foros.environ.pop()andos.environ[]operations). This import should not be in the "unused import" section since it's legitimately needed.