Replace cassandra hack with fixture to fail badly named test files

[potiuk]

The cassandra hack added in #30315 does not seem to have a chance to get away. Neither Pytest pytest-dev/pytest#10844 nor Datastax apache/cassandra-python-driver#1142 want to own the problem for now (though there is a proposal from pytest contributors on how Datastax could refactor their code to avoid the problem)

However during the discussion an idea popped in my head on how we could come back to test_* pattern with far less probability of missing some tests that are added to wrong files. Seems that we can add a fixture that will outright fail tests if they are placed if files not following the test_* pattern. While it would not help in case test would be wrongly named in the first place, it would definitely help to not to add more tests in wrongly named files because it will be literally impossible to run the tests added in a wrong file, even if you manualy do pytest somefile.py and avoid running collection.

I also did a quick check to try to find cases where the test_* file name was already violated and I found (and renamed) two that I have found. It seems it is quite likely that similar mistake could be done in the future - but with the fixture I added it should be far less likely someone adds tests in a wrongly named file.

---

^ Add meaningful description above

Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in newsfragments.

[potiuk]

@Taragolis @o-nikolas @hussein-awala -> follow up after #30315 -> seems that neither Cassandra nor Pytest do not want to own the problem, however during the discussion I have found a creative way how we can avoid the problem of accidentally adding tests (that would be skipped during collection) in badly named files - I added a custom fixture to fail such tests if they are run "manually".

And of course while doing it, I found that we have such tests already, so I was not wrong in assesment that just adding test_*.py is a bad idea without having some kind of protection against such tests being added. BTW. I am running some extra checks and maybe I will find more such cases (my check was quite rudimentary)

[potiuk]

I run all the tests with *.py and with the fixture (and having cassandra hack in) and I have not found any more tests in wrong filenames. So we should be good - those were. the only two wrongly named files that contained tests.

[hussein-awala]

I tested it locally by adding some new test files, and it seems to work. LGTM

[vincbeck]

All our system tests are failing because of this: https://aws-mwaa.github.io/open-source/system-tests/dashboard.html

ERROR tests/system/providers/amazon/aws/example_cloudformation.py::test_run - Exception: All test method files in tests/ must start with 'test_'. Seems that example_cloudformation.py contains <function get_test_run.<locals>.test_run at 0x7f8dbf4efdc0> that looks like a test case. Please rename the file to follow the test_* pattern if you want to run the tests in it.

Should we exclude system tests from this check? As far as I know all system tests are named example_......

[potiuk]

Yeah we should add exclusion in https://github.com/apache/airflow/pull/32626/files#diff-e52e4ddd58b7ef887ab03c04116e676f6280b824ab7469d5d3080e5cba4f2128R911 the check should allow "example_" in case there are in "tests/system" folder.

I am not at my PC now, but maybe you can add a fix ?

[vincbeck]

Yep, I can do that

[vincbeck]

Fix: #32655