Introducing --track-filter option

[NickDris]

Introducing a --track-filter argument specified by a comma separated list of track names.
This defines a fixture meant for specific track inclusion, excluding any tracks you don't need to test.
Limiting the test coverage to specific tracks can dramatically decrease the test times of your track repository.

[Copilot]

Pull Request Overview

This PR introduces a new --track-filter command-line option to allow filtering tests by specific track names, enabling users to run tests only for selected tracks to reduce test execution time.

• Adds --track-filter CLI option that accepts comma-separated track names
• Implements filtering logic in test generation to include only specified tracks
• Creates a session-scoped fixture to access the track filter configuration

Reviewed Changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.

File	Description
pytest_rally/plugin.py	Adds CLI option definition, parsing logic, and filtering implementation in test generation
pytest_rally/fixtures.py	Adds session-scoped fixture to expose track filter configuration to tests

---

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

[gbanasiak]

This looks good. I've refreshed tests in 1bed71d. Can you add a unit test that would capture --track-filter functionality?

[NickDris]

This looks good. I've refreshed tests in 1bed71d. Can you add a unit test that would capture --track-filter functionality?

Done in latest commits

[Copilot]

Pull Request Overview

Copilot reviewed 13 out of 14 changed files in this pull request and generated 3 comments.

---

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

[gbanasiak]

I spent some time experimenting with this. I think we can avoid repetition of the code in conftest.py. It would also be nice to add test case for --skip-autogenerated-tests. Other comments are of lesser importance. Thank you for working on this, almost there :)

[Copilot]

Pull Request Overview

Copilot reviewed 12 out of 13 changed files in this pull request and generated 2 comments.

---

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

[gbanasiak]

LGTM, thanks for iterating

[NickDris]

Merged without the requested transformation for the following review comment:

in current form we verify only expected tracks run, but not if expected unit test methods based on the content of marked_tracks.py

and that's because some unit test methods runs multiple tracks and we cannot easily auto-discover their names. In the current form the check is asserting that pytest runs only the tracks with the same name as the markers in the track-filter.

To solve that I could be making raced_tracks an ordered list and produce the expected raced_tracks by handpicking tracks from the marked_tracks file in order, but that would make the function too large without an autodiscovery option, because the expected track list for each track-filter is different.

If you believe that this would be a better option, let's make another PR for it.

[gbanasiak]

I think marked_tracks.py could simply be structured in such way that filtering for a given track produces a unique function name, and then the test could both check whether only the filtered track was run, and whether only the expected item does not have a skip marker added (in collection phase). We can tackle this some other time, if needed.