Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Excluded Mutation Operators Still Getting Run #522

Closed
FloatingSunfish opened this issue Aug 5, 2021 · 8 comments
Closed

Excluded Mutation Operators Still Getting Run #522

FloatingSunfish opened this issue Aug 5, 2021 · 8 comments

Comments

@FloatingSunfish
Copy link

FloatingSunfish commented Aug 5, 2021

I have excluded the following mutation operators in my Cosmic Ray configuration file:

[cosmic-ray.filters.operators-filter]
exclude-operators = [
    "core/ReplaceComparisonOperator_Eq_Is",
    "core/ReplaceComparisonOperator_NotEq_IsNot"
]

I also reran init before running mutation tests.

However, when I ran an HTML report, the mutation operators were still getting run:

1
2

NOTE: I tried a second time, this time running cr-filter-operators mutation_test_session.sqlite (which I assume isn't needed because the configuration file already has all the filters) after running init and it didn't solve the problem.

@abingham
Copy link
Contributor

abingham commented Aug 5, 2021

This features doesn't work right now. See #517.

@abingham abingham closed this as completed Aug 5, 2021
@FloatingSunfish
Copy link
Author

FloatingSunfish commented Aug 5, 2021

@abingham
I see. From what I understood from looking at #517, it was only excluding modules that didn't work, not excluding specific mutation operators.

Out of curiosity, how closely related are these two issues? Because if resolving one won't automatically resolve the other, perhaps this needs to be its own issue.

Thanks!

@abingham
Copy link
Contributor

abingham commented Aug 5, 2021

Ah, you're right, sorry for not reading closely enough. I'll look into this.

@abingham abingham reopened this Aug 5, 2021
@abingham
Copy link
Contributor

abingham commented Aug 5, 2021

The problem has to do with a mistake in how the filter operator is implemented. The --config option should actually be required, so you can fix your immediate issue by doing cr-filters-operators --config <config file> mutation_test_session.sqlite. I'll update the filter to require a config; it's currently optional because of a legacy way sessions used to work.

which I assume isn't needed because the configuration file already has all the filters

It is required. Cosmic Ray doesn't run any filters on its own, and requires users to run them externally. We used to try to manage all of the filtering internally, and - while it's definitely technically possible - it was more trouble than it was worth.

@FloatingSunfish
Copy link
Author

@abingham
>We used to try to manage all of the filtering internally
By this, do you mean you tried just using the config file and no longer requiring users to run filter commands manually?

Also, thanks for the workaround while we wait for this issue to get resolved. 🙂

@abingham
Copy link
Contributor

abingham commented Aug 5, 2021

By this, do you mean you tried just using the config file and no longer requiring users to run filter commands manually?

Correct. CR used to run certain kinds of filtering as part of the init phase, and it would read the filtering criteria from the config in the same way that e.g. cr-filter-operators does. But it added complexity to the implementation without adding sufficient benefits, i.e. it's very simple to run filters, add them to initialization scripts, or whatever.

@abingham
Copy link
Contributor

abingham commented Aug 7, 2021

This should be fixed in 8.3.3.

@abingham abingham closed this as completed Aug 7, 2021
@FloatingSunfish
Copy link
Author

@abingham
Perhaps it would be a good idea to add a new section in the cr-filter-operators documentation like:

cr-filter-operators session.sqlite config.toml

Just to make it more uniform with cr-filter-pragma having an example.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants