settings.py not covered when plugin enabled on Django 1.10

[twschiller]

When I enable django_coverage_plugin for Django 1.10, the settings.py file for my project is not covered. However, I know settings.py is being run because the tests are still passing.

Coverage metrics:

• Without plugin: https://coveralls.io/builds/7668992/source?filename=openintel%2Fsettings.py
• With plugin: https://coveralls.io/builds/7668159/source?filename=openintel%2Fsettings.py

Here's a diff showing how I enabled the plugin: https://github.com/twschiller/open-synthesis/compare/template-coverage-dev

I'm running coverage as: coverage run --source='.' manage.py test

[nedbat]

I guess this is because the plugin imports the settings before coverage has really started. Thanks for the report, though I'm not sure I have many options to fix it...?

[twschiller]

Glancing at the coverage docs, I'm thinking this might be a bug in coverage instead of django_coverage_plugin since django_coverage_plugin is only recording metrics for templates. Your file_tracer method seems reasonable, i.e., that it wouldn't be accidentally picking up settings.py.

Overall, this isn't a major issue for me, as the coverage for the other files will monitor results/impact of changing settings.

[nedbat]

My first thought was, "this isn't about the plugin, because it's a .py file, and the plugin is about .html (or whatever) files." But you've shown that the plugin is the determining factor, and the reason it is is that it imports settings.py before coverage measurement begins. Coverage.py can't measure a file that is imported before it begins measurement.

[twschiller]

I'd have to look at trace of which files/modules are being loaded and when. I would have thought that the Django app would have loaded the settings module before an executed file would pass the test in file_tracer(...). Maybe it's a matter of delaying the TEMPLATE_DEBUG check even further. Not sure if it's worth the time to dig into.

[PamelaM]

@twschiller : Please try plugin version 1.4 which now supports Django 1.10

[PamelaM]

@twschiller I hope that the latest version (1.4.1) addresses your issue. If there are no problems reported by Feb 1st, we'll consider this issue closed.

[twschiller]

Looks like it's being included now. Thanks!