[msm] use Ipython's ShimModule approach to warn upon access of a moved module. #805
Conversation
|
Good cal. What does "Shim" mean? Is that a standard term? Am 17/05/16 um 18:42 schrieb Martin K. Scherer:
Prof. Dr. Frank Noe Phone: (+49) (0)30 838 75354 Mail: Arnimallee 6, 14195 Berlin, Germany |
|
what about a DeprecationWarning? Am 17/05/16 um 18:42 schrieb Martin K. Scherer:
Prof. Dr. Frank Noe Phone: (+49) (0)30 838 75354 Mail: Arnimallee 6, 14195 Berlin, Germany |
|
Thats the term of Ipython folks. I haven't encountered it before.
I would also like to use a plain DeprecationWarning, but some library or even the Anaconda folks have decided to ignore DepracationWarnings by default (they are
setting a filter to ignore these types of warnings somewhere, which is really hard to figure out). Once we had the same problem with mdtraj. Somebody forgot to
remove this debug statement and so globally for every module in the interpreter this was turned of. It is a real mess.
|
|
marscher@jigsaw:~/anaconda$ python --version
Python 2.7.11 :: Anaconda 2.1.0 (64-bit)
marscher@jigsaw:~/anaconda$ python -c "import warnings; print(warnings.filters)"
[('ignore', None, <type 'exceptions.DeprecationWarning'>, None, 0), ('ignore', None, <type 'exceptions.PendingDeprecationWarning'>, None, 0), ('ignore', None,
<type 'exceptions.ImportWarning'>, None, 0), ('ignore', None, <type 'exceptions.BytesWarning'>, None, 0)]
It really seems like the Anaconda people decided to hide these warnings by default, which is really bad.
|
|
ShimModules were used during the transition from Ipython3 to Jupyter, where some packages got split up and so the developers decided to warn upon the usage of the old path. So I'd guess at least some users could have seen "ShimWarnings" along their path. |
|
https://en.wikipedia.org/wiki/Shim_%28spacer%29 Shims are used to "fill in the gaps". The term is stolen from construction. Thank you guys so much for discussing everything in English! :) Much appreciated |
|
Ah nice to know. Thanks :)
|
|
Thanks Matthew! We're pretty international in the group, so it wouldn't How weird, so we use IPython's deprecation warning also outside IPython What about providing a shortcut e.g. Am 17/05/16 um 20:09 schrieb Matthew Harrigan:
Prof. Dr. Frank Noe Phone: (+49) (0)30 838 75354 Mail: Arnimallee 6, 14195 Berlin, Germany |
|
Actually as I have demonstrated, the fault lies not on IPythons side, but has something to do with Anacondas default settings. Wrapping around this issue in the meantime seems like a good solution. |
|
It is not Anacondas fault, but mine, since I didn't read the Python documentation: Developer related warnings (eg. DeprecationWarnings) are filtered by default. So we should really think of introducing our own type, since we want the users to be informed if they are using features, which will be removed soon. |
|
What about changing the filter to show DeprecationWarnings? Am 17/05/16 um 20:55 schrieb Martin K. Scherer:
Prof. Dr. Frank Noe Phone: (+49) (0)30 838 75354 Mail: Arnimallee 6, 14195 Berlin, Germany |
|
Am 17.05.2016 um 21:00 schrieb Frank Noe:
Since the warnings filter is global, it applies for every imported package. Especially jupyter is very verbose. So I guess introducing our own type will not hurt anybody. The code needs to changed in two places. |
|
You're right. Am 17/05/16 um 21:08 schrieb Martin K. Scherer:
Prof. Dr. Frank Noe Phone: (+49) (0)30 838 75354 Mail: Arnimallee 6, 14195 Berlin, Germany |
|
It is very unclear to me why those warnings are note triggered on Travis for Python2. The tests pass for Py2 on my machine... |
|
It seems like Nose is catching them cleverly. I really want to transition to pytest soon... |
|
Maybe the nose environment changes the filters? Am 18/05/16 um 00:34 schrieb Martin K. Scherer:
Prof. Dr. Frank Noe Phone: (+49) (0)30 838 75354 Mail: Arnimallee 6, 14195 Berlin, Germany |
|
I've done everything I could to have a clean filters environment. It looks like how the tests are invoked.. If the whole suite is run, then error occurs, but if run alone it works. |
|
Hm, no idea. |
|
Yes. This is doing all the magic. I've just interchanged the type of warning |
|
I see... nosetest is changing the default import behaviour. So this can not work. The warnings show up as expected in Travis during the first (nose import everything)... This is actually not the first time I encountered problems with nose - I'm really in the mood making the transition to pytest now. |
|
now as in now? Am 18/05/16 um 01:04 schrieb Martin K. Scherer:
Prof. Dr. Frank Noe Phone: (+49) (0)30 838 75354 Mail: Arnimallee 6, 14195 Berlin, Germany |
|
It is impossible to change this behaviour according to the documentation. So yes, I want to do it now. |
|
Sounds like a big change. I guess all tests need to be changed, or is Am 18/05/16 um 01:11 schrieb Martin K. Scherer:
Prof. Dr. Frank Noe Phone: (+49) (0)30 838 75354 Mail: Arnimallee 6, 14195 Berlin, Germany |
|
We're using the standard unit testing framework from Python. This framework provides the Unittest classes to set up test cases. This way we are pretty much independent of the running framework. The running framework simply collects the all the test files (and doctests) and runs them. For instance nose ignores the tests in pyemma._base package because it starts with an underscore. We even had undiscovered bugs because of that. The change is actually not that massive and it only requires the adoption of devtools/conda-recipe/run_test.py |
|
Ah I see! I though you wanted to change the way tests are written, but Am 18/05/16 um 01:17 schrieb Martin K. Scherer:
Prof. Dr. Frank Noe Phone: (+49) (0)30 838 75354 Mail: Arnimallee 6, 14195 Berlin, Germany |
|
pytest even has a (working) plugin to distribute tests to multiple subprocesses, which the nosetests plugin couldn't achieve every time I tested it. I will implement the change and then see how it will be working. |
|
ok, thanks Am 18/05/16 um 01:20 schrieb Martin K. Scherer:
Prof. Dr. Frank Noe Phone: (+49) (0)30 838 75354 Mail: Arnimallee 6, 14195 Berlin, Germany |
|
As it turns out pytest is even more buggy than nose 👍 |
…d module. This adresses markovmodel#550. Remove those packages again in version 2.3
marscher
force-pushed
the
deprecate_msm_lowlevel_package
branch
from
a96ce1d
to
005532e
Compare
This adresses #550. Remove those packages again in version 2.3
The user will get a "ShimWarning", when he/she tries to access an attribute from a moved module.
All msmtools sub-packages like analysis and estimation are now ShimModules, which redirects the functionality to the actual implementation, but print a warning upon doing so.
This way we can safely remove pyemma.msm.msmtools stuff in the next minor version.