Add travis support for basic start of spyder #2281

goanpeca · 2015-03-27T21:29:06Z

Related to #2268

Todo:

Use bootstrap options
Change user help message
Include optional dependencies

goanpeca · 2015-03-27T21:29:47Z

Just a basic start, so far I got these versions of python running

Build #85.... yes I tried many times!

@blink1073, @ccordoba12

blink1073 · 2015-03-27T21:36:13Z

This is awesome @goanpeca!

@ccordoba12, since you are the owner of the repo you'll have to be the one to enable it on Travis.

log into travis-ci.org
sign in with Github
click on your avatar on the top right and select Accounts
slide the spyder repo on

If you don't see the spyder repos, you'll have to go to the spyder-ide org on github and set yourself to "public" on the owner team.

goanpeca · 2015-03-27T21:58:05Z

I think this PR is enough on its own, for the actual tests I prefer to open a different PR, what do you guys think?

ccordoba12 · 2015-03-27T23:12:59Z

continuous_integration/travis_install.sh

+    #	
+
+  elif [ "$USE_QT_API" = "PyQt4" ]; then
+    conda create -q -n test-environment python=$PY_VERSION sphinx sip qt pyqt;


We should include here more packages. At least ipython-qtconsole, matplotlib and pandas.

Sure, any other packages besides the ones you mention?

Update

I would add the versions we are supposed to support, ? to make the test better? it would not add to much to the pipeline (for conda only!), that way we can test any particular versions that might be giving us trouble right now.

No, not the minimal versions, those are usually too old. We need to test things against their latest versions.

I'd say we should test against all packages mentioned in Optional dependencies

ccordoba12 · 2015-03-29T19:00:41Z

Thanks a lot for working on this @goanpeca, besides the couple of comments I made, I think this is ready for merging :-)

As you said, we can leave moving our tests to a tests directory for another PR.

goanpeca · 2015-03-30T22:06:35Z

Ok so I made the changes suggested,

@ccordoba12, can you go to https://travis-ci.org/spyder-ide/spyder/settings and activate testing only if a .yml is present?

goanpeca · 2015-03-30T22:09:19Z

@blink1073 , can you take a look at the build in my test repo?

https://travis-ci.org/goanpeca/spyder/builds/56333253

I am not sure what might be the problem on the build 92.6, thanks :-)

blink1073 · 2015-03-31T01:05:11Z

@goanpeca, to get PySide to work you use backend: qt4agg and backend.qt4 = pyside. See https://github.com/scikit-image/scikit-image/blob/master/tools/travis_script.sh#L107.

goanpeca · 2015-03-31T07:12:07Z

@blink1073 thanks, but why this needs to be done? doesnt spyder take care of this normally?

blink1073 · 2015-03-31T13:08:08Z

That's a good question. Somewhere the matplotlibrc is being set up on our behalf. You could try a python -c "import matplotlib" prior to the bootstrap to narrow it down to Spyder itself.

ccordoba12 · 2015-03-31T14:37:14Z

@goanpeca, I switched on the .yml option.

About mpl and PySide: we don't set the mpl backend globally, just for our Python consoles through our sitecustomize.py. I don't know if that answers your question.

How much is left to merge this one?

goanpeca · 2015-03-31T14:53:10Z

I have to see what generates the error with that build (it only happens on that build) and then we should be almost ready to go.

ccordoba12 · 2015-03-31T16:56:29Z

.travis.yml

+  - source activate test-environment;
+  - QT_API=$USE_QT_API;
+  - echo $QT_API;
+  - python setup.py install;


This is not necessary because you're using bootstrap to start Spyder, so I think it should be removed.

True, maybe we should run both from bootstrap, and then from installed... that way we can check that it is installing correctly, no?

Mmmhh, but then you should need to add the --test flag again to the app cli options.

hmmmmm right... now, would it make sense to test the install or just keep the bootstrap?

For now we should just use bootstrap. Testing that Spyder installs should be done separately.

ccordoba12 · 2015-03-31T17:04:45Z

The PySide error doesn't seem to do with matplotlib (mpl just issues a warning) but with conda package manager.

Let me remove it to see if that fixes the problem.

goanpeca · 2015-03-31T17:07:11Z

ahhh true, I never tried the conda package thinguie with pyside on python 3....

ccordoba12 · 2015-03-31T17:27:05Z

Ok, I removed our conda plugin, so please try again to see if everything is passing now :-)

goanpeca · 2015-03-31T17:29:15Z

Ok, but I need to rebase my branch?

ccordoba12 · 2015-03-31T17:31:51Z

I don't know how travis works exactly :-) You shouldn't in principle (because there are no conflicts between removing conda and introducing travis.yml) but you probably should so that travis tests Spyder with the new changes

goanpeca · 2015-03-31T17:35:47Z

Ok, I will do this later

ccordoba12 · 2015-03-31T17:38:31Z

Could you finish this now? I'd like to merge other PRs and (because I didn't deactivate the yml option) they appear as Failed in the GH interface.

I wouldn't like to have this message for future references :-)

goanpeca · 2015-03-31T17:43:50Z

There is nothing that can be done with those messages, as they where generated where there was no actual yml to test, if you want to get those messages out, people would need to rebase after this PR is merged, so that their branches include the yml etc....

And sorry cannot do this now. I can in a couple of hours.

ccordoba12 · 2015-03-31T17:47:04Z

Ok, no biggie then :-) But I really like to merge this one ASAP!

blink1073 · 2015-03-31T20:28:46Z

@goanpeca, you'll have to rebase to pick up those changes, because your branch still includes the conda package.

goanpeca · 2015-03-31T21:16:43Z

Ok rebased...

but now I get this error on build 8.2

from PySide.QtGui import *                                # analysis:ignore
ImportError: /home/travis/miniconda/envs/test-environment/lib/python2.7/site-packages/PySide/QtGui.so: undefined symbol: _ZN8QPainter10drawPixmapERK6QRectFRK7QPixmapS2_

I found this when looking at google https://groups.google.com/a/continuum.io/forum/#!topic/anaconda/sUWNlhLykAU

And since anaconda recently updated to 2.2 (miniconda maybe did an update?) this maybe has to do something with that.

Comments @ccordoba12 , @blink1073 ?

ccordoba12 · 2015-03-31T21:25:24Z

That one is my fault ;-) I updated Qt on Anaconda, so the PySide packages need to be rebuilt against that new Qt version (we missed this fact, so thanks for letting us know about it :-)

I'll try to do that ASAP, but for now please disable the PySide/conda combination (until I let you know that is working again).

blink1073 · 2015-04-01T16:33:25Z

@Nodd, PySide is available in Conda for Python 2, but not Python 3.

goanpeca · 2015-04-02T09:03:26Z

Why wont continuum make a conda package for pyside and Python3?

Nodd · 2015-04-02T13:53:37Z

Well, threads like this one show that pyside development has gone very slow: http://lists.qt-project.org/pipermail/pyside/2015-February/002251.html

goanpeca · 2015-04-02T14:54:59Z

@Nodd Yes I have seen the thread, but if you are going to offer PySide for Python2 why not for Python3 ?

Why not just say that "support of pyside is over" and take all of it out of continuum repos, instead of offering half of it... if there are wheels that work, why not conda packages?

ccordoba12 · 2015-04-02T15:54:14Z

@goanpeca, I can answer that because I'm in charge of Qt/PyQt at Continuum:

Anaconda started using PySide (for licensing issues).
The only available Python version at the beginning was Python 2.
Anaconda maintainers (not me) noticed that PySide was very unstable (because at that time it was at version 1.1), so they switched to use PyQt4.
Then it came Anaconda for Python 3. At this time, PySide was not used anymore in the official installers, so no one felt the need to create a package for this Python version.

The end of the history is this: Continuum is not committed to support PySide anymore because PyQt4 are more stable and better maintained Qt bindings than PySide. I think the PySide package will continue to remain in the repos because of historical reasons and nothing more.

So instead of keep complaining about it, please remove the Conda/PySide combo from the testing matrix. We can still test PySide using Ubuntu and those wheels, I can't see any really problem with that.

Even if we found problems with conda and PySide, no one at Continuum (meaning me and the Anaconda team) would care about it. They would just say: "please use PyQt4 instead".

goanpeca · 2015-04-02T16:56:07Z

@ccordoba12 I do not have a problem in removing it from the matrix.

I use anaconda and I use pyside by default, so it is very annoying that after doing

conda update conda
conda update anaconda

Spyder goes @#%#$^$%^#%... so yes I should be able to complain "So instead of keep complaining about it, " is unnecessary

It would be better if they (continuum) did not offer pyside support at all instead of a half baked solution.

ccordoba12 · 2015-04-02T17:03:18Z

I'll talk to them but I wouldn't hold my breath. If you are using Anaconda, you should be using PyQt4, that's all :-)

Could you comment PySide from .travis.yml so we can merge this one? Then you can add it to in another PR when you make it work again.

Edit: If you want to use PySide, please use the Ubuntu packages instead (that's what I do :-), and change your interpreter to point to the Anaconda one.

goanpeca · 2015-04-02T17:16:11Z

@ccordoba12 give me a sec, I am doing the travis runs on a separate branch just to check that it runs before making the PR....

goanpeca · 2015-04-02T21:22:54Z

Ok, I guess now it should be fine....

build results

goanpeca · 2015-04-02T21:27:10Z

@ccordoba12 since you are the only one doing direct commits to the repo, it would be better if you used [ci skip] somewhere in the commit message (not the first line...!) and maybe only leave testing for PRs to avoid making too many build tests? not sure how fair use is defined in travis-ci

http://docs.travis-ci.com/user/how-to-skip-a-build/

ccordoba12 · 2015-04-02T21:37:00Z

For that to work I should start to do PRs for everything I do, instead of committing directly to the repo.

How else would we guarantee that Spyder is passing the tests all the time if not? ;-)

ccordoba12 · 2015-04-02T22:07:14Z

continuous_integration/travis_install.sh

+  export SOURCE=`source activate test-environment`
+  install_conda;
+else
+  export TEST_PACKAGES="IPython jedi matplotlib pandas pep8 psutil pyflakes pygments pylint sphinx sympy"


After seeing the travis build logs, I think this is causing matplotlib (and all the other packages in this line) to be installed with pip instead of using apt-get. Two questions:

Why not use apt-get, at least for mpl? The other packages are just pure Python packages, so there is no problem with using pip.

If that's not possible, can't use a wheel for mpl?

I say it because the workers are taking a huge amount of time to compile mpl, which is reasonable because mpl is a big package. But we shouldn't build mpl if we can avoid it :-)

Travis uses environments (with virtual env), so using apt-get will install packages for the global python which is 2.7 and that makes no sense for python 3.3, and 3.4. Travis VMs are all the same, what changes is the virtual envs....

Yes, we can use wheels (if they are available...) for all the other packages that take too long too compile. Otherwise we would need to make our own wheels for the different packages we are testing.

Then (as I said), please use a wheel for mpl. The other packages are fine.

For the moment only conda then.... I will add the wheels later on...

ccordoba12 · 2015-04-02T22:28:24Z

I think we should reduce the testing matrix, and just use conda for now. And no, it's not because I'm working at Continuum :-), it's because conda packages are (almost always) up to date and quite reliable.

I also say it because the wheels in http://travis-wheels.scikit-image.org/ seem outdated (there is no mpl 1.4.3 for example).

At the end, the tests using conda are already passing, and do so only taking 1 min or so. So why not start with that and leave the other options for later?

goanpeca · 2015-04-02T22:31:44Z

ok

goanpeca · 2015-04-02T22:59:01Z

So for the moment only conda... when we have a set of wheels ready and updated we can activate the rest

ccordoba12 · 2015-04-03T00:49:04Z

Yes, that's ok.

One last question: what do you recommend for my particular workflow? Should I start making PRs instead of pushing directly to master?

blink1073 · 2015-04-03T00:54:55Z

I agree with using all conda all the time, and letting you do whatever is easiest for you @ccordoba12. For instance, minrk pushes directly to master on IPython quite often.

goanpeca · 2015-04-03T13:02:18Z

Lets just keep doing things as usual and see how it goes 😄.

Is this ready for merging, or is there something I need to fix before?

I am starting with the tests now, specifically the source code editor.

Cheers

ccordoba12 · 2015-04-03T17:03:39Z

Ok, merging then! Thanks a lot for your work on this @goanpeca, this is immensely useful!

About the other tests, please start simple. Let's start with the Editor in one PR, and then the other plugins in further PRs.

Add travis support to test that Spyder starts correctly.

SylvainCorlay · 2015-04-05T16:07:37Z

Sorry I am late in the battle. I have the same issue when testing qtawesome with PySide: undefined symbol when trying to import it. Although I still think that supporting PySide is valuable, especially license-wise. Should I drop it from the conda tests?

goanpeca · 2015-04-05T18:38:36Z

@SylvainCorlay, no battle here ;-), sure disabled (comment the lines...) I will add pyside from normal installs (using wheels) later

Cheers

SylvainCorlay · 2015-04-05T18:53:53Z

@goanpeca it is just an idiom. / I hope anaconda will support pyside again in the future.

ccordoba12 · 2015-04-06T22:26:26Z

The official answer is that Continuum doesn't support PySide anymore.

However, I'll try to update it on my free time.

ccordoba12 added the type:Techdebt label Mar 27, 2015

ccordoba12 added this to the v3.0 milestone Mar 27, 2015

ccordoba12 reviewed Mar 27, 2015
View reviewed changes

goanpeca self-assigned this Mar 31, 2015

ccordoba12 reviewed Mar 31, 2015
View reviewed changes

Nodd mentioned this pull request Apr 1, 2015

Anaconda 2.2 update causes Spyder to crash on KDE based Linux distros #2294

Closed

ccordoba12 reviewed Apr 2, 2015
View reviewed changes

Add travis support for basic start of spyder

1637bc4

ccordoba12 added a commit that referenced this pull request Apr 3, 2015

Merge pull request #2281 from goanpeca/travis-test

0df9bb6

Add travis support to test that Spyder starts correctly.

ccordoba12 merged commit 0df9bb6 into spyder-ide:master Apr 3, 2015

goanpeca deleted the travis-test branch April 11, 2015 10:33

Add travis support for basic start of spyder #2281

Add travis support for basic start of spyder #2281

Conversation

goanpeca commented Mar 27, 2015

goanpeca commented Mar 27, 2015

blink1073 commented Mar 27, 2015

goanpeca commented Mar 27, 2015

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ccordoba12 commented Mar 29, 2015

goanpeca commented Mar 30, 2015

goanpeca commented Mar 30, 2015

blink1073 commented Mar 31, 2015

goanpeca commented Mar 31, 2015

blink1073 commented Mar 31, 2015

ccordoba12 commented Mar 31, 2015

goanpeca commented Mar 31, 2015

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ccordoba12 commented Mar 31, 2015

goanpeca commented Mar 31, 2015

ccordoba12 commented Mar 31, 2015

goanpeca commented Mar 31, 2015

ccordoba12 commented Mar 31, 2015

goanpeca commented Mar 31, 2015

ccordoba12 commented Mar 31, 2015

goanpeca commented Mar 31, 2015

ccordoba12 commented Mar 31, 2015

blink1073 commented Mar 31, 2015

goanpeca commented Mar 31, 2015

ccordoba12 commented Mar 31, 2015

blink1073 commented Apr 1, 2015

goanpeca commented Apr 2, 2015

Nodd commented Apr 2, 2015

goanpeca commented Apr 2, 2015

ccordoba12 commented Apr 2, 2015

goanpeca commented Apr 2, 2015

ccordoba12 commented Apr 2, 2015

goanpeca commented Apr 2, 2015

goanpeca commented Apr 2, 2015

goanpeca commented Apr 2, 2015

ccordoba12 commented Apr 2, 2015

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ccordoba12 commented Apr 2, 2015

goanpeca commented Apr 2, 2015

goanpeca commented Apr 2, 2015

ccordoba12 commented Apr 3, 2015

blink1073 commented Apr 3, 2015

goanpeca commented Apr 3, 2015

ccordoba12 commented Apr 3, 2015

SylvainCorlay commented Apr 5, 2015

goanpeca commented Apr 5, 2015

SylvainCorlay commented Apr 5, 2015

ccordoba12 commented Apr 6, 2015