Paver (Part 1) -- Attempt 2

[wedaly]

When I merged the Paver PR into master, Jenkins started failing in bizarre ways. It turns out that:

1. When we run the test suite, we pip install -e . (here: https://github.com/edx/edx-platform/blob/master/requirements/edx/local.txt#L2)
2. This adds the path to the current repo directory into sys.path.
3. However, in Jenkins, there are multiple copies of the repository (for each job, and each sub-job in a multiconfig job).
4. Therefore, when we first use Python, sys.path may contain the path to another repo.
5. Because we run paver before re-running pip install on local requirements, the pavelib/assets.py in another copy of the repository was taking precedence over the local copy.
6. The other copy of pavelib/assets.py constructed a path to the wrong repository, where the prereqs cache indicated that coffee had already been installed (in "node_modules/.bin")
7. Since coffee had NOT, in fact, been installed in the current repository, coffescript compilation would fail.

Whew!

The right solution is probably to have Jenkins better isolate virtualenvs between jobs. But that would be unacceptably slow, which is why we don't do it.

The solution I've adopted here is to insert the current working directory as the first item in sys.path before loading the pavement modules. This ensures that Paver thinks it's in the right directory.

The only difference between this and the original PR is the sys.path manipulation in pavement.py, which includes a detailed comment.

[yarko]

@wedaly - ah; makes sense. Soon as you merge this, I'll rebase my https://github.com/yarko/edx-platform/tree/yarko/pr-2520-default-variant and PR it to add fixes for custom theme behavior (similar to what #2387 fixes w/ rake).

[jzoldak]

@wedaly I think you want to sys.path.insert(1,foo)?
That seems to be the standard way for prepending a dir to the python path.
See http://docs.python.org/2/library/sys.html#sys.path

[jzoldak]

@wedaly, @Cale had this on his branch and noticed that the list of python prereqs wasn't sufficient (it didn't include github.txt or private.txt). Let's investigate on the next round.

[wedaly]

@jzoldak "github.txt" is included implicitly in "base.txt" (by way of "repo.txt").

I'm not sure why "private.txt" is necessary -- why wouldn't someone just install extra requirements in the virtualenv?