Older PHP versions no longer available on travis' trusty image

[notnownikki]

We've switched to using precise for older PHP tests in our travis configuration, because they're migrating to trusty and older PHP and PHPUnit versions are no longer installable on trusty. This is only a temporary measure, as in September, the precise image will be going away.

I've started work on scripts that will install PHP5.3 and PHP5.2 on a trusty install, along with the needed versions of PHPUnit. The installs and the scripts to switch between PHP versions will be contained within the travis user's home directory, so travis will be able to cache them so we're not building PHP every time we run tests.

[ntwb]

Thanks 👍

See also #WP:41292#comment:34 Switch Travis CI environment to Trusty - PHP 5.2 & 5.3

[notnownikki]

Work in progress here #2049

Supports PHP 5.2 and 5.3, relies on the SWITCH_TO_PHP env var to tell it what old PHP version to prepare and switch to.

[notnownikki]

There are a couple of issues I know about already:

• SSL support doesn't seem to work, couldn't fetch https urls on my local tests. Some build option isn't right here, but not sure what yet.
• The check to see if phpbrew has already run and done the needed build is wrong, it's checking for the phpbrew bashrc and not the requested PHP version. This would mess up when we cache the phpbrew directory, and have only done one of the builds.

[notnownikki]

Also there's a problem switching to the correct phpunit version which I'm not seeing locally.

[notnownikki]

https://travis-ci.org/WordPress/gutenberg/jobs/258073280 - successful run on trusty, using php5.2 and phpunit3.6

[notnownikki]

Time to talk about caching.

Travis can cache the built PHP versions, but it doesn't work out very efficient in terms of storage.

Travis caches per branch. If cache does not exist for a branch, it gets the cache for master. That's not a problem, when this goes into master it'll build and would cache the built PHP.

But, when a branch builds, or a PR builds, it'll create new caches for the branch and PR. That's going to be an extra 147Mb for every branch and every PR (assuming we are building for PHP5.2 and PHP5.3, as we would on WP core).

Our options seem to be:

• Take the storage hit. (Do we get charged for storage? Are these caches frequently cleared out? I can't find that information)
• Build PHP/PHPUnit each time we test. This adds around 10 minutes to the run time.
• Build PHP/PHPUnit once, create a tarball, and host it somewhere that the test can download.

@nylen I'd appreciate your input on this one!

[notnownikki]

I see this in the travis cache docs:

Only modifications made to the cached directories from normal pushes are stored.

Although when I look at the cache sizes for each branch and PR in gutenberg, they're all about the same size.

Perhaps we just go with the travis caching, monitor the size, ask support if it's an issue, and host a tarball if it is?

[ntwb]

Also there's a problem switching to the correct phpunit version which I'm not seeing locally.
https://travis-ci.org/WordPress/gutenberg/jobs/258073280 - successful run on trusty, using php5.2 and phpunit3.6

In this job I see PHPUnit version 6.2.3 being ran and not 3.6 https://travis-ci.org/WordPress/gutenberg/jobs/258073280#L3727

---

Our use of Travis CI is 100% Free, we shouldn't abuse it, but we are free to use it.

I wouldn't worry about the size of any caching, as part of #WP30462 we're looking at 7 x 300mb of additional MySQL packages being added to the Travis CI cache, from testing that, it worked fine, though it's been a while.

caching side note: There's an insane number of stale caches there from old pull requests, not sure how we clean those up, it's all or one by one, maybe we should flush them all?

Edit: We should cache all the things we possibly can

I think working as the PR is currently architected should be fine to get "proof of concept" up and running, once we're happy with that and see it working there's a few options available, for the time it takes for the job to complete we can set this job up to run once daily rather than every build, for performance we can build it once and host the tarball on w.org or a GitHub repo and leverage Travis CI's cache for it.

[notnownikki]

I see PHPUnit version 6.2.3 being ran and not 3.6

I'm going through that now... ugh, at least it's building... perhaps there's something not persisting between the before_script and the main script.

[ntwb]

I've not yet had the opportunity to try out Travis CI's "debug mode"

https://docs.travis-ci.com/user/running-build-in-debug-mode/

I'm off for the night here now (~9pm local time here), I'll take a closer look at the PR in the morning and take a stab at some debugging if you've not beaten me to it.

[notnownikki]

I think it's a path issue, and as it's only midday here I should have it working by the time you wake up :)

[ntwb]

One last note, it might be worth moving as much of this out of the before_script section into a new before_install section, at worst it will separate a few things for readability, or it might help in how Travis CI determines the load order of the extra packages loaded via apt

https://docs.travis-ci.com/user/customizing-the-build#The-Build-Lifecycle

Thanks @notnownikki 👍

💤💤💤

[notnownikki]

Path issues! Travis installs a set of php shims, which includes phpunit, so we have to override that. I'll need to write a small script that determines which phpunit path to use based on the version of php in use.

[notnownikki]

PHPUnit shim is in there now, it's running tests with the correct phpunit and php version.

Now it seems that phpbrew hasn't enabled mysql properly. Thankfully I can debug that locally...

[nylen]

Perhaps we just go with the travis caching, monitor the size, ask support if it's an issue, and host a tarball if it is?

This is my vote as well. I can't find any specific information about disk size limits, though I did find a reference to an execution time limit of 30 minutes.

Frankly I don't feel too bad about the extra disk space because (1) they dropped support for PHP 5.2 (understandably, but we still need it for now), and (2) we are not on their free plan.

[notnownikki]

Progress! It works, and is using the correct php and phpunit versions.

When we set the env var SWITCH_TO_PHP to 5.3 or 5.2, we build both versions of PHP. This is so we can cache the .phpbrew directory without race conditions. Next time, the builds come from the cache and we skip the build process.

PHPUnit 3.6 and 4.8 get installed. 4.8 from a phar, 3.6 from git repositories. There's a shim which picks the right version depending on which version of php you're running. If you're not running 5.2 or 5.3, the shim doesn't get installed.

The build script and php patches have been moved into the repo instead of existing in the travis config or getting downloaded each time.

Initial build takes about 25 minutes, but that's only the first time it's run on a branch. Once it's in master, branches will pick up the cached build from master.

[ntwb]

(2) we are not on their free plan.

❤️ Holy concurrent jobs batman! This is fantastic news 💥

Thanks @nylen for organizing this, it was on my todo list to negotiate a paid plan for Travis CI

p.s. I never saw the announcement post as make/core subs were not working for a few weeks

[notnownikki]

I've simplified the phpunit installation now, and removed the WIP status from the PR. I'm happy with the state of it now, reviews are welcomed!

(And yes, I will squash before commit!!)

[ntwb]

💥 LGTM, fantastic job @notnownikki 👍

We can monitor the cache size, job times over the coming days and see if any tweaks may be required