Skip to content
This repository has been archived by the owner on Feb 12, 2024. It is now read-only.

chore: split test jobs into stages #2636

Closed
wants to merge 1 commit into from
Closed

Conversation

alanshaw
Copy link
Member

The current test stage uses all the workers we have available in Travis and starves other projects from running their tests. This change separates interop, externals and examples into stages that only execute if the previous was successful. This means that we use fewer workers concurrently and fewer workers overall since a failure in the main tests will not process and run interop/externals and examples tests. This is at the expense of slower overall CI.

The current test stage uses _all_ the workers we have available in Travis and starves other projects from running their tests. This change separates interop, externals and examples into stages that only execute if the previous was successful. This means that we use fewer workers concurrently and fewer workers overall since a failure in the main tests will not process and run interop/externals and examples tests. This is at the expense of slower overall CI.

License: MIT
Signed-off-by: Alan Shaw <[email protected]>
@alanshaw alanshaw marked this pull request as ready for review November 26, 2019 09:58
@achingbrain
Copy link
Member

Assuming zero flaky tests this will extend CI time to almost two hours (sum of the longest build time in each stage) which starts to impede how many builds we can do in a day. We really need to make our tests faster!

@daviddias
Copy link
Member

We really need to make our tests faster!

We got an @hugomrdias working on just that! :D #2276

@alanshaw
Copy link
Member Author

Perhaps limiting concurrent jobs is a better option? https://docs.travis-ci.com/user/customizing-the-build/#limiting-concurrent-jobs

@alanshaw
Copy link
Member Author

I've limited to 20, so that there's always at least 5 workers available to service other projects.

@alanshaw
Copy link
Member Author

alanshaw commented Dec 2, 2019

Closing this for now. @daviddias please shout if you still feel js-ipfs is still not being a good CI citizen.

@alanshaw alanshaw closed this Dec 2, 2019
@daviddias
Copy link
Member

I'll avoid policing and let the team find the optimal point on ways it share the CI. My only remarks are:

  • An artificial number will always be arbitrary and non optimal
  • If all repos take a similar approach (in which they have to run 20+ jobs), using a 20 jobs ceiling would not scale.
  • An observation is that if the primary tests of JS IPFS do not capture all the breakage over time, then they are probably not through enough.
  • Another observation, If the tests still take what it feels like too much time, it is probably because they are not parallel enough

Remarks said, as long as the current situation works for everyone, this doesn't feel urgent, so prioritize accordingly :)

@daviddias daviddias deleted the chore/moar-stages branch December 2, 2019 17:17
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants