MINOR: Switch to use AWS spot instance to save 76% in cost

[maxzheng]

Pricing for m3.xlarge: On-Demand is at $0.266. Reserved is at about $0.16 (40% discount). And Spot is at $0.0627 (76% discount relative to On-Demand, or 60% discount relative to Reserved). Insignificant fluctuation in the past 3 months.

Ran on branch builder and works as expected -- each worker is created using spot instances (https://jenkins.confluent.io/job/system-test-kafka-branch-builder/1982/console)

This can be safely backported to 0.10.2 (tested using https://jenkins.confluent.io/job/system-test-kafka-branch-builder/1983/)

Committer Checklist (excluded from commit message)

• Verify design and implementation
• Verify test coverage and CI build status
• Verify documentation (including upgrade notes)

[maxzheng]

FYI Based on the 6 branches (12 workers per branch from 0.10.2 to trunk), that's 72 hosts running about ~11 hours per day, the cost for on-demand is about $6,300 per month. The spot instance price is at $1,500 per month, so we will save about $4,800 per month with this change.

CC @ewencp

[ewencp]

overall seems fine except for question re: spot pricing.

if there are no concerns there and re: overrides, happy to merge and cherry-pick back as far as needed. sounds like 0.10.2 is where we'd definitely want it, should we go any further back or is 0.10.2 enough?

[ewencp]

I see we can override this via env variable, but having it fixed in code as a default seems like a bad baseline. Are we just always overriding this value in practice anyway?

[maxzheng]

There is not much use for the env as it is normally not used, so removing it in favor of a sane default based on instance type.

As for cherry-picking, as someone started testing 0.10.1, I think we can go back to there. There will be a merge conflict, but it's pretty straight forward / just due to Vagrant version change, which is the same version in 0.10.2 and that was tested to work.

[ewencp]

LGTM, thanks!