Move authoritative release storage to the cloud

[kenperkins]

Currently, the single server for iojs.org is the authoritative source for builds. When a build server finishes, it directly scp's the build to iojs.org.

I'm proposing we make the build outputs store the builds in the cloud, and then sync back down to the iojs.org website. This would allow a quick recovery should we lose our webserver, or if we need to spin up additional capacity.

This could theoretically work in conjunction to #55.

[jbergstroem]

Using github for releases would provide s3 storage but that still leaves nightlies out in the cold.

[rvagg]

having run in to space constraints on the (only) server we're running iojs.org on this week I'm thinking that this is perhaps even more urgent to get sorted out. S3 would be great but we'd need $$ for that. I'm not as familiar with the rackspace alternative(s) but I'm guessing that the costs could be covered under our sponsorship there.

This also goes hand-in-hand with HA for iojs.org, we need to rearchitect the whole deal for the website and hosting.

[kenperkins]

I'm thinking I want to do a cluster (2 minimum) behind a load balancer, with block storage volumes for a local cache of builds and authoritative storage in cloud files.

[rvagg]

👍 sounds ideal to me, not something we can do on DO atm so I guess Rackspace is the likely new home for this but I know very little about both the block storage or the cloud files service there so you'll have to drive that architecture.

[rosskukulinski]

While @kenperkins knows Rackspace's products backwards and forwards, I'd be happy to lend a hand with architecture design review and/or development if you'd like @rvagg. I've been using Rackspace in production for 2+ years now, so I should be just dangerous enough.

[rvagg]

brilliant! make sure you stay tuned in to this then @rosskukulinski

[rosskukulinski]

Will do @rvagg. I'm in the same tz as @kenperkins so I'll try to make it to the next build WG.

[jbergstroem]

We're now using cloudflare to front our downloads. I wouldn't call it storage (rather cache), but it at least offers redundancy and availability should we briefly go offline. We should still explore s3 for proper storage though

[rvagg]

We should .. and tbh I don't think we've actually got full redundancy with cloudflare yet—we have it configured so that it does a pass-through for tarballs and installers so we can log them, the intention was for the configuration to also serve them direct from CF if the server isn't reachable but I don't think that's working in practice. I'm obviously not keen to take the server down to test it out but we need to work it out.

[jbergstroem]

I just saw those settings. Must be a better way..

[rvagg]

the better way is to use the log file upload feature of enterprise CF, that's been offered and we just need to experiment with it.

[DavidTPate]

With CloudFront + S3 what you would want to do for this is setup an S3 bucket for the downloads which can be either Private (and accessible only through CloudFront) or Public (there's other policies too, but for downloads they don't matter as much). You would then set it up to use the S3 bucket as your origin and then be able to use CloudFront as the delivery mechanism.

To get the download counts you would want to turn on Logs for CloudFront and then use some code (or a service) to analyze those to get statistics.

[Trott]

@nodejs/build @kenperkins Should this remain open?

[sam-github]

Closing as stale, but if anyone wants to take this up feel free to reopen.