"I say we take off and nuke the site from orbit. It's the only way to be sure."
-- Ripley, Aliens
S3Nuke uses EventMachine to run many operations in parallel on Amazon S3 buckets, allowing faster completion than ordinary mortal single-threaded clients. It can be used to download, upload, delete, or change headers on buckets with millions of objects. It offers both a Ruby API for applying your own code to each retrieved object in a bucket and a command-line client for executing the most common operations. It may also be a floor wax and a dessert topping, but those use cases are untested.
HISTORICAL NOTE: There was once a single-file command line script called s3nuke
by the same author, which had the sole function of deleting large buckets using multiple threads. This gem is a complete reconception and shares no code with that script. It can still be found on the Github branch named "original", or you can check out an improved version by Robert LaThanh. This author apologizes for any confusion caused by reusing the name. (But does not repent.)
Come on kids, we've all done this a lot:
gem install s3nuke
Or add it to your Gemfile. The gem depends on eventmachine (for concurrency) and clamp (for the command line utility). Run s3nuke
to see the options.
(Still in progress...)