Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

r/aws_s3_bucket: Speed up deletion #24020

Merged
merged 23 commits into from
Apr 5, 2022
Merged

r/aws_s3_bucket: Speed up deletion #24020

merged 23 commits into from
Apr 5, 2022

Commits on Apr 5, 2022

  1. The start of some ideas around improving bucket emptying.

    @ewbankkit
    ewbankkit committed Apr 5, 2022
    Configuration menu
    Copy the full SHA
    4485e12 View commit details
    Browse the repository at this point in the history

  2. Ignore any object versions that do not have the required Key.

    @ewbankkit
    ewbankkit committed Apr 5, 2022
    Configuration menu
    Copy the full SHA
    d8cc0e6 View commit details
    Browse the repository at this point in the history

  3. Add 'deleteDeleteMarkerListIterator'.

    @ewbankkit
    ewbankkit committed Apr 5, 2022
    Configuration menu
    Copy the full SHA
    658f663 View commit details
    Browse the repository at this point in the history

  4. Add 'bypassGovernanceRetention' flag.

    @ewbankkit
    ewbankkit committed Apr 5, 2022
    Configuration menu
    Copy the full SHA
    d69ce4b View commit details
    Browse the repository at this point in the history

  5. Start implementing 'emptyBucket'.

    @ewbankkit
    ewbankkit committed Apr 5, 2022
    Configuration menu
    Copy the full SHA
    4cf860d View commit details
    Browse the repository at this point in the history

  6. Drop use of 's3manager.BatchDelete'.

    @ewbankkit
    ewbankkit committed Apr 5, 2022
    Configuration menu
    Copy the full SHA
    7d97ae5 View commit details
    Browse the repository at this point in the history

  7. Tweak error messages.

    @ewbankkit
    ewbankkit committed Apr 5, 2022
    Configuration menu
    Copy the full SHA
    ff9fd29 View commit details
    Browse the repository at this point in the history

  8. Simple Python script to populate a versioned S3 bucket with objects.

    @ewbankkit
    ewbankkit committed Apr 5, 2022
    Configuration menu
    Copy the full SHA
    0c0f977 View commit details
    Browse the repository at this point in the history

  9. Make 'EmptyBucket' public so that it can be tested.

    @ewbankkit
    ewbankkit committed Apr 5, 2022
    Configuration menu
    Copy the full SHA
    9252e2d View commit details
    Browse the repository at this point in the history

  10. Test harness for the EmptyBucket function.

    @ewbankkit
    ewbankkit committed Apr 5, 2022
    Configuration menu
    Copy the full SHA
    cdab1b8 View commit details
    Browse the repository at this point in the history

  11. Add ability to lock an S3 object.

    @ewbankkit
    ewbankkit committed Apr 5, 2022
    Configuration menu
    Copy the full SHA
    1c7a86a View commit details
    Browse the repository at this point in the history

  12. Attempt to remove legal hold from locked S3 objects.

    @ewbankkit
    ewbankkit committed Apr 5, 2022
    Configuration menu
    Copy the full SHA
    e3e05dc View commit details
    Browse the repository at this point in the history

  13. Attempt to delete the object once the legal hold has been removed.

    @ewbankkit
    ewbankkit committed Apr 5, 2022
    Configuration menu
    Copy the full SHA
    1f5f443 View commit details
    Browse the repository at this point in the history

  14. r/aws_s3_bucket: Use 'EmptyBucket' to speed up 'force_destroy'.

    @ewbankkit
    ewbankkit committed Apr 5, 2022
    Configuration menu
    Copy the full SHA
    04ff2d6 View commit details
    Browse the repository at this point in the history

  15. Add CHANGELOG entry.

    @ewbankkit
    ewbankkit committed Apr 5, 2022
    Configuration menu
    Copy the full SHA
    3668084 View commit details
    Browse the repository at this point in the history

  16. Return the number of deleted S3 objects from 'EmptyBucket'.

    @ewbankkit
    ewbankkit committed Apr 5, 2022
    Configuration menu
    Copy the full SHA
    10b819c View commit details
    Browse the repository at this point in the history

  17. Return the number of deleted S3 objects from 'DeleteAllObjectVersions'.

    @ewbankkit
    ewbankkit committed Apr 5, 2022
    Configuration menu
    Copy the full SHA
    66a0d54 View commit details
    Browse the repository at this point in the history

  18. Fix S3 sweeper.

    @ewbankkit
    ewbankkit committed Apr 5, 2022
    Configuration menu
    Copy the full SHA
    23a6d36 View commit details
    Browse the repository at this point in the history

  19. Use 'go test' to test 'EmptyBucket'.

    @ewbankkit
    ewbankkit committed Apr 5, 2022
    Configuration menu
    Copy the full SHA
    c92fe35 View commit details
    Browse the repository at this point in the history

  20. Add 'TestDeleteAllObjectVersions'.

    @ewbankkit
    ewbankkit committed Apr 5, 2022
    Configuration menu
    Copy the full SHA
    731c697 View commit details
    Browse the repository at this point in the history

  21. Skip 's3.ErrCodeNoSuchKey' errors when emptying an S3 bucket.

    @ewbankkit
    ewbankkit committed Apr 5, 2022
    Configuration menu
    Copy the full SHA
    74fc23d View commit details
    Browse the repository at this point in the history

  22. Minor tweak.

    @ewbankkit
    ewbankkit committed Apr 5, 2022
    Configuration menu
    Copy the full SHA
    3b2c9b4 View commit details
    Browse the repository at this point in the history

  23. Update 24020.txt

    @ewbankkit
    ewbankkit authored Apr 5, 2022
    Configuration menu
    Copy the full SHA
    2ebac57 View commit details
    Browse the repository at this point in the history