Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

MalformedXML Error when deleting versioned S3 bucket with force_destroy #3981

Closed
aviflax opened this issue Nov 19, 2015 · 6 comments
Closed

MalformedXML Error when deleting versioned S3 bucket with force_destroy #3981

aviflax opened this issue Nov 19, 2015 · 6 comments
Labels
bug provider/aws

Comments

@aviflax
Copy link

aviflax commented Nov 19, 2015

As first reported in this mailing list thread.

My bucket resource looks like this:

resource "aws_s3_bucket" "REDACTED_site" {
    bucket = "${var.teleport_site_bucket_name_prefix}.${var.environment}"
    acl = "public-read"
    policy = "${template_file.REDACTED_site_bucket_policy.rendered}"
    force_destroy = true

    website {
        index_document = "REDACTED.html"
        error_document = "error.html"
    }

    versioning {
        enabled = true
    }

    tags {
        Component = "REDACTED"
    }
}

The bucket creates just fine and works just fine, and I can put objects into it and those objects are correctly versioned.

But when I try to destroy that bucket — when there are objects in it — I get this:

aws_s3_bucket.REDACTED_site (destroy)
2015/11/19 09:19:04 Skipping orphan transformer because we have targets.
2015/11/19 09:19:04 [TRACE] Graph after step *terraform.OrphanTransformer:

aws_s3_bucket.REDACTED_site (destroy)
2015/11/19 09:19:04 [TRACE] Graph after step *terraform.DeposedTransformer:

aws_s3_bucket.REDACTED_site (destroy)
2015/11/19 09:19:04 [TRACE] Graph after step *terraform.RootTransformer:

aws_s3_bucket.REDACTED_site (destroy)
2015/11/19 09:19:04 [DEBUG] vertex root.aws_s3_bucket.REDACTED_site (destroy): walking
2015/11/19 09:19:04 [DEBUG] vertex root.aws_s3_bucket.REDACTED_site (destroy): evaluating
2015/11/19 09:19:04 [DEBUG] root: eval: *terraform.EvalOpFilter
2015/11/19 09:19:04 [DEBUG] root: eval: *terraform.EvalSequence
2015/11/19 09:19:04 [DEBUG] root: eval: *terraform.EvalReadDiff
2015/11/19 09:19:04 [DEBUG] root: eval: *terraform.EvalFilterDiff
2015/11/19 09:19:04 [DEBUG] root: eval: *terraform.EvalIf
2015/11/19 09:19:04 [DEBUG] root: eval: terraform.EvalNoop
2015/11/19 09:19:04 [DEBUG] root: eval: *terraform.EvalGetProvider
2015/11/19 09:19:04 [DEBUG] root: eval: *terraform.EvalReadState
2015/11/19 09:19:04 [DEBUG] root: eval: *terraform.EvalRequireState
2015/11/19 09:19:04 [DEBUG] root: eval: *terraform.EvalApply
aws_s3_bucket.REDACTED_site: Destroying...
2015/11/19 09:19:04 [DEBUG] apply: aws_s3_bucket.REDACTED_site: executing Apply
2015/11/19 09:19:04 [DEBUG] terraform-provider-aws: 2015/11/19 09:19:04 [DEBUG] S3 Delete Bucket: peggedsoftware.REDACTED.site.staging
2015/11/19 09:19:04 [DEBUG] terraform-provider-aws: 2015/11/19 09:19:04 [DEBUG] S3 Bucket attempting to forceDestroy BucketNotEmpty: The bucket you tried to delete is not empty. You must delete all versions in the bucket.
2015/11/19 09:19:04 [DEBUG] terraform-provider-aws:     status code: 409, request id:
2015/11/19 09:19:04 [DEBUG] terraform-provider-aws: 2015/11/19 09:19:04 [DEBUG] S3 Delete Bucket: peggedsoftware.REDACTED.site.staging
2015/11/19 09:19:05 [DEBUG] terraform-provider-aws: 2015/11/19 09:19:05 [DEBUG] S3 Bucket attempting to forceDestroy BucketNotEmpty: The bucket you tried to delete is not empty. You must delete all versions in the bucket.
2015/11/19 09:19:05 [DEBUG] terraform-provider-aws:     status code: 409, request id:
2015/11/19 09:19:05 [DEBUG] root: eval: *terraform.EvalWriteState
2015/11/19 09:19:05 [DEBUG] root: eval: *terraform.EvalApplyPost
2015/11/19 09:19:05 [ERROR] root: eval: *terraform.EvalApplyPost, err: 1 error(s) occurred:

* aws_s3_bucket.REDACTED_site: Error S3 Bucket force_destroy error deleting: MalformedXML: The XML you provided was not well-formed or did not validate against our published schema
    status code: 400, request id: 
2015/11/19 09:19:05 [ERROR] root: eval: *terraform.EvalSequence, err: 1 error(s) occurred:

* aws_s3_bucket.REDACTED_site: Error S3 Bucket force_destroy error deleting: MalformedXML: The XML you provided was not well-formed or did not validate against our published schema
    status code: 400, request id: 
2015/11/19 09:19:05 [ERROR] root: eval: *terraform.EvalOpFilter, err: 1 error(s) occurred:

* aws_s3_bucket.REDACTED_site: Error S3 Bucket force_destroy error deleting: MalformedXML: The XML you provided was not well-formed or did not validate against our published schema
    status code: 400, request id: 
2015/11/19 09:19:05 [DEBUG] vertex aws_s3_bucket.REDACTED_site, got dep: aws_s3_bucket.REDACTED_site (destroy)
2015/11/19 09:19:05 [DEBUG] vertex provider.aws (close), got dep: aws_s3_bucket.REDACTED_site
2015/11/19 09:19:05 [DEBUG] vertex root, got dep: provider.aws (close)
2015/11/19 09:19:05 [DEBUG] vertex root, got dep: provisioner.remote-exec
2015/11/19 09:19:05 [DEBUG] vertex root, got dep: provisioner.chef
2015/11/19 09:19:05 [DEBUG] vertex root, got dep: provisioner.file
Error applying plan:

1 error(s) occurred:

* aws_s3_bucket.REDACTED_site: Error S3 Bucket force_destroy error deleting: MalformedXML: The XML you provided was not well-formed or did not validate against our published schema
    status code: 400, request id:

While the bucket is not destroyed, I can confirm that all the objects in the bucket appear to be, at least in the S3 console. However, it’s possible that the objects were only logically deleted, i.e. replaced with “deletion markers” in S3 parlance, i.e. the “versions” still exist in the bucket. Oh wait, scratch that, I just found the “show versions” button in the console, and that is indeed the case.

I’m running Terraform 0.6.6 installed via Homebrew on Mac OS X 10.11.1.
@combor
Copy link
Contributor

combor commented Dec 4, 2015

We have created #4168 to fix this.

@aviflax
Copy link
Author

aviflax commented Dec 5, 2015 via email

mtekel added a commit to alphagov/paas-cf that referenced this issue Dec 7, 2015
@mtekel
Add destroy pipeline
6f91e9e 
Add a pipeline to destroy environments. It currently consists of 2 jobs
- one to destroy VPC which then triggers a job to destroy S3 bucket. We
used default/recommended pipeline/task/script distribution with this
pipeline. We include taks from file, because this task repeats. Never-
theless, in this case, the pipeline needs to contain quite chunky
config definitions (params), because they can't be included in the task
itself, due to fly only expanding params in the pipeline.

In comparison with deploy pipeline, we don't use terraform's ability to
pull remote state file, due to bug encountered with `bucket.tfstate`:
hashicorp/terraform#4154

The remote state file worked with VPC, but we wanted to unify both
tasks on same code to demonstrate default concourse pipeline file
organization.

We also use fake/dummy `trigger-bucket-destroy` resource here to trigger
bucket destroy job after vpc destroy job.

The terraform bucket destroy job currently doesnt't work, due to bugs:
hashicorp/terraform#3981

and missing functionality - the old file versions are actually not
removed. @combor has risen PR to add this functionality here:
hashicorp/terraform#4168
@catsby
Copy link
Contributor

catsby commented Dec 9, 2015

#4168 was merged! I'm going to close this 😄

@catsby catsby closed this as completed Dec 9, 2015
@aviflax
Copy link
Author

aviflax commented Dec 10, 2015

🎉

@jayudhandha
Copy link

jayudhandha commented Sep 28, 2018 •
edited
Loading

Hi,

Today i got this issue with v0.11.8. But this is not reproducible now.

aws_s3_bucket.test_bucket: Error S3 Bucket force_destroy error deleting: MalformedXML: The XML you provided was not well-formed or did not validate against our published schema

I re-run the stack again, And it works fine!

Any help?

@ghost
Copy link

ghost commented Apr 1, 2020

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

@ghost ghost locked and limited conversation to collaborators Apr 1, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
bug provider/aws
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@catsby @aviflax @combor @jen20 @jayudhandha