Add dead letter policy to pubsub

[maroux]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment. If the issue is assigned to the "modular-magician" user, it is either in the process of being autogenerated, or is planned to be autogenerated soon. If the issue is assigned to a user, that user is claiming responsibility for the issue. If the issue is assigned to "hashibot", a community member has claimed the issue already.

Description

Pubsub recently added native support for dead letter policy and it went GA today. The API does support it but it's not documented here yet, but can be seen in the discovery document.

New or Affected Resource(s)

• google_pubsub_subscription

Potential Terraform Configuration

resource "google_pubsub_subscription" "subscription" {
  name  = "my-subscription"
  topic = "my-topic"

  dead_letter_policy {
    topic = "my-dlq-topic"
    topic_project = "my-dlq-project"
  }
}

References

From gcloud v278.0.0 release notes (which haven't been updated on the website yet):

     Dead Letter Queue Options. The Cloud Pub/Sub service account associated
     with the enclosing subscription's parent project (i.e.,
     service-{project_number}@gcp-sa-pubsub.iam.gserviceaccount.com) must have
     permission to Publish() to this topic and Acknowledge() messages on this
     subscription.
       --max-delivery-attempts=MAX_DELIVERY_ATTEMPTS
          Maximum number of delivery attempts for any message. The value must
          be between 5 and 100. Defaults to 5. --dead-letter-topic must also be
          specified.
       Dead letter topic resource - Name of the topic to publish dead letter
       messages to. The arguments in this group can be used to specify the
       attributes of this resource.
         --dead-letter-topic=DEAD_LETTER_TOPIC
            ID of the dead-letter-topic or fully qualified identifier for the
            dead-letter-topic. This flag must be specified if any of the other
            arguments in this group are specified.
         --dead-letter-topic-project=DEAD_LETTER_TOPIC_PROJECT
            Project ID of the Google Cloud Platform project for the
            dead-letter-topic.

[maroux]

I can work on this if there's no objection to adding a feature which is publicly available, but isn't publicly documented just yet (it's just a delay in updating the webpage).

[danawillow]

Hey @maroux, that's totally fine! Thanks for helping. You can read more about contributing at https://github.com/terraform-providers/terraform-provider-google/blob/master/.github/CONTRIBUTING.md. The pubsub subscription resource is a generated one, so you'll want to make your changes at https://github.com/GoogleCloudPlatform/magic-modules.

Assigning to hashibot to show this is being worked on.

[danawillow]

Oh cool apparently I don't need to assign to hashibot anymore. It's yours!

[danawillow]

Hey @maroux, just checking in here. Did you still want to contribute this, or should I unassign you?

[maroux]

hey @danawillow - I was waiting to get access to DLQ beta so I don't work blind but reps are being slow. Happy for someone else to take over if they're willing.

[danawillow]

Got it, I didn't realize the feature wasn't public yet. We generally try to keep the provider for publicly available features only, so might just be best to wait for a public beta.

[shaxbee]

This seems to be available in v285:

NAME
    gcloud pubsub subscriptions update - updates an existing Cloud Pub/Sub
        subscription

SYNOPSIS
    gcloud pubsub subscriptions update SUBSCRIPTION
        [--ack-deadline=ACK_DEADLINE] [--expiration-period=EXPIRATION_PERIOD]
        [--message-retention-duration=MESSAGE_RETENTION_DURATION]
        [--push-auth-service-account=SERVICE_ACCOUNT_EMAIL]
        [--push-auth-token-audience=OPTIONAL_AUDIENCE_OVERRIDE]
        [--push-endpoint=PUSH_ENDPOINT] [--retain-acked-messages]
        [--update-labels=[KEY=VALUE,...]]
        [--clear-dead-letter-policy
          | --max-delivery-attempts=MAX_DELIVERY_ATTEMPTS
          [--dead-letter-topic=DEAD_LETTER_TOPIC
          : --dead-letter-topic-project=DEAD_LETTER_TOPIC_PROJECT]]
        [--clear-labels | --remove-labels=[KEY,...]] [GCLOUD_WIDE_FLAG ...]

[shaxbee]

@danawillow submitted PR upstream based on discovery endpoint for v1. Using this feature heavily :-) I've tested it against our GCP project and it seems working.

[maroux]

ah thanks @shaxbee I missed that.

[nareshgnT]

Documentation should be updated with DLQ details, currently documentation does not updated with DLQ details

[slevenick]

@nareshgnT I see the documentation here: https://www.terraform.io/docs/providers/google/r/pubsub_subscription.html

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. If you feel I made an error 🤖 🙉 , please reach out to my human friends 👉 [email protected]. Thanks!