feat(pubsub): Add support for use_table_schema feature in subscriptio…

…n (#9831) (#645)

* Remove wait in Pub/Sub Schema tests as server-side changes have been made so that this is not necessary

* Add documentation indicating that schema udpates generate revisions that may need to be manually deleted

* Remove default_from_api setting for Topic schemaSettings, which is never set with a default from the server

* Revert "Remove default_from_api setting for Topic schemaSettings, which is never set with a default from the server"

This reverts commit 6163e1669fc328b8433119efa6f953cd4bd972fc.

* feat: Add Pub/Sub Subscription support for use_table_schema

* feat: Add Pub/Sub Subscription support for use_table_schema

* feat: Add Pub/Sub Subscription support for use_table_schema

* Apply suggestions from code review



* Add example of use_table_schema

* Add update test for use_table_schema

* Fix formatting

---------


[upstream:20a9ffe2394214386a5e4739ef4f8c635c8289fa]

Signed-off-by: Modular Magician <[email protected]>

pubsub_subscription_push_bq_table_schema/backing_file.tf

@@ -0,0 +1,15 @@
+# This file has some scaffolding to make sure that names are unique and that
+# a region and zone are selected when you try to create your Terraform resources.
+
+locals {
+  name_suffix = "${random_pet.suffix.id}"
+}
+
+resource "random_pet" "suffix" {
+  length = 2
+}
+
+provider "google" {
+  region = "us-central1"
+  zone   = "us-central1-c"
+}

pubsub_subscription_push_bq_table_schema/main.tf

@@ -0,0 +1,51 @@
+resource "google_pubsub_topic" "example" {
+  name = "example-topic-${local.name_suffix}"
+}
+
+resource "google_pubsub_subscription" "example" {
+  name  = "example-subscription-${local.name_suffix}"
+  topic = google_pubsub_topic.example.id
+
+  bigquery_config {
+    table = "${google_bigquery_table.test.project}.${google_bigquery_table.test.dataset_id}.${google_bigquery_table.test.table_id}"
+    use_table_schema = true
+  }
+
+  depends_on = [google_project_iam_member.viewer, google_project_iam_member.editor]
+}
+
+data "google_project" "project" {
+}
+
+resource "google_project_iam_member" "viewer" {
+  project = data.google_project.project.project_id
+  role   = "roles/bigquery.metadataViewer"
+  member = "serviceAccount:service-${data.google_project.project.number}@gcp-sa-pubsub.iam.gserviceaccount.com"
+}
+
+resource "google_project_iam_member" "editor" {
+  project = data.google_project.project.project_id
+  role   = "roles/bigquery.dataEditor"
+  member = "serviceAccount:service-${data.google_project.project.number}@gcp-sa-pubsub.iam.gserviceaccount.com"
+}
+
+resource "google_bigquery_dataset" "test" {
+  dataset_id = "example_dataset-${local.name_suffix}"
+}
+
+resource "google_bigquery_table" "test" {
+  deletion_protection = false
+  table_id   = "example_table-${local.name_suffix}"
+  dataset_id = google_bigquery_dataset.test.dataset_id
+
+  schema = <<EOF
+[
+  {
+    "name": "data",
+    "type": "STRING",
+    "mode": "NULLABLE",
+    "description": "The data"
+  }
+]
+EOF
+}

pubsub_subscription_push_bq_table_schema/motd

@@ -0,0 +1,7 @@
+===
+
+These examples use real resources that will be billed to the
+Google Cloud Platform project you use - so make sure that you
+run "terraform destroy" before quitting!
+
+===

pubsub_subscription_push_bq_table_schema/tutorial.md

@@ -0,0 +1,79 @@
+# Pubsub Subscription Push Bq Table Schema - Terraform
+
+## Setup
+
+<walkthrough-author name="[email protected]" analyticsId="UA-125550242-1" tutorialName="pubsub_subscription_push_bq_table_schema" repositoryUrl="https://github.com/terraform-google-modules/docs-examples"></walkthrough-author>
+
+Welcome to Terraform in Google Cloud Shell! We need you to let us know what project you'd like to use with Terraform.
+
+<walkthrough-project-billing-setup></walkthrough-project-billing-setup>
+
+Terraform provisions real GCP resources, so anything you create in this session will be billed against this project.
+
+## Terraforming!
+
+Let's use {{project-id}} with Terraform! Click the Cloud Shell icon below to copy the command
+to your shell, and then run it from the shell by pressing Enter/Return. Terraform will pick up
+the project name from the environment variable.
+
+```bash
+export GOOGLE_CLOUD_PROJECT={{project-id}}
+```
+
+After that, let's get Terraform started. Run the following to pull in the providers.
+
+```bash
+terraform init
+```
+
+With the providers downloaded and a project set, you're ready to use Terraform. Go ahead!
+
+```bash
+terraform apply
+```
+
+Terraform will show you what it plans to do, and prompt you to accept. Type "yes" to accept the plan.
+
+```bash
+yes
+```
+
+
+## Post-Apply
+
+### Editing your config
+
+Now you've provisioned your resources in GCP! If you run a "plan", you should see no changes needed.
+
+```bash
+terraform plan
+```
+
+So let's make a change! Try editing a number, or appending a value to the name in the editor. Then,
+run a 'plan' again.
+
+```bash
+terraform plan
+```
+
+Afterwards you can run an apply, which implicitly does a plan and shows you the intended changes
+at the 'yes' prompt.
+
+```bash
+terraform apply
+```
+
+```bash
+yes
+```
+
+## Cleanup
+
+Run the following to remove the resources Terraform provisioned:
+
+```bash
+terraform destroy
+```
+```bash
+yes
+```