Add bootstrapped test networks for service networking tests (#5328)

* Use more markdown for Bug * Consistently use sentences for each bullet * Rewrite bug reproduction block * Allow domain mapping to succeed if DNS is pending Signed-off-by: Modular Magician <[email protected]> * Updated google_folder.html (#4149) * Updated google_folder.html The page in the first example shows that you should use organization_id with value of 1234567. In the Import example, it's not clear whether organization_id is user, or folder_id is used. API call behind this import command is only accepting folder_id (can be checked when setting TF_LOG to trace and viewing the API call) * Update website/docs/r/google_folder.html.markdown Co-Authored-By: Dana Hoffman <[email protected]> Co-authored-by: Dana Hoffman <[email protected]> * add google_kms_secret_ciphertext resource, deprecate datasource (#5314) Signed-off-by: Modular Magician <[email protected]> Co-authored-by: Dana Hoffman <[email protected]> * Allow add/removing Bigtable clusters (#5318) Signed-off-by: Modular Magician <[email protected]> Co-authored-by: Riley Karson <[email protected]> * Add bootstrapped test networks for service networking tests (#5316) Signed-off-by: Modular Magician <[email protected]> Co-authored-by: emily <[email protected]> * Add bootstrapped test networks for service networking tests (#2920) Merged PR #2920. Co-authored-by: Josh Soref <[email protected]> Co-authored-by: Chris Stephens <[email protected]> Co-authored-by: Petar Marinkovic <[email protected]> Co-authored-by: Dana Hoffman <[email protected]> Co-authored-by: megan07 <[email protected]> Co-authored-by: Riley Karson <[email protected]> Co-authored-by: emily <[email protected]>
hashicorp · Jan 7, 2020 · ca27045 · ca27045
1 parent fc38db6
commit ca27045
Show file tree

Hide file tree

Showing 14 changed files with 486 additions and 186 deletions.
diff --git a/.changelog/2920.txt b/.changelog/2920.txt
diff --git a/.github/ISSUE_TEMPLATE/bug.md b/.github/ISSUE_TEMPLATE/bug.md
@@ -9,10 +9,10 @@ about: For when something is there, but doesn't work how it should.
 
 ### Community Note
 
-* Please vote on this issue by adding a 👍 [reaction](https://blog.github.com/2016-03-10-add-reactions-to-pull-requests-issues-and-comments/) to the original issue to help the community and maintainers prioritize this request
-* Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
-* If you are interested in working on this issue or have submitted a pull request, please leave a comment
-* If an issue is assigned to the "modular-magician" user, it is either in the process of being autogenerated, or is planned to be autogenerated soon. If an issue is assigned to a user, that user is claiming responsibility for the issue. If an issue is assigned to "hashibot", a community member has claimed the issue already.
+* Please vote on this issue by adding a 👍 [reaction](https://blog.github.com/2016-03-10-add-reactions-to-pull-requests-issues-and-comments/) to the original issue to help the community and maintainers prioritize this request.
+* Please do not leave _+1_ or _me too_ comments, they generate extra noise for issue followers and do not help prioritize the request.
+* If you are interested in working on this issue or have submitted a pull request, please leave a comment.
+* If an issue is assigned to the `modular-magician` user, it is either in the process of being autogenerated, or is planned to be autogenerated soon. If an issue is assigned to a user, that user is claiming responsibility for the issue. If an issue is assigned to `hashibot`, a community member has claimed the issue already.
 
 <!--- Thank you for keeping this note for the community --->
 
@@ -31,12 +31,16 @@ about: For when something is there, but doesn't work how it should.
 <!--- Information about code formatting: https://help.github.com/articles/basic-writing-and-formatting-syntax/#quoting-code --->
 
 ```tf
-# Copy-paste your Terraform configurations here - for large Terraform configs,
-# please use a service like Dropbox and share a link to the ZIP file. For
-# security, you can also encrypt the files using our GPG public key: https://www.hashicorp.com/security
+# Copy-paste your Terraform configurations here.
+#
+# For large Terraform configs, please use a service like Dropbox and share a link to the ZIP file.
+# For security, you can also encrypt the files using our GPG public key:
+#    https://www.hashicorp.com/security
+#
 # If reproducing the bug involves modifying the config file (e.g., apply a config,
-# change a value, apply the config again, see the bug) then please include both the
-# version of the config before the change, and the version of the config after the change.
+# change a value, apply the config again, see the bug), then please include both:
+# * the version of the config before the change, and
+# * the version of the config after the change.
 ```
 
 ### Debug Output

diff --git a/google/bootstrap_utils_test.go b/google/bootstrap_utils_test.go
@@ -6,6 +6,7 @@ import (
 	"log"
 	"os"
 	"testing"
+	"time"
 
 	"google.golang.org/api/cloudkms/v1"
 	"google.golang.org/api/iam/v1"
@@ -230,3 +231,63 @@ func BootstrapServiceAccount(t *testing.T, project, testRunner string) string {
 
 	return sa.Email
 }
+
+const SharedTestNetworkPrefix = "tf-bootstrap-net-"
+
+// BootstrapSharedServiceNetworkingConsumerNetwork will return a shared compute network
+// for service networking test to prevent hitting limits on tenancy projects.
+//
+// This will either return an existing network or create one if it hasn't been created
+// in the project yet. One consumer network/tenant project we don't own is created
+// per producer network (i.e. network created by test), with a hard limit set.
+func BootstrapSharedServiceNetworkingConsumerNetwork(t *testing.T, testId string) string {
+	if v := os.Getenv("TF_ACC"); v == "" {
+		log.Println("Acceptance tests and bootstrapping skipped unless env 'TF_ACC' set")
+		// If not running acceptance tests, return an empty string
+		return ""
+	}
+
+	project := getTestProjectFromEnv()
+	networkName := SharedTestNetworkPrefix + testId
+	config := &Config{
+		Credentials: getTestCredsFromEnv(),
+		Project:     project,
+		Region:      getTestRegionFromEnv(),
+		Zone:        getTestZoneFromEnv(),
+	}
+	ConfigureBasePaths(config)
+	if err := config.LoadAndValidate(context.Background()); err != nil {
+		t.Errorf("Unable to bootstrap network: %s", err)
+	}
+
+	log.Printf("[DEBUG] Getting shared test network %q", networkName)
+	_, err := config.clientCompute.Networks.Get(project, networkName).Do()
+	if err != nil && isGoogleApiErrorWithCode(err, 404) {
+		log.Printf("[DEBUG] Network %q not found, bootstrapping", networkName)
+		url := fmt.Sprintf("%sprojects/%s/global/networks", config.ComputeBasePath, project)
+		netObj := map[string]interface{}{
+			"name":                  networkName,
+			"autoCreateSubnetworks": false,
+		}
+
+		res, err := sendRequestWithTimeout(config, "POST", project, url, netObj, 4*time.Minute)
+		if err != nil {
+			t.Fatalf("Error bootstrapping shared test network %q: %s", networkName, err)
+		}
+
+		log.Printf("[DEBUG] Waiting for network creation to finish")
+		err = computeOperationWaitTime(config, res, project, "Error bootstrapping shared test network", 4)
+		if err != nil {
+			t.Fatalf("Error bootstrapping shared test network %q: %s", networkName, err)
+		}
+	}
+
+	network, err := config.clientCompute.Networks.Get(project, networkName).Do()
+	if err != nil {
+		t.Errorf("Error getting shared test network %q: %s", networkName, err)
+	}
+	if network == nil {
+		t.Fatalf("Error getting shared test network %q: is nil", networkName)
+	}
+	return network.Name
+}
diff --git a/google/data_source_google_kms_secret_ciphertext.go b/google/data_source_google_kms_secret_ciphertext.go
@@ -13,7 +13,8 @@ import (
 
 func dataSourceGoogleKmsSecretCiphertext() *schema.Resource {
 	return &schema.Resource{
-		Read: dataSourceGoogleKmsSecretCiphertextRead,
+		DeprecationMessage: "Use the google_kms_secret_ciphertext resource instead.",
+		Read:               dataSourceGoogleKmsSecretCiphertextRead,
 		Schema: map[string]*schema.Schema{
 			"crypto_key": {
 				Type:     schema.TypeString,

diff --git a/google/data_source_google_kms_secret_ciphertext_test.go b/google/data_source_google_kms_secret_ciphertext_test.go
@@ -1,113 +1,41 @@
 package google
 
 import (
-	"encoding/base64"
 	"fmt"
-	"log"
 	"testing"
 
 	"github.com/hashicorp/terraform-plugin-sdk/helper/acctest"
 	"github.com/hashicorp/terraform-plugin-sdk/helper/resource"
 	"github.com/hashicorp/terraform-plugin-sdk/terraform"
-	"google.golang.org/api/cloudkms/v1"
 )
 
-func TestAccKmsSecretCiphertext_basic(t *testing.T) {
+func TestAccDataKmsSecretCiphertext_basic(t *testing.T) {
 	t.Parallel()
 
-	projectOrg := getTestOrgFromEnv(t)
-	projectBillingAccount := getTestBillingAccountFromEnv(t)
-
-	projectId := "terraform-" + acctest.RandString(10)
-	keyRingName := fmt.Sprintf("tf-test-%s", acctest.RandString(10))
-	cryptoKeyName := fmt.Sprintf("tf-test-%s", acctest.RandString(10))
+	kms := BootstrapKMSKey(t)
 
 	plaintext := fmt.Sprintf("secret-%s", acctest.RandString(10))
 
-	// The first test creates resources needed to encrypt plaintext and produce ciphertext
 	resource.Test(t, resource.TestCase{
 		PreCheck:  func() { testAccPreCheck(t) },
 		Providers: testAccProviders,
 		Steps: []resource.TestStep{
 			{
-				Config: testGoogleKmsCryptoKey_basic(projectId, projectOrg, projectBillingAccount, keyRingName, cryptoKeyName),
+				Config: testGoogleKmsSecretCiphertext_datasource(kms.CryptoKey.Name, plaintext),
 				Check: func(s *terraform.State) error {
-					cryptoKeyId, err := getCryptoKeyId(s, "google_kms_crypto_key.crypto_key")
+					plaintext, err := testAccDecryptSecretDataWithCryptoKey(s, kms.CryptoKey.Name, "data.google_kms_secret_ciphertext.acceptance")
 
 					if err != nil {
 						return err
 					}
 
-					// The second test asserts that the data source created a ciphertext that can be decrypted to the correct plaintext
-					resource.Test(t, resource.TestCase{
-						PreCheck:  func() { testAccPreCheck(t) },
-						Providers: testAccProviders,
-						Steps: []resource.TestStep{
-							{
-								Config: testGoogleKmsSecretCiphertext_datasource(cryptoKeyId.terraformId(), plaintext),
-								Check: func(s *terraform.State) error {
-									plaintext, err := testAccDecryptSecretDataWithCryptoKey(s, cryptoKeyId, "data.google_kms_secret_ciphertext.acceptance")
-
-									if err != nil {
-										return err
-									}
-
-									return resource.TestCheckResourceAttr("data.google_kms_secret_ciphertext.acceptance", "plaintext", plaintext)(s)
-								},
-							},
-						},
-					})
-
-					return nil
+					return resource.TestCheckResourceAttr("data.google_kms_secret_ciphertext.acceptance", "plaintext", plaintext)(s)
 				},
 			},
 		},
 	})
 }
 
-func getCryptoKeyId(s *terraform.State, cryptoKeyResourceName string) (*kmsCryptoKeyId, error) {
-	config := testAccProvider.Meta().(*Config)
-	rs, ok := s.RootModule().Resources[cryptoKeyResourceName]
-	if !ok {
-		return nil, fmt.Errorf("Resource not found: %s", cryptoKeyResourceName)
-	}
-
-	return parseKmsCryptoKeyId(rs.Primary.Attributes["id"], config)
-}
-
-func testAccDecryptSecretDataWithCryptoKey(s *terraform.State, cryptoKeyId *kmsCryptoKeyId, secretCiphertextResourceName string) (string, error) {
-	config := testAccProvider.Meta().(*Config)
-	rs, ok := s.RootModule().Resources[secretCiphertextResourceName]
-	if !ok {
-		return "", fmt.Errorf("Resource not found: %s", secretCiphertextResourceName)
-	}
-	ciphertext, ok := rs.Primary.Attributes["ciphertext"]
-	if !ok {
-		return "", fmt.Errorf("Attribute 'ciphertext' not found in resource '%s'", secretCiphertextResourceName)
-	}
-
-	kmsDecryptRequest := &cloudkms.DecryptRequest{
-		Ciphertext: ciphertext,
-	}
-
-	decryptResponse, err := config.clientKms.Projects.Locations.KeyRings.CryptoKeys.Decrypt(cryptoKeyId.cryptoKeyId(), kmsDecryptRequest).Do()
-
-	if err != nil {
-		return "", fmt.Errorf("Error decrypting ciphertext: %s", err)
-	}
-
-	plaintextBytes, err := base64.StdEncoding.DecodeString(decryptResponse.Plaintext)
-
-	if err != nil {
-		return "", err
-	}
-
-	plaintext := string(plaintextBytes)
-	log.Printf("[INFO] Successfully decrypted ciphertext and got plaintext: %s", plaintext)
-
-	return plaintext, nil
-}
-
 func testGoogleKmsSecretCiphertext_datasource(cryptoKeyTerraformId, plaintext string) string {
 	return fmt.Sprintf(`
 data "google_kms_secret_ciphertext" "acceptance" {

diff --git a/google/provider.go b/google/provider.go
@@ -477,9 +477,9 @@ func Provider() terraform.ResourceProvider {
 	return provider
 }
 
-// Generated resources: 95
+// Generated resources: 96
 // Generated IAM resources: 45
-// Total generated resources: 140
+// Total generated resources: 141
 func ResourceMap() map[string]*schema.Resource {
 	resourceMap, _ := ResourceMapWithErrors()
 	return resourceMap
@@ -597,6 +597,7 @@ func ResourceMapWithErrors() (map[string]*schema.Resource, error) {
 			"google_identity_platform_tenant":                              resourceIdentityPlatformTenant(),
 			"google_kms_key_ring":                                          resourceKMSKeyRing(),
 			"google_kms_crypto_key":                                        resourceKMSCryptoKey(),
+			"google_kms_secret_ciphertext":                                 resourceKMSSecretCiphertext(),
 			"google_logging_metric":                                        resourceLoggingMetric(),
 			"google_ml_engine_model":                                       resourceMLEngineModel(),
 			"google_monitoring_alert_policy":                               resourceMonitoringAlertPolicy(),