fix: Fix datasource acceptance tests setup part2 by sfc-gh-asawicki · Pull Request #3445 · snowflakedb/terraform-provider-snowflake

sfc-gh-asawicki · 2025-03-14T12:20:29Z

Continuation of #3443.

Use the pre-created database or create a database with the parameters set (part 2):

the pre-created database not changed yet (it still does not have the parameters)
going alphabetically, last is the grants datasource
generate model builders for grants datasource
remove file setups when unnecessary (do not migrate the ones with complex multiresource setups)
add missing testenvs.EnableAcceptance and acc.TestAccPreCheck(t)
do not add the builders for non-stable datasources
added a workaround for single attribute datasources
remove unused functions

Next PRs:

datasources continuation
resources
pre-created database changes

github-actions · 2025-03-14T14:41:17Z

Integration tests failure for de7c8bc738e06b150d39675dc175549eabe42686

github-actions · 2025-03-14T18:44:45Z

Integration tests failure for d304753e974f5f6445c494edac3a58c2e3d8a1d5

sfc-gh-jmichalak · 2025-03-17T10:59:00Z

 }

-func zeroFileFormats(databaseName string, schemaName string) string {
+func zeroFileFormats() string {


Nit: fix the name, as the contents of this function don't suggest this ensures zero matches.

Fixed in the follow-up PR

sfc-gh-jmichalak · 2025-03-17T11:00:59Z

    future {
      object_type_plural = "TABLES"
-      in_schema          = "\"${snowflake_database.test.name}\".\"${snowflake_schema.test.name}\""
+      in_schema          = "\"${var.database}\".\"${var.schema}\""


Nit: extract to a local variable (on L19 as well).

Fixed in the follow-up PR

sfc-gh-jmichalak · 2025-03-17T11:02:53Z

+
+	viewId := acc.TestClient().Ids.RandomSchemaObjectIdentifier()
+	statement := "SELECT ROLE_NAME FROM INFORMATION_SCHEMA.APPLICABLE_ROLES"
+	columnNames := []string{"ROLE_NAME"}


Nit: I think this is not needed in this case, right?

The statement has column, so I added this for completeness in the resource def too.

sfc-gh-jcieslak · 2025-03-17T11:09:02Z

+// Conversion to HCL using hcl v1 has problem with nested jsons where there is only one child value.
+// Current workaround used is to add the placeholder attribute which is later removed from the resulting HCL.
+func removeSingleAttributeWorkaroundLines(s string) (string, error) {
+	lineToRemoveRegex := regexp.MustCompile(fmt.Sprintf(`( +)"(.*)"( += +)"%s"\n`, SnowflakeProviderConfigSingleAttributeWorkaround))


nit: Could be moved out as a package private variable so that it's not recomputed every time it's called.

True. However, it's more readable if you do replacements. As these are workarounds used only in the tests code, I would leave them as is, WDYT?

That's fine. The local variable could be also placed right above this function for readability, but I'm fine with leaving it as is.

Continuation of #3443 and #3445. Use the pre-created database or create a database with the parameters set (part 3): - the pre-created database not changed yet (it still does not have the parameters) - going alphabetically, last is the security integrations datasource - generate model builders for NetworkPolicy, ScimSecurityIntegration, Saml2SecurityIntegration, ApiAuthenticationIntegrationWithAuthorizationCodeGrant, ApiAuthenticationIntegrationWithClientCredentials, and ExternalOauthSecurityIntegration resources - generate model builders for NetworkPolicies, ResourceMonitors, Schemas, Secrets, and SecurityIntegrations datasources - remove file setups when unnecessary (do not migrate the ones with complex multiresource setups) - add missing testenvs.EnableAcceptance and acc.TestAccPreCheck(t) - do not add the builders for non-stable datasources - add multiline overrides to OauthIntegrationForCustomClients resource model builder - address issues from the previous review - fix resource name for ExternalOauthSecurityIntegration Next PRs: - datasources continuation (part - last) - resources - pre-created database changes

Continuation of #3443, #3445, and #3448. Use the pre-created database or create a database with the parameters set (part 4): - the pre-created database not changed yet (it still does not have the parameters) - going alphabetically, datasources finished - generate model builders for Streamlit resource - generate model builders for Streamlits, Streams, Tags, Tasks, Users, Views, and Warehouses datasources - remove file setups when unnecessary (do not migrate the ones with complex multiresource setups) - add missing testenvs.EnableAcceptance and acc.TestAccPreCheck(t) - do not add the builders for non-stable datasources - address issues from the previous review - expect AAD_PROVISIONER role at test setup - change default connection helper constructor Next PRs: - resources - pre-created database changes

Continuation of #3443. Use the pre-created database or create a database with the parameters set (part 2): - the pre-created database not changed yet (it still does not have the parameters) - going alphabetically, last is the grants datasource - generate model builders for grants datasource - remove file setups when unnecessary (do not migrate the ones with complex multiresource setups) - add missing testenvs.EnableAcceptance and acc.TestAccPreCheck(t) - do not add the builders for non-stable datasources - added a workaround for single attribute datasources - remove unused functions Next PRs: - datasources continuation - resources - pre-created database changes

Continuation of #3443 and #3445. Use the pre-created database or create a database with the parameters set (part 3): - the pre-created database not changed yet (it still does not have the parameters) - going alphabetically, last is the security integrations datasource - generate model builders for NetworkPolicy, ScimSecurityIntegration, Saml2SecurityIntegration, ApiAuthenticationIntegrationWithAuthorizationCodeGrant, ApiAuthenticationIntegrationWithClientCredentials, and ExternalOauthSecurityIntegration resources - generate model builders for NetworkPolicies, ResourceMonitors, Schemas, Secrets, and SecurityIntegrations datasources - remove file setups when unnecessary (do not migrate the ones with complex multiresource setups) - add missing testenvs.EnableAcceptance and acc.TestAccPreCheck(t) - do not add the builders for non-stable datasources - add multiline overrides to OauthIntegrationForCustomClients resource model builder - address issues from the previous review - fix resource name for ExternalOauthSecurityIntegration Next PRs: - datasources continuation (part - last) - resources - pre-created database changes

Continuation of #3443, #3445, and #3448. Use the pre-created database or create a database with the parameters set (part 4): - the pre-created database not changed yet (it still does not have the parameters) - going alphabetically, datasources finished - generate model builders for Streamlit resource - generate model builders for Streamlits, Streams, Tags, Tasks, Users, Views, and Warehouses datasources - remove file setups when unnecessary (do not migrate the ones with complex multiresource setups) - add missing testenvs.EnableAcceptance and acc.TestAccPreCheck(t) - do not add the builders for non-stable datasources - address issues from the previous review - expect AAD_PROVISIONER role at test setup - change default connection helper constructor Next PRs: - resources - pre-created database changes

🤖 I have created a release *beep* *boop* --- ## [1.0.5](v1.0.4...v1.0.5) (2025-03-25) ### 🔧 **Misc** * Bump tracking version to v1.0.5 ([#3460](#3460)) ([2af79e6](2af79e6)) * Bump tracking version to v1.0.5 ([#3460](#3460)) ([3aaf555](3aaf555)) * Document godebug flag usage ([#3429](#3429)) ([5f04947](5f04947)) * Document godebug flag usage ([#3429](#3429)) ([2dce172](2dce172)) * fix failing tests ([#3428](#3428)) ([884ca54](884ca54)) * fix failing tests ([#3428](#3428)) ([245fd86](245fd86)) * Fix migration guide ([#3468](#3468)) ([6540841](6540841)) * GitHub actions cleanup ([#3431](#3431)) ([1cc6d80](1cc6d80)) * GitHub actions cleanup ([#3431](#3431)) ([a6e7429](a6e7429)) * Protect pentesting user and role from sweepers ([#3426](#3426)) ([1478c5e](1478c5e)) * Protect pentesting user and role from sweepers ([#3426](#3426)) ([6354dba](6354dba)) * Remove driver instrumentation ([#3439](#3439)) ([61802a8](61802a8)) * Remove driver instrumentation ([#3439](#3439)) ([e9fc6ed](e9fc6ed)) * Remove SF_TF_ADDITIONAL_DEBUG_LOGGING ([#3441](#3441)) ([cab63e9](cab63e9)) * Remove SF_TF_ADDITIONAL_DEBUG_LOGGING ([#3441](#3441)) ([b57db81](b57db81)) * Update GitHub actions ([#3455](#3455)) ([3b027a8](3b027a8)) * Update GitHub actions ([#3455](#3455)) ([368f090](368f090)) * Use environmental secrets in GitHub Actions ([#3440](#3440)) ([6048977](6048977)) * Use environmental secrets in GitHub Actions ([#3440](#3440)) ([8e5a262](8e5a262)) ### 🐛 **Bug fixes:** * Add boolean env validations and unit tests for TOML config validation ([#3453](#3453)) ([c113928](c113928)) * Add boolean env validations and unit tests for TOML config validation ([#3453](#3453)) ([408db4e](408db4e)) * Adjust docs ([#3451](#3451)) ([46d071d](46d071d)) * Adjust docs ([#3451](#3451)) ([29b0e4d](29b0e4d)) * Apply new assertions setup ([#3409](#3409)) ([e5fdced](e5fdced)) * Apply new assertions setup ([#3409](#3409)) ([af83da0](af83da0)) * Correct a typo in a tag resource example ([#3446](#3446)) ([a0202d1](a0202d1)) * Correct a typo in a tag resource example ([#3446](#3446)) ([afd19a2](afd19a2)) * Fix datasource acceptance tests setup part1 ([#3443](#3443)) ([58126ea](58126ea)) * Fix datasource acceptance tests setup part1 ([#3443](#3443)) ([5046654](5046654)) * Fix datasource acceptance tests setup part2 ([#3445](#3445)) ([6c7ceeb](6c7ceeb)) * Fix datasource acceptance tests setup part2 ([#3445](#3445)) ([4ba2bb6](4ba2bb6)) * Fix datasource acceptance tests setup part3 ([#3448](#3448)) ([5475c69](5475c69)) * Fix datasource acceptance tests setup part3 ([#3448](#3448)) ([f6fdc17](f6fdc17)) * Fix datasource acceptance tests setup part4 ([#3449](#3449)) ([04e082a](04e082a)) * Fix datasource acceptance tests setup part4 ([#3449](#3449)) ([935d969](935d969)) * Fix tasks tests ([#3434](#3434)) ([33ab740](33ab740)) * Fix tasks tests ([#3434](#3434)) ([b5457f6](b5457f6)) * Handle TOML file permissions ([#3444](#3444)) ([6c746b4](6c746b4)) * Handle TOML file permissions ([#3444](#3444)) ([e4ed171](e4ed171)) * Limit TOML file size ([#3432](#3432)) ([55fcbe3](55fcbe3)) * Limit TOML file size ([#3432](#3432)) ([2e12981](2e12981)) * Quick fix assertions ([#3438](#3438)) ([0e05c6f](0e05c6f)) * Quick fix assertions ([#3438](#3438)) ([a430ba7](a430ba7)) * Skip file permission verification by default ([#3476](#3476)) ([562173d](562173d)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please). Co-authored-by: snowflake-release-please[bot] <105954990+snowflake-release-please[bot]@users.noreply.github.com>

Continuation of #3443, #3445, #3448, and #3449. Use the pre-created database or create a database with the parameters set (resource - part1): - the pre-created database not changed yet (it still does not have the parameters) - going alphabetically through resources, last one was grant ownership - generate model builders for AccountRole resource - remove file setups when unnecessary (do not migrate the ones with complex multiresource setups) - remove database from setups where not needed - add missing testenvs.EnableAcceptance and acc.TestAccPreCheck(t) - do not add the builders for non-stable datasources - skip tests in the database resource and in external_oauth_integration_acceptance_test.go (follow-up needed for these to discuss and address all the nondeterministic tests) - remove tests for parameters that can be set on other level than account (in pkg/resources/account_parameter_acceptance_test.go) Next PRs: - resources continuation (1 or 2 more PRs) - pre-created database changes - follow-up for the problematic ones

Continuation of #3443, #3445, #3448, #3449, and #3450. Use the pre-created database or create a database with the parameters set (resource - part2): - the pre-created database not changed yet (it still does not have the parameters) - going alphabetically through resources, last one was grant privileges to database role - remove database from setups where not needed - add missing testenvs.EnableAcceptance and acc.TestAccPreCheck(t) - renamed database client method for database with parameters - added helper method to the database client helper Next PRs: - resources continuation (1 more PR est.) - pre-created database changes - follow-up for the problematic ones

Continuation of #3443, #3445, #3448, #3449, #3450, and #3480. Use the pre-created database or create a database with the parameters set (resource - part3): - the pre-created database not changed yet (it still does not have the parameters) - going alphabetically through resources, last one was oauth integration for partner applications - remove database from setups where not needed - add missing testenvs.EnableAcceptance and acc.TestAccPreCheck(t) - adjust tests after BCR 2024_08 general availability - secondary roles - defaults in users - managed account show changes - sync_password default change in scim - `TestAcc_User_handleChangesToDefaultSecondaryRoles` skipped for now (next PR) - `TestAcc_User_migrateFromVersion094_noDefaultSecondaryRolesSet` removed completely because of lack of way to set it up with current "ALL" default and 0.94.1 version of the provider not allowing to set it up through resource - some tests started to fail because of the secondary roles default on the CI user - changed it back to empty - adjust to changes in validation of network policy (`Error: 098519 (22023): Empty network policy [id] cannot be active.`) - adjust changes in validation for network rules addition (ENGRESS -> INGRESS changes in tests) - merge two upgrader helper files Next PRs: - resources continuation (1 more PR est.) - pre-created database changes - follow-up for the problematic ones

Continuation of #3443, #3445, #3448, #3449, #3450, #3480, and #3489. Use the pre-created database or create a database with the parameters set (resource - part4): - the pre-created database not changed yet (it still does not have the parameters) - going alphabetically through resources, last one was secondary database - remove database from setups where not needed - add missing testenvs.EnableAcceptance and acc.TestAccPreCheck(t) Next PRs: - resources continuation (1 more PR est.) - pre-created database changes - follow-up for the problematic ones

Continuation of #3443, #3445, #3448, #3449, #3450, #3480, #3489, and #3504. Use the pre-created database or create a database with the parameters set (resource - part, last): - the pre-created database not changed yet (it still does not have the parameters) - going alphabetically through resources, all completed - remove database from setups where not needed - add missing testenvs.EnableAcceptance and acc.TestAccPreCheck(t) - generate shared database resource config builder Next PRs: - pre-created database changes - fix a few failing tests from different runs - follow-up for the problematic ones

Continuation of #3443, #3445, #3448, #3449, #3450, #3480, #3489, #3504, and #3514. - change the pre-created database (set parameters) - fix a few failing tests from different runs - replace external providers setup with the existing function Next PRs: - follow-up for the problematic tests

sfc-gh-asawicki added 25 commits March 14, 2025 13:19

part 2 starts here

ec8c81f

Fix dynamic tables datasource acceptance test setup

1a6885d

Fix external functions datasource acceptance test setup

e880f6c

Fix external tables datasource acceptance test setup

f6cfd8a

Fix failover groups datasource acceptance test setup

1cbe723

Fix file formats datasource acceptance test setup

4597891

Fix functions datasource acceptance test setup

80140b1

Generate grants datasource config builder

e43c9bc

Add single value workaround

57770d9

Regenerate datasource model builders

0da69cb

Test grants in account model builder

fbe0dec

Remove unused func

92fa71d

Fix grants on account object test setup

bb6ab3d

Remove unused func

5334d51

Fix grants on database object acceptance test setup

7c550ba

Fix grants on schema object acceptance test setup

b2ba2e2

Fix grants on schema object with arguments acceptance test setup

9742472

Use builders for invalid grants on acceptance tests

65cfc17

Fix grants to database role acceptance test setup

cd484cd

Fix grants to user and share acceptance tests setup

3ca0bf0

Fix grants of database role acceptance tests setup

f114cd1

Fix grants of share acceptance tests setup

ab9c335

Fix grants future in acceptance tests setup

81c7cbe

Fix grants future to acceptance tests setup

8b1b1a9

Run make pre-push

de7c8bc

Fix unit tests

d304753

sfc-gh-asawicki marked this pull request as ready for review March 16, 2025 22:54

sfc-gh-asawicki requested a review from sfc-gh-jcieslak March 16, 2025 22:54

sfc-gh-asawicki requested a review from sfc-gh-jmichalak March 16, 2025 22:55

sfc-gh-jmichalak approved these changes Mar 17, 2025

View reviewed changes

sfc-gh-jcieslak approved these changes Mar 17, 2025

View reviewed changes

Merge branch 'dev' into fix-datasource-acceptance-tests-setup-part2

305f3c5

sfc-gh-asawicki had a problem deploying to test March 17, 2025 11:29 — with GitHub Actions Error

sfc-gh-asawicki merged commit 4ba2bb6 into dev Mar 17, 2025

sfc-gh-asawicki deleted the fix-datasource-acceptance-tests-setup-part2 branch March 17, 2025 11:31

This was referenced Mar 17, 2025

fix: Fix datasource acceptance tests setup part3 #3448

Merged

fix: Fix datasource acceptance tests setup part4 #3449

Merged

sfc-gh-asawicki mentioned this pull request Mar 18, 2025

fix: Fix resource acceptance tests setup part1 #3450

Merged

snowflake-release-please Bot mentioned this pull request Mar 24, 2025

chore(main): release 1.0.5 #3463

Merged

sfc-gh-asawicki mentioned this pull request Mar 26, 2025

fix: Fix resource acceptance tests setup part2 #3480

Merged

sfc-gh-asawicki mentioned this pull request Mar 27, 2025

fix: Fix resource acceptance tests setup part3 #3489

Merged

sfc-gh-asawicki mentioned this pull request Apr 1, 2025

fix: Fix resource acceptance tests setup part4 #3504

Merged

sfc-gh-asawicki mentioned this pull request Apr 4, 2025

fix: Fix resource acceptance tests setup part5 #3514

Merged

sfc-gh-asawicki mentioned this pull request Apr 6, 2025

fix: Change setup for precreated objects in acceptance tests #3515

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix: Fix datasource acceptance tests setup part2#3445

fix: Fix datasource acceptance tests setup part2#3445
sfc-gh-asawicki merged 27 commits into
devfrom
fix-datasource-acceptance-tests-setup-part2

sfc-gh-asawicki commented Mar 14, 2025 •

edited

Loading

Uh oh!

github-actions Bot commented Mar 14, 2025

Uh oh!

github-actions Bot commented Mar 14, 2025

Uh oh!

sfc-gh-jmichalak Mar 17, 2025

Uh oh!

sfc-gh-asawicki Mar 17, 2025

Uh oh!

sfc-gh-jmichalak Mar 17, 2025

Uh oh!

sfc-gh-asawicki Mar 17, 2025

Uh oh!

sfc-gh-jmichalak Mar 17, 2025

Uh oh!

sfc-gh-asawicki Mar 17, 2025

Uh oh!

sfc-gh-jcieslak Mar 17, 2025

Uh oh!

sfc-gh-asawicki Mar 17, 2025

Uh oh!

sfc-gh-jcieslak Mar 17, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

sfc-gh-asawicki commented Mar 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Mar 14, 2025

Uh oh!

github-actions Bot commented Mar 14, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

sfc-gh-asawicki commented Mar 14, 2025 •

edited

Loading