-
Notifications
You must be signed in to change notification settings - Fork 1.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Support BigQuery authorized routines #6680
Changes from 5 commits
eb62c88
b365c86
15892ac
cb66e9f
8fc5fa6
37db76f
bbbca23
07a79e8
e7bd83c
91656ad
6d032b8
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change | ||||||
---|---|---|---|---|---|---|---|---|
|
@@ -41,6 +41,14 @@ overrides: !ruby/object:Overrides::ResourceOverrides | |||||||
private: "private" | ||||||||
public: "public" | ||||||||
account_name: "bqowner" | ||||||||
- !ruby/object:Provider::Terraform::Examples | ||||||||
name: "bigquery_dataset_authorized_routine" | ||||||||
primary_resource_id: "dataset" | ||||||||
vars: | ||||||||
private: "private" | ||||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. These need a hyphen so that they automatically get prefixed with
Suggested change
|
||||||||
public: "public" | ||||||||
Tei1988 marked this conversation as resolved.
Show resolved
Hide resolved
|
||||||||
test_env_vars: | ||||||||
service_account: :SERVICE_ACCT | ||||||||
virtual_fields: | ||||||||
- !ruby/object:Api::Type::Boolean | ||||||||
name: 'delete_contents_on_destroy' | ||||||||
|
@@ -109,6 +117,13 @@ overrides: !ruby/object:Overrides::ResourceOverrides | |||||||
vars: | ||||||||
private: "private" | ||||||||
public: "public" | ||||||||
- !ruby/object:Provider::Terraform::Examples | ||||||||
name: "bigquery_dataset_access_authorized_routine" | ||||||||
skip_test: true # not importable | ||||||||
primary_resource_id: "access" | ||||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. In this example, the routine is actually set on
Suggested change
|
||||||||
vars: | ||||||||
private: "private" | ||||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. same here |
||||||||
public: "public" | ||||||||
properties: | ||||||||
datasetId: !ruby/object:Overrides::Terraform::PropertyOverride | ||||||||
ignore_read: true | ||||||||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,36 @@ | ||
resource "google_bigquery_dataset" "public" { | ||
dataset_id = "<%= ctx[:vars]['public'] %>" | ||
description = "This dataset is public" | ||
} | ||
|
||
resource "google_bigquery_routine" "public" { | ||
dataset_id = google_bigquery_dataset.public.dataset_id | ||
routine_id = "sample_routine" | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Is this automatically deleted if the dataset is deleted? And/or is there any issue with quotas for this resource, or name conflicts if parallel test runs happen at the same time? Otherwise it might be worth making this use a var as well. |
||
routine_type = "TABLE_VALUED_FUNCTION" | ||
language = "SQL" | ||
definition_body = <<-EOS | ||
SELECT 1 + value AS value | ||
EOS | ||
arguments { | ||
name = "value" | ||
argument_kind = "FIXED_TYPE" | ||
data_type = jsonencode({ "typeKind" = "INT64" }) | ||
} | ||
return_table_type = jsonencode({ "columns" = [ | ||
{ "name" = "value", "type" = { "typeKind" = "INT64" } }, | ||
] }) | ||
} | ||
|
||
resource "google_bigquery_dataset" "private" { | ||
dataset_id = "<%= ctx[:vars]['private'] %>" | ||
description = "This dataset is private" | ||
} | ||
|
||
resource "google_bigquery_dataset_access" "authorized_routine" { | ||
dataset_id = google_bigquery_dataset.private.dataset_id | ||
routine { | ||
project_id = google_bigquery_routine.public.project | ||
dataset_id = google_bigquery_routine.public.dataset_id | ||
routine_id = google_bigquery_routine.public.routine_id | ||
} | ||
} |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,50 @@ | ||
resource "google_bigquery_dataset" "public" { | ||
dataset_id = "<%= ctx[:vars]['public'] %>" | ||
description = "This dataset is public" | ||
} | ||
|
||
resource "google_bigquery_routine" "public" { | ||
dataset_id = google_bigquery_dataset.public.dataset_id | ||
routine_id = "sample_routine" | ||
routine_type = "TABLE_VALUED_FUNCTION" | ||
language = "SQL" | ||
definition_body = <<-EOS | ||
SELECT 1 + value AS value | ||
EOS | ||
arguments { | ||
name = "value" | ||
argument_kind = "FIXED_TYPE" | ||
data_type = jsonencode({ "typeKind" = "INT64" }) | ||
} | ||
return_table_type = jsonencode({ "columns" = [ | ||
{ "name" = "value", "type" = { "typeKind" = "INT64" } }, | ||
] }) | ||
} | ||
|
||
resource "google_bigquery_dataset" "private" { | ||
dataset_id = "<%= ctx[:vars]['private'] %>" | ||
description = "This dataset is private" | ||
access { | ||
role = "WRITER" | ||
specialGroup = "projectWriters" | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. this would need to be There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. @melinath On the other hands, I'm worried about the failed test. |
||
} | ||
access { | ||
role = "OWNER" | ||
specialGroup = "projectOwners" | ||
} | ||
access { | ||
role = "OWNER" | ||
user_by_email = "<%= ctx[:test_env_vars]['service_account'] %>" | ||
} | ||
access { | ||
role = "READER" | ||
specialGroup = "projectReaders" | ||
} | ||
access { | ||
routine { | ||
project_id = google_bigquery_routine.public.project | ||
dataset_id = google_bigquery_routine.public.dataset_id | ||
routine_id = google_bigquery_routine.public.routine_id | ||
} | ||
} | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This needs to match the second part of a
dataset
resource's terraform id. For example, for:the resource address is
google_bigquery_dataset.public
, so primary_resource_id would need to bepublic
.The "primary resource" will be imported as part of the test. In this case, the important thing is to make sure that imports work correctly for the
routine
field (which is on theprivate
dataset)