Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use of Unique IDs breaks resource.name IAM condition expressions #14419

Open
rifelpet opened this issue Apr 25, 2023 · 12 comments
Open

Use of Unique IDs breaks resource.name IAM condition expressions #14419

rifelpet opened this issue Apr 25, 2023 · 12 comments

Comments

@rifelpet
Copy link

rifelpet commented Apr 25, 2023

Community Note

  • Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request.
  • Please do not leave +1 or me too comments, they generate extra noise for issue followers and do not help prioritize the request.
  • If you are interested in working on this issue or have submitted a pull request, please leave a comment.
  • If an issue is assigned to the modular-magician user, it is either in the process of being autogenerated, or is planned to be autogenerated soon. If an issue is assigned to a user, that user is claiming responsibility for the issue. If an issue is assigned to hashibot, a community member has claimed the issue already.

Terraform Version

Terraform v1.4.5
on darwin_arm64
+ provider registry.terraform.io/hashicorp/google v4.61.0
+ provider registry.terraform.io/hashicorp/google-beta v4.61.0

Affected Resource(s)

  • google_compute_instance_template

Terraform Configuration Files

Copied from the documentation example:

resource "google_compute_instance_template" "default" {
  name        = "appserver-template"
  description = "This template is used to create app server instances."

  tags = ["foo", "bar"]

  labels = {
    environment = "dev"
  }

  instance_description = "description assigned to instances"
  machine_type         = "e2-medium"
  can_ip_forward       = false

  scheduling {
    automatic_restart   = true
    on_host_maintenance = "MIGRATE"
  }

  // Create a new boot disk from an image
  disk {
    source_image      = "debian-cloud/debian-11"
    auto_delete       = true
    boot              = true
  }

  network_interface {
    network = "default"
  }

  metadata = {
    foo = "bar"
  }
}

Debug Output

https://gist.github.com/rifelpet/993436c77c6b3e11df4a3373e6e50515

Panic Output

Expected Behavior

The plan succeeds.

Actual Behavior

The plan fails:

│ Error: Error when reading or editing Instance Template "appserver-template": googleapi: Error 403: Required 'compute.instanceTemplates.get' permission for 'projects/REDACTED/global/instanceTemplates/6773357995241391260', forbidden
│ 
│   with google_compute_instance_template.default,
│   on example.tf line 17, in resource "google_compute_instance_template" "default":
│   17: resource "google_compute_instance_template" "default" {

Steps to Reproduce

  1. Setup a GCP Service Account with the following IAM permissions:

    resource "google_project_iam_custom_role" "compute-instancetemplates" {
      role_id     = "compute_instancetemplates"
      title       = "compute.instancetemplates"
      description = "compute.instancetemplates"
      permissions = [
        "compute.instanceTemplates.create",
        "compute.instanceTemplates.delete",
        "compute.instanceTemplates.get",
        "compute.instanceTemplates.getIamPolicy",
        "compute.instanceTemplates.list",
        "compute.instanceTemplates.setIamPolicy",
        "compute.instanceTemplates.useReadOnly",
      ]
    }
    resource "google_project_iam_member" "account-worker-compute-instancetemplates" {
      role    = google_project_iam_custom_role.compute-instancetemplates.id
      member  = "serviceAccount:SERVICE_ACCOUNT_HERE"
      condition {
        title       = "condition"
        description = "condition"
        expression = <<EOF
        resource.name.endsWith('appserver-template')
        EOF
      }
    }
  2. Run terraform plan using the service account's credentials

Important Factoids

This started happening after #14128. This changed terraform's read request from:

GET /compute/v1/projects/REDACTED/global/instanceTemplates/appserver-template?alt=json&prettyPrint=false HTTP/1.1

to

GET /compute/v1/projects/REDACTED/global/instanceTemplates/6773357995241391260?alt=json&prettyPrint=false HTTP/1.1

which now fails the resource.name.endsWith('appserver-template') IAM condition expression.

In the spirit of least priviledge I would like to continue using resource.name IAM condition expressions but having the provider use the Unique ID integer as the resource ID seems to prevent this.

References

b/308755809

@rifelpet rifelpet added the bug label Apr 25, 2023
@edwardmedia edwardmedia self-assigned this Apr 25, 2023
@edwardmedia
Copy link
Contributor

edwardmedia commented Apr 25, 2023

@rifelpet did you notice below error in the log? It seems the account itself has not been authenticated before hitting the error you listed.

2023-04-25T16:12:39.695Z [DEBUG] provider.terraform-provider-google_v4.63.0_x5: 2023/04/25 16:12:39 [INFO] Authenticating using DefaultClient...
2023-04-25T16:12:39.695Z [DEBUG] provider.terraform-provider-google_v4.63.0_x5: 2023/04/25 16:12:39 [INFO]   -- Scopes: [https://www.googleapis.com/auth/cloud-platform https://www.googleapis.com/auth/userinfo.email]
2023-04-25T16:12:39.695Z [DEBUG] provider.terraform-provider-google_v4.63.0_x5: 2023/04/25 16:12:39 [INFO] Authenticating using DefaultClient...
2023-04-25T16:12:39.695Z [DEBUG] provider.terraform-provider-google_v4.63.0_x5: 2023/04/25 16:12:39 [INFO]   -- Scopes: [https://www.googleapis.com/auth/cloud-platform https://www.googleapis.com/auth/userinfo.email]
2023-04-25T16:12:39.695Z [DEBUG] provider.terraform-provider-google_v4.63.0_x5: 2023/04/25 16:12:39 [DEBUG] Waiting for state to become: [success]
2023-04-25T16:12:39.727Z [DEBUG] provider.terraform-provider-google_v4.63.0_x5: 2023/04/25 16:12:39 [INFO] error retrieving userinfo for your provider credentials. have you enabled the 'https://www.googleapis.com/auth/userinfo.email' scope? error: error retrieving userinfo for your provider credentials. have you enabled the 'https://www.googleapis.com/auth/userinfo.email' scope? error: googleapi: got HTTP response code 401 with body: {
2023-04-25T16:12:39.727Z [DEBUG] provider.terraform-provider-google_v4.63.0_x5:   "error": "invalid_request",
2023-04-25T16:12:39.727Z [DEBUG] provider.terraform-provider-google_v4.63.0_x5:   "error_description": "Invalid Credentials"
2023-04-25T16:12:39.727Z [DEBUG] provider.terraform-provider-google_v4.63.0_x5: }

Using the config you provided (3 resources), I am not able to hit the error. Please detail the steps for reproducing the issues.

@rifelpet
Copy link
Author

rifelpet commented Apr 25, 2023

Hi @edwardmedia, thanks for the quick response.

Here is a gist with v4.59.0 (the last release before the PR I mentioned) which successfully creates and reads the launch template. The same authentication error is in these logs, leading me to believe the failure may be an attempt to find credentials from a variety of sources and a different source succeeds. In this case the credentials are provided via http://metadata.google.internal.

@edwardmedia
Copy link
Contributor

edwardmedia commented Apr 25, 2023

@rifelpet how did you upgrade your provider version from v4.59.0 to v4.63.0? Do you still have the log when the google_compute_instance_template was created or when the version was v4.59.0? If yes, can you check if 6773357995241391260 in the log?

I do see this id, 1362663345129631346, in the log. Can you run below command to upgrade the provider on this and then try if the issue is resolved?

terraform init --upgrade

@rifelpet
Copy link
Author

rifelpet commented Apr 25, 2023

Here is a full gist showing the following steps starting with an empty terraform state:

  1. terraform init and terraform apply with v4.59.0
  2. Update the version requirement to v4.63.0
  3. terraform init -upgrade and terraform apply. NOTE: this apply succeeds, I'm guessing because the id stored in the statefile from v4.59.0 still uses the resource name
  4. a second terraform apply with v4.63.0 fails, presumably because the first v4.63.0 apply updated the statefile to use the unique ID, so this apply's read operation uses the unique ID which fails due to permissions.

@rileykarson
Copy link
Collaborator

Using the config you provided (3 resources), I am not able to hit the error.

@edwardmedia Did you run the second terraform plan / can you post a successful plan as a gist? I suspect that this error can only be triggered on the second plan/apply, as the code in question will only trigger if a specific output-only field is present.

did you notice below error in the log? It seems the account itself has not been authenticated before hitting the error you listed.

the error retrieving userinfo for your provider credentials logging almost never means anything. With the right scopes / authentication methods the provider emits the user's Terraform runner to the logs to help debug cases where folks are running as a different account than they think.

@edwardmedia
Copy link
Contributor

edwardmedia commented Apr 25, 2023

@rileykarson here is a fresh log that follows the steps here. Did not hit an error on the subsequent plan/apply.

https://gist.github.com/edwardmedia/17484267f83f1c886767978ae890235e

@rileykarson
Copy link
Collaborator

Discussed offline- to repro this the instance template call would need to come from the second service account (who was granted the conditional permission) rather than from the general Terraform runner that set the IAM policy.

@edwardmedia
Copy link
Contributor

edwardmedia commented Apr 25, 2023

Changed the runner does lead to the error in the 2nd plan/apply. Below is the log

https://gist.github.com/edwardmedia/c27b81075d2fdbc54110f60c10d08ec3

@edwardmedia edwardmedia assigned melinath and unassigned edwardmedia Apr 25, 2023
@rifelpet
Copy link
Author

I think that log shows a failure to read google_project_iam_custom_role.compute-instancetemplates which makes sense if terraform is using credentials that only have the instance template permissions.

Its the reading of actual google_compute_instance_template resources that succeeded with v4.59.0 but now fails with v4.63.0 because the provider is no longer requesting the resource by its name (see the differing GET requests in the Important Factoids section)

@rileykarson
Copy link
Collaborator

Yeah, that log is a different error (we expect you to receive that one). The second service account can only make the instance template call.

@rifelpet
Copy link
Author

rifelpet commented Apr 25, 2023

It seems that #14128 's intent to mitigate TOCTOU attacks is fundamentally at odds with the current GCP API's handling of resource.name IAM condition expressions.

Unless GCP IAM begins supporting resource.name conditions with requests for unique IDs, would some sort of use_unique_id bool provider-level configuration field be acceptable? In my case, TOCTOU attacks aren't a concern for our security posture precisely because we utilize resource-based IAM condition expressions: untrusted tenants can't create resources with the same names and therefore can't replace others' resources. This would allow us to optionally revert back to the original behavior across all resource types.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

5 participants