vault provider does not use namespace creating child token #602

matteomazza91 · 2019-11-12T13:07:19Z

Terraform Version

Terraform v0.12.6

provider.vault v2.6.0

Terraform Configuration Files

main.tf

provider "vault" {
  version = "~> 2.6"
  address = "https://MY_VAULT_HOSTNAME/"
  namespace = "MY_NAMESPACE"
  alias   = "my-alias"

  auth_login {
    path = "auth/ldap/login/${var.login_username}"
    parameters = {
      password = var.login_password
    }
  }
}

module "my_test_module" {
  providers = {
    vault = "vault.my-alias"
  }
  source = "./modules/vault/"
}
output "vault_test" {
  value = module.my_test_module.vault_test
}

./modules/vault/main.tf

data "vault_generic_secret" "vault_secret_test" {
  path = "some/path"
}

output "vault_test" {
  value = data.vault_generic_secret.vault_secret_test.data
}

Debug Output

2019-11-12T12:47:48.990Z [DEBUG] plugin.terraform-provider-vault_v2.6.0_x4: -----------------------------------------------------
2019-11-12T12:47:49.398Z [DEBUG] plugin.terraform-provider-vault_v2.6.0_x4: 2019/11/12 12:47:49 [DEBUG] Vault API Request Details:
2019-11-12T12:47:49.398Z [DEBUG] plugin.terraform-provider-vault_v2.6.0_x4: ---[ REQUEST ]---------------------------------------
2019-11-12T12:47:49.398Z [DEBUG] plugin.terraform-provider-vault_v2.6.0_x4: POST /v1/auth/token/create HTTP/1.1
2019-11-12T12:47:49.398Z [DEBUG] plugin.terraform-provider-vault_v2.6.0_x4: Host: MY_VAULT_HOSTNAME
2019-11-12T12:47:49.398Z [DEBUG] plugin.terraform-provider-vault_v2.6.0_x4: User-Agent: Go-http-client/1.1
2019-11-12T12:47:49.398Z [DEBUG] plugin.terraform-provider-vault_v2.6.0_x4: Content-Length: 128
2019-11-12T12:47:49.398Z [DEBUG] plugin.terraform-provider-vault_v2.6.0_x4: X-Vault-Namespace: 
2019-11-12T12:47:49.398Z [DEBUG] plugin.terraform-provider-vault_v2.6.0_x4: X-Vault-Token: VAULT_TOKEN_CREATED_BY_LOGIN
2019-11-12T12:47:49.398Z [DEBUG] plugin.terraform-provider-vault_v2.6.0_x4: Accept-Encoding: gzip
2019-11-12T12:47:49.398Z [DEBUG] plugin.terraform-provider-vault_v2.6.0_x4: 
2019-11-12T12:47:49.398Z [DEBUG] plugin.terraform-provider-vault_v2.6.0_x4: {
2019-11-12T12:47:49.398Z [DEBUG] plugin.terraform-provider-vault_v2.6.0_x4:  "ttl": "1200s",
2019-11-12T12:47:49.398Z [DEBUG] plugin.terraform-provider-vault_v2.6.0_x4:  "explicit_max_ttl": "1200s",
2019-11-12T12:47:49.398Z [DEBUG] plugin.terraform-provider-vault_v2.6.0_x4:  "display_name": "terraform",
2019-11-12T12:47:49.398Z [DEBUG] plugin.terraform-provider-vault_v2.6.0_x4:  "num_uses": 0,
2019-11-12T12:47:49.398Z [DEBUG] plugin.terraform-provider-vault_v2.6.0_x4:  "renewable": false,
2019-11-12T12:47:49.398Z [DEBUG] plugin.terraform-provider-vault_v2.6.0_x4:  "type": "",
2019-11-12T12:47:49.398Z [DEBUG] plugin.terraform-provider-vault_v2.6.0_x4:  "entity_alias": ""
2019-11-12T12:47:49.398Z [DEBUG] plugin.terraform-provider-vault_v2.6.0_x4: }
2019-11-12T12:47:49.398Z [DEBUG] plugin.terraform-provider-vault_v2.6.0_x4: -----------------------------------------------------

[...]

2019-11-12T12:47:49.584Z [DEBUG] plugin.terraform-provider-vault_v2.6.0_x4: 2019/11/12 12:47:49 [DEBUG] Vault API Response Details:
2019-11-12T12:47:49.584Z [DEBUG] plugin.terraform-provider-vault_v2.6.0_x4: ---[ RESPONSE ]--------------------------------------
2019-11-12T12:47:49.584Z [DEBUG] plugin.terraform-provider-vault_v2.6.0_x4: HTTP/1.1 403 Forbidden
2019-11-12T12:47:49.584Z [DEBUG] plugin.terraform-provider-vault_v2.6.0_x4: Content-Length: 60
2019-11-12T12:47:49.584Z [DEBUG] plugin.terraform-provider-vault_v2.6.0_x4: Cache-Control: no-store
2019-11-12T12:47:49.584Z [DEBUG] plugin.terraform-provider-vault_v2.6.0_x4: Content-Type: application/json
2019-11-12T12:47:49.584Z [DEBUG] plugin.terraform-provider-vault_v2.6.0_x4: Date: Tue, 12 Nov 2019 12:47:49 GMT
2019-11-12T12:47:49.584Z [DEBUG] plugin.terraform-provider-vault_v2.6.0_x4: Strict-Transport-Security: max-age=15638400; includeSubDomains
2019-11-12T12:47:49.584Z [DEBUG] plugin.terraform-provider-vault_v2.6.0_x4: 
2019-11-12T12:47:49.584Z [DEBUG] plugin.terraform-provider-vault_v2.6.0_x4: {
2019-11-12T12:47:49.584Z [DEBUG] plugin.terraform-provider-vault_v2.6.0_x4:  "errors": [
2019-11-12T12:47:49.584Z [DEBUG] plugin.terraform-provider-vault_v2.6.0_x4:   "1 error occurred:\n\t* permission denied\n\n"
2019-11-12T12:47:49.584Z [DEBUG] plugin.terraform-provider-vault_v2.6.0_x4:  ]
2019-11-12T12:47:49.584Z [DEBUG] plugin.terraform-provider-vault_v2.6.0_x4: }
2019-11-12T12:47:49.584Z [DEBUG] plugin.terraform-provider-vault_v2.6.0_x4: 
2019-11-12T12:47:49.584Z [DEBUG] plugin.terraform-provider-vault_v2.6.0_x4: -----------------------------------------------------
2019/11/12 12:47:49 [ERROR] <root>: eval: *terraform.EvalConfigProvider, err: failed to create limited child token: Error making API request.

Expected Behavior

set X-Vault-Namespace in /v1/auth/token/create call.

Actual Behavior

X-Vault-Namespace is not set

Steps to Reproduce

Please list the steps required to reproduce the issue, for example:

terraform apply

References

possibly related issue:

Cannot create child token without root priviliges #368

The text was updated successfully, but these errors were encountered:

Clete2 · 2020-02-12T18:09:56Z

Same issue here. Config:

provider "vault" {
  address   = "https://my.vault.domain:8200"
  namespace = "org/suborg/mynamespace"
  auth_login {
    path = "auth/k8s-test/login"

    parameters = {
      role      = "my-k8s-role"
      jwt       = var.jwt
    }
  }
}

Clete2 · 2020-02-12T18:24:50Z

Hey OP, the issue is that the Vault provider documentation is lacking.

You need to also put your namespace inside the auth_login block. We found this out by looking at the source code.

provider "vault" {
  address   = "https://my.vault.domain:8200"
  namespace = "org/suborg/mynamespace"
  auth_login {
    path      = "auth/k8s-test/login"
    namespace = "org/suborg/mynamespace"

    parameters = {
      role = "myrole"
      jwt  = var.jwt
    }
  }
}

matteomazza91 · 2020-05-05T12:16:00Z

thanks @Clete2 for your workaround. However it's not working for me.

That maybe works if the approle login-method is mounted in the specified workspace. However
in my scenario I use the ldap auth at root level (no namespace) to login.

m0ps · 2020-05-25T11:05:25Z

@matteomazza91 in 2.11.0 version headers configuration setting was added (#730). It allows adding additional HTTP headers to all requests. It's useful in such a case, with which you (and me) are faced.
I've switched from LDAP auth to token, and token obtained with auth via API (curl) during running terraform Jenkins job.

sacchettom · 2021-02-25T17:41:03Z

@m0ps trying it out in 2.18.0 and it doesn't work. I've tried specifying the header in the provider's configuration, but the one I see being used in trace logs is still empty. I can't seem to find the code where the call is actually manufactured, the interesting point is that the X-Vault-Namespace header is always present (even when not added to the provider's configuration) and is always empty on the request for the token.

browley86 · 2023-05-02T15:07:48Z

This is still an issue. This is particularly insidious because the original error message would serve no indication of what the problem is. I was having an issue with AppRole login and, because of the error message, I went down multiple different paths to try and solve the issue:

│ Error: Error making API request.
│
│ URL: PUT https://<vault url>/v1/auth/approle/login
│ Code: 400. Errors:
│
│ * invalid role ID
│
│   with provider["registry.terraform.io/hashicorp/vault"],
│   on provider.tf line 86, in provider "vault":
│   86: provider "vault" {

It was only I used TF_LOG=trace option that I noticed the issue that the X-Vault-Namespace was missing from the request header. This is particularly problematic because:

There are two different spots in the Provider code to specify a namespace and, in using both entries or one entry per section, the appropriate header still does not come through
The namespace functionality is part of the Enterprise variant of HashiCorp Vault which means people are paying for this to work which leads to...
Unless someone is the admin for the enterprise vault instance (I am not), there is no way to reproduce this without an Enterprise key

Number 3 there is particularly maddening because this worked perfectly the first time when I spun up a dev instance of vault but, because I'm not the admin of the enterprise vault, I cannot easily get a key for enterprise. Furthermore, specifying the headers param as suggested above will work for anything other than X-Vault-Namespace:

provider "vault" {
  address          = var.vault_address
  namespace        = var.vault_namespace
  skip_child_token = true
  headers {
    name  = "X-Vault-Namespace"
    value = var.vault_namespace
  }

  headers {
    name  = "BOGUS"
    value = "BOGUS"
  }

  auth_login {
    path = var.vault_auth_login_path

    parameters = {
      namespace = var.vault_namespace
      role_id   = var.vault_role_id
      secret_id = var.vault_secret_id
    }
  }

The trace logs show the Bogus Header but not the namespace header:

---[ REQUEST ]---------------------------------------
PUT /v1/auth/approle/login HTTP/1.1
Host: <vault addr>
User-Agent: Go-http-client/1.1
Content-Length: 125
Bogus: BOGUS
X-Vault-Request: true
X-Vault-Token: <vault token>
Accept-Encoding: gzip

So, clearly, there is a bug here. The proof is that the provider is, quite clearly, handling the X-Vault-Namespace header in some way but completely failing to use it when it actually goes to make the connection. For what it's worth, I tried just the headers alone as well as every permutation of the headers with the namespace param in each section to no avail.

benashz · 2023-05-02T15:24:20Z

Hi @browley86 , sorry to hear that you are having issues with the provider. I believe that #1830 will resolve this issue, since it will take the namespace from provider{}.namespace when authenticating with your auth_login config.

The provider handles the namespace internally, so even if the X-Vault-Namespace is set in headers{} the value will be overwritten from what is set on provider{}.namespace.

benashz · 2023-05-02T16:19:43Z

Fixed in #1830

browley86 · 2023-05-11T15:08:00Z

Hey @benashz - Looks like this was merged then rolled back. Looks like the current PR to fix it is #1841. Could we please re-open this until that's merged?

benashz mentioned this issue May 2, 2023

Ensure that the auth_login honours the provider's namespace #1830

Merged

benashz closed this as completed May 2, 2023

benashz reopened this May 17, 2023

benashz mentioned this issue May 17, 2023

Add back support for deriving the provider namespace from the Vault token's #1841

Merged

2 tasks

benashz closed this as completed in #1841 Jun 5, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

vault provider does not use namespace creating child token #602

vault provider does not use namespace creating child token #602

matteomazza91 commented Nov 12, 2019

Clete2 commented Feb 12, 2020

Clete2 commented Feb 12, 2020

matteomazza91 commented May 5, 2020

m0ps commented May 25, 2020

sacchettom commented Feb 25, 2021

browley86 commented May 2, 2023

benashz commented May 2, 2023

benashz commented May 2, 2023

browley86 commented May 11, 2023

vault provider does not use namespace creating child token #602

vault provider does not use namespace creating child token #602

Comments

matteomazza91 commented Nov 12, 2019

Terraform Version

Terraform Configuration Files

Debug Output

Expected Behavior

Actual Behavior

Steps to Reproduce

References

Clete2 commented Feb 12, 2020

Clete2 commented Feb 12, 2020

matteomazza91 commented May 5, 2020

m0ps commented May 25, 2020

sacchettom commented Feb 25, 2021

browley86 commented May 2, 2023

benashz commented May 2, 2023

benashz commented May 2, 2023

browley86 commented May 11, 2023