Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CD: integrate nomad with vault #36

Closed
3 tasks done
noahehall opened this issue Dec 29, 2022 · 0 comments
Closed
3 tasks done

CD: integrate nomad with vault #36

noahehall opened this issue Dec 29, 2022 · 0 comments

Comments

@noahehall
Copy link
Contributor

noahehall commented Dec 29, 2022

C

  • integrating nomad with vault will harden and obfuscate the app as the dev cycle transitions from dev to dev validation
  • it will also serve as validation for all other downstream envs where nomad is the foundation for CD
  • the stack input changes drastically from dev where most things are managable, publically visibile and accessible, into test where visibility is minimal, human interaction is limited and managemnet is automated

T

  • update placeholder policies and token roles required for nomad infra
  • validate periodic_infra token role for nomad server (no errors reported on server start)
  • validate batch_infra_short token role for nomad job tasks

A


issue 1: @see hashicorp/nomad#15629

  • its an edge issue: if we curl 8300 it works, check the deny rules
  • its not an edge issue, its a security issue, nomad doesnt set user agent when making request
    • http-request deny if { req.hdr(user-agent) -m len le 32 }
$     2022-12-31T06:53:10.877-0700 [WARN]  nomad.raft: heartbeat timeout reached, starting election: last-leader-addr= last-leader-id=
    2022-12-31T06:53:38.937-0700 [WARN]  nomad.vault: failed to contact Vault API: retry=30s
  error=
  | Error making API request.
  | 
  | URL: GET https://dev.nirv.ai:8200/ui/v1/sys/health?drsecondarycode=299&performancestandbycode=299&sealedcode=299&standbycode=299&uninitcode=299
  | Code: 403. Raw Message:
  | 
  | <html><body><h1>403 Forbidden</h1>
  | Request forbidden by administrative rules.
  | </body></html>
  

issue 2: validate self token

  • i copypasta the nomad policy, so debug then create an issue with nomad
  • user error: somethings wrong with how we're applying the policy to the role
    • we diverged from the docs by using two token role tokens instead of one token & one token role
      • periodic_infra token role > set this in nomad config:
      • batch_infra_short token role > this is for nomad tasks
    2022-12-31T08:18:11.918-0700 [ERROR] nomad.vault: failed to validate self token/role: retry=30s
  error=
  | 4 errors occurred:
  | \t* token must have one of the following capabilities ["update" "root"] on "auth/token/revoke-accessor"; has [deny]
  | \t* token must have one of the following capabilities ["read" "root"] on "auth/token/roles/periodic_infra"; has [deny]
  | \t* token must have one of the following capabilities ["update" "root"] on "auth/token/create/periodic_infra"; has [deny]
  | \t* failed to lookup role "periodic_infra": Error making API request.
  | 
  | URL: GET https://dev.nirv.ai:8300/v1/auth/token/roles/periodic_infra
  | Code: 403. Errors:
  | 
  | * 1 error occurred:
  | \t* permission denied
  | 
  | 
  | 
  
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: DEPLOYED
Development

No branches or pull requests

1 participant