Skip to content

Backport of [CC-5718] Remove HCP token requirement during bootstrap into release/1.15.x#18228

Closed
hc-github-team-consul-core wants to merge 659 commits intorelease/1.15.xfrom
backport/jer/ccm-read-only/unlikely-glad-snapper
Closed

Backport of [CC-5718] Remove HCP token requirement during bootstrap into release/1.15.x#18228
hc-github-team-consul-core wants to merge 659 commits intorelease/1.15.xfrom
backport/jer/ccm-read-only/unlikely-glad-snapper

Conversation

@hc-github-team-consul-core
Copy link
Collaborator

Backport

This PR is auto-generated from #18140 to be assessed for backporting due to the inclusion of the label backport/1.15.

The below text is copied from the body of the original PR.


Description

We are removing the requirement for HCP to provide Consul with a management token to support read-only tokens. This changes the validations to allow for a management token to be missing/ignored during bootstrapping with HCP.

Links

Ticket
RFC

PR Checklist

  • updated test coverage
  • external facing docs updated
  • appropriate backport labels added
  • not a security concern

Overview of commits

huikang and others added 30 commits May 15, 2023 09:33
This will aggregate all HealthStatus objects owned by the Node and update the status of the Node with an overall health.
#17317)

* fix(connect envoy): set initial_fetch_timeout to wait for initial xDS indefinitely

---------

Co-authored-by: Kiril Angov <kiril.angov@gmail.com>
Signed-off-by: Dan Bond <danbond@protonmail.com>
* Rename hcp-metrics-collector to consul-telemetry-collector

* Fix docs

* Fix doc comment

---------

Co-authored-by: Ashvitha Sridharan <ashvitha.sridharan@hashicorp.com>
* Add ACLs Enabled field to consul agent startup status message

* Add changelog

* Update startup messages to include default ACL policy configuration

* Correct import groupings
* reformatted IGW conf ref

* set up nav structure for IGW docs

* added main usage IGW usage doc

* added usage for serving custom tls certs

* updated internal links

* Update website/content/docs/connect/config-entries/ingress-gateway.mdx

* Apply suggestions from code review

Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com>

* changed filenames for IGW usage pages

* Apply suggestions from code review

Co-authored-by: Tu Nguyen <im2nguyen@users.noreply.github.com>

---------

Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com>
Co-authored-by: Tu Nguyen <im2nguyen@users.noreply.github.com>
…entries (#17145)

* service-resolve configuration entry reference

* Updates

* missing backtick

* service router configuration entry reference

* link fixes + tab fixes

* link and tab fixes

* link fixes

* service resolver improvements

* hierarchy fixes

* spacing

* links + formatting

* proofing fixes

* mmore fixes

* Apply suggestions from code review

suggestions from code review for service resolver

Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>

* policy sections edits

* service router code review

* Tables to sections - service router HCL

* YAML tables to sections

* formatting fixes

* converting tables to sections - service resolver

* final tables to sections

* Adjustments/alignments

* nanosecond fix

* Update website/content/docs/connect/config-entries/service-router.mdx

Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>

* link to filter example config

---------

Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>
* endpoints xds cluster configuration

* resources test fix

* fix reversion in resources_test

* Update agent/proxycfg/api_gateway.go

Co-authored-by: John Maguire <john.maguire@hashicorp.com>

* gofmt

* Modify getReadyUpstreams to filter upstreams by listener (#17410)

Each listener would previously have all upstreams from any route that bound to the listener. This is problematic when a route bound to one listener also binds to other listeners and so includes upstreams for multiple listeners. The list for a given listener would then wind up including upstreams for other listeners.

* Update agent/proxycfg/api_gateway.go

Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com>

* Restore import blocking

* Skip to next route if route has no upstreams

* cleanup

* change set from bool to empty struct

---------

Co-authored-by: John Maguire <john.maguire@hashicorp.com>
Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com>
* JWT Authentication with service intentions: update xds package to translate config to envoy
This change enables workflows where you are reapplying a resource that should have an owner ref to publish modifications to the resources data without performing a read to figure out the current owner resource incarnations UID.

Basically we want workflows similar to `kubectl apply` or `consul config write` to be able to work seamlessly even for owned resources.

In these cases the users intention is to have the resource owned by the “current” incarnation of the owner resource.
* endpoints xds cluster configuration

* clusters xds native generation

* resources test fix

* fix reversion in resources_test

* Update agent/proxycfg/api_gateway.go

Co-authored-by: John Maguire <john.maguire@hashicorp.com>

* gofmt

* Modify getReadyUpstreams to filter upstreams by listener (#17410)

Each listener would previously have all upstreams from any route that bound to the listener. This is problematic when a route bound to one listener also binds to other listeners and so includes upstreams for multiple listeners. The list for a given listener would then wind up including upstreams for other listeners.

* Update agent/proxycfg/api_gateway.go

Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com>

* Restore import blocking

* Undo removal of unrelated code

---------

Co-authored-by: John Maguire <john.maguire@hashicorp.com>
Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com>
* API Gateway XDS Primitives, endpoints and clusters (#17002)

* XDS primitive generation for endpoints and clusters

Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com>

* server_test

* deleted extra file

* add missing parents to test

---------

Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com>

* Routes for API Gateway (#17158)

* XDS primitive generation for endpoints and clusters

Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com>

* server_test

* deleted extra file

* add missing parents to test

* checkpoint

* delete extra file

* httproute flattening code

* linting issue

* so close on this, calling for tonight

* unit test passing

* add in header manip to virtual host

* upstream rebuild commented out

* Use consistent upstream name whether or not we're rebuilding

* Start working through route naming logic

* Fix typos in test descriptions

* Simplify route naming logic

* Simplify RebuildHTTPRouteUpstream

* Merge additional compiled discovery chains instead of overwriting

* Use correct chain for flattened route, clean up + add TODOs

* Remove empty conditional branch

* Restore previous variable declaration

Limit the scope of this PR

* Clean up, improve TODO

* add logging, clean up todos

* clean up function

---------

Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com>

* checkpoint, skeleton, tests not passing

* checkpoint

* endpoints xds cluster configuration

* resources test fix

* fix reversion in resources_test

* checkpoint

* Update agent/proxycfg/api_gateway.go

Co-authored-by: John Maguire <john.maguire@hashicorp.com>

* unit tests passing

* gofmt

* add deterministic sorting to appease the unit test gods

* remove panic

* Find ready upstream matching listener instead of first in list

* Clean up, improve TODO

* Modify getReadyUpstreams to filter upstreams by listener (#17410)

Each listener would previously have all upstreams from any route that bound to the listener. This is problematic when a route bound to one listener also binds to other listeners and so includes upstreams for multiple listeners. The list for a given listener would then wind up including upstreams for other listeners.

* clean up todos, references to api gateway in listeners_ingress

* merge in Nathan's fix

* Update agent/consul/discoverychain/gateway.go

* cleanup current todos, remove snapshot manipulation from generation code

* Update agent/structs/config_entry_gateways.go

Co-authored-by: Thomas Eckert <teckert@hashicorp.com>

* Update agent/consul/discoverychain/gateway.go

Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com>

* Update agent/consul/discoverychain/gateway.go

Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com>

* Update agent/proxycfg/snapshot.go

Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com>

* clarified header comment for FlattenHTTPRoute, changed RebuildHTTPRouteUpstream to BuildHTTPRouteUpstream

* simplify cert logic

* Delete scratch

* revert route related changes in listener PR

* Update agent/consul/discoverychain/gateway.go

* Update agent/proxycfg/snapshot.go

* clean up uneeded extra lines in endpoints

---------

Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com>
Co-authored-by: John Maguire <john.maguire@hashicorp.com>
Co-authored-by: Thomas Eckert <teckert@hashicorp.com>
To avoid unintended tampering with remote downstreams via service
config, refactor BasicEnvoyExtender and RuntimeConfig to disallow
typical Envoy extensions from being applied to non-local proxies.

Continue to allow this behavior for AWS Lambda and the read-only
Validate builtin extensions.

Addresses CVE-2023-2816.
* Only synthesize anonymous token in primary DC
* Add integration test for wan fed issue
* Integration test for permissive mTLS
@hc-github-team-consul-core hc-github-team-consul-core force-pushed the backport/jer/ccm-read-only/unlikely-glad-snapper branch from 0637e3e to 46f7677 Compare July 21, 2023 17:34
@hc-github-team-consul-core hc-github-team-consul-core force-pushed the backport/jer/ccm-read-only/unlikely-glad-snapper branch 2 times, most recently from 46f7677 to 1f60c7d Compare July 21, 2023 17:34
@hc-github-team-consul-core hc-github-team-consul-core enabled auto-merge (squash) July 21, 2023 17:34
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Auto approved Consul Bot automated PR

@github-actions github-actions bot added type/docs Documentation needs to be created/updated/clarified theme/api Relating to the HTTP API interface theme/cli Flags and documentation for the CLI interface theme/config Relating to Consul Agent configuration, including reloading theme/ui Anything related to the UI theme/connect Anything related to Consul Connect, Service Mesh, Side Car Proxies theme/tls Using TLS (Transport Layer Security) or mTLS (mutual TLS) to secure communication type/ci Relating to continuous integration (CI) tooling for testing or releases pr/dependencies PR specifically updates dependencies of project theme/envoy/xds Related to Envoy support theme/contributing Additions and enhancements to community contributing materials theme/internals Serf, Raft, SWIM, Lifeguard, Anti-Entropy, locking topics theme/certificates Related to creating, distributing, and rotating certificates in Consul theme/agent-cache Agent Cache theme/consul-terraform-sync Relating to Consul Terraform Sync and Network Infrastructure Automation labels Jul 21, 2023
@hc-github-team-consul-core
Copy link
Collaborator Author

🤔 This PR has changes in the website/ directory but does not have a type/docs-cherrypick label. If the changes are for the next version, this can be ignored. If they are updates to current docs, attach the label to auto cherrypick to the stable-website branch after merging.

@vercel vercel bot temporarily deployed to Preview – consul-ui-staging July 21, 2023 17:37 Inactive
@vercel vercel bot temporarily deployed to Preview – consul July 21, 2023 17:39 Inactive
auto-merge was automatically disabled July 26, 2023 16:15

Pull request was closed

@github-actions github-actions bot deleted the backport/jer/ccm-read-only/unlikely-glad-snapper branch August 29, 2025 06:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

pr/dependencies PR specifically updates dependencies of project theme/agent-cache Agent Cache theme/api Relating to the HTTP API interface theme/certificates Related to creating, distributing, and rotating certificates in Consul theme/cli Flags and documentation for the CLI interface theme/config Relating to Consul Agent configuration, including reloading theme/connect Anything related to Consul Connect, Service Mesh, Side Car Proxies theme/consul-terraform-sync Relating to Consul Terraform Sync and Network Infrastructure Automation theme/contributing Additions and enhancements to community contributing materials theme/envoy/xds Related to Envoy support theme/internals Serf, Raft, SWIM, Lifeguard, Anti-Entropy, locking topics theme/tls Using TLS (Transport Layer Security) or mTLS (mutual TLS) to secure communication theme/ui Anything related to the UI type/ci Relating to continuous integration (CI) tooling for testing or releases type/docs Documentation needs to be created/updated/clarified

Projects

None yet

Development

Successfully merging this pull request may close these issues.