Skip to content

Backport of NET-4967: Fix helm install when setting copyAnnotations or nodeSelector for apiGateway into release/1.2.x#2673

Closed
hc-github-team-consul-core wants to merge 97 commits intorelease/1.2.xfrom
backport/multiline-args/uniformly-grand-snapper
Closed

Backport of NET-4967: Fix helm install when setting copyAnnotations or nodeSelector for apiGateway into release/1.2.x#2673
hc-github-team-consul-core wants to merge 97 commits intorelease/1.2.xfrom
backport/multiline-args/uniformly-grand-snapper

Conversation

@hc-github-team-consul-core
Copy link
Copy Markdown
Collaborator

Backport

This PR is auto-generated from #2597 to be assessed for backporting due to the inclusion of the label backport/1.2.x.

The below text is copied from the body of the original PR.


Changes proposed in this PR:
Modify Helm chart to properly support passing multiline strings for api-gateway nodeSelector and copyAnnotations

How I've tested this PR:
helm install with a values.yaml containing multiline values for the impacted args. The Job modified in this PR should correctly install and run as a post-install hook. You should also see the copyAnnotations and nodeSelector value make it onto the GatewayClassConfig in their entirety.

global:
  name: consul

  tls:
    enabled: true

connectInject:
  apiGateway:
    managedGatewayClass:
      nodeSelector: |
        kubernetes.io/os: linux
        kubernetes.io/is: windows
      copyAnnotations:
        service:
          annotations: |
            - service.beta.kubernetes.io/aws-load-balancer-scheme
            - service.beta.kubernetes.io/aws-load-balancer-name
➜  consul git:(multiline-args) helm upgrade --install consul ~/workspaces/consul-k8s/charts/consul --values ~/workspaces/lab/node-selector/values.yaml
Release "consul" has been upgraded. Happy Helming!
...

➜  consul git:(multiline-args) kubectl get gatewayclassconfig consul-api-gateway -o yaml
apiVersion: consul.hashicorp.com/v1alpha1
kind: GatewayClassConfig
...
spec:
  copyAnnotations:
    service:
    - service.beta.kubernetes.io/aws-load-balancer-scheme
    - service.beta.kubernetes.io/aws-load-balancer-name
  deployment:
    defaultInstances: 1
    maxInstances: 1
    minInstances: 1
  nodeSelector:
    kubernetes.io/is: windows
    kubernetes.io/os: linux
  serviceType: LoadBalancer

How I expect reviewers to test this PR:
See above

Checklist:


Overview of commits

curtbushko and others added 30 commits June 7, 2023 19:51
* Add FIPS builds for linux amd64

* add version check

* fix CI labels and add local dev commands

* fix ci version tagging

* switch to ubuntu 20.04

* add CLI version tag

* add gcompat for alpine glibc cgo compatibility

* remove FIPS version check from connect-init

* address comments
- making this trigger nightly until after 1.2.0 GA
- leaving 0.49.x active until after 1.2.0 GA
* first run through, needs help

* still need to make secure pass

* left something uncommented

* it works and also cleanup

* fix acceptance tests
* [API Gateway] Add acceptance test for cluster peering

* Fix linter

* Fix random unrelated linter errors to get CI to run: revert later?

* one more linter fix to later probably revert

* more linter fixes

* Revert "more linter fixes"

This reverts commit 6210dff.

* Revert "one more linter fix to later probably revert"

This reverts commit 030c563.

* Revert "Fix random unrelated linter errors to get CI to run: revert later?"

This reverts commit fdeccab.
…ersion of kind and k8s 1.27 (#2304)

* update cloud tests to use 1.24, 1.25 and 1.26 version of kubernetes for more coverage

* updated readme for supported kubernetes versions

* added changelog
* [API Gateway] WAN Federation test and fixes

* Fix unit tests
* Fix when gateways are deleted before we get services populated into cache

* a bit of cleanup
…assConfig are obeyed (#2272)

* Add unit tests verifying that scaling parameters on GatewayClassConfig are obeyed

* Add test case for scaling w/ no min or max configured
* Rename GatewayClassController to prevent name collision

* Use gateway instead of gatewayclass in name

* Use the constant in ownership checks

* Change GatewayClass name to "consul"

* Change GatewayClass name in cases

* Change ApiGatewayClass back
* Fix SupportedKinds array to be what Conformance test expects

* Fix cert validation status condition for listeners

* Add programmed condition for listeners

* Fix unit test

---------

Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com>
* first pass at halting: got httproute and api-gateway done

* clean up test

* Handle all set for infinite reconcile check

* Add table tests for minimal setup

* Added some odd field names to test normalization is handled correctly

* Use funky casing http routes
* Added helm inputs for managing audit logs
* Remove unwanted changes from values
* fix: use correct flag when translating namespaces

* Use non-normalized namespace when deregistering services

* Guard against namespace queries when namespaces not enabled in cache
* added imagePullPolicy for images in values.yaml

* fix: renamed pullPolicy key according to image

* fixed dafault always in tmpl

* changed structure of image in yaml

* revert changes

* added global imagePullPolicy

* fixed typo

* added changelog file
This brings consul-k8s in line with consul.
Most importantly, the backport assistant was updated to automatically assign created PRs to the author of the PR that is being backported.
* update changelog based on changes made to 1.2.x

* fixed test cases
- enterprise cases were in the OSS test cases
* trigger conformance tests nightly, squash

* remove extra line

* Update nightly-api-gateway-conformance.yml
making scripts more robust and removing changing helm chart
* Fix cache and service deletion issue

* Add comments

* add in acceptance test

* Fix indentation

* Fix unit test for deleting gateway w/ consul services

* Remove redundant service deregistration code

* Exit loop early once registration is found for service

* Fix import blocking

* Set status on pods added to test

* Apply suggestions from code review

* Reduce count of test gateways to 10 from 100

---------

Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com>
Co-authored-by: Sarah Alsmiller <sarah.alsmiller@hashicorp.com>
* Adding support for weighted k8s service

* Adding changelog

* if per-app weight is 0 then pull the weight to 1

* Addressing review comments

* Addressing review comments

* Addressing review comments

* Comment update

* Comment update

* Parameterized table test

* Parameterized table test

* fixing linting issue

* fixing linting issue

---------

Co-authored-by: srahul3 <rahulsharma@hashicorp.com>
* Bumping go-discover to the lastest version
Ashwin Venkatesh and others added 22 commits July 19, 2023 12:53
Changes proposed in this PR:
- Removed unused workflow inputs.
Changes proposed in this PR:
- Update actions that are out of date

How I've tested this PR:

👀 

How I expect reviewers to test this PR:

👀 


Checklist:
- [ ] Tests added
- [ ] [CHANGELOG entry
added](https://github.com/hashicorp/consul-k8s/blob/main/CONTRIBUTING.md#adding-a-changelog-entry)
Add guidance for proper configuration when joining to a secondary
cluster using WAN fed with external servers also enabled.

Also clarify federation requirements and fix formatting for an unrelated
value.

Changes proposed in this PR:
- Update base content for generating Helm chart docs to clarify the use
case encountered in #2138
- Minor additional fixes
- _Follow-up: propagate generated doc changes to `consul` and
additionally update
https://developer.hashicorp.com/consul/docs/k8s/deployment-configurations/servers-outside-kubernetes
there_

How I've tested this PR: N/A (docs only)

How I expect reviewers to test this PR: 👀 


Checklist:
- [ ] Tests added
- [ ] [CHANGELOG entry
added](https://github.com/hashicorp/consul-k8s/blob/main/CONTRIBUTING.md#adding-a-changelog-entry)
…og (#2571)

- In the past, kubernetes nodes were used as the source of truth to
determine the list of services that should exist in Consul.
- In most cases this was ok but becomes a problem when nodes are quickly
deleted from kubernetes such as the case when using spot instances.
- Instead, use consul synthetic-nodes to get the list of services and
deregister the services that do not have endpoint addresses.

---------
Co-authored-by: mr-miles <miles.waller@gmail.com>
…g Vault Version for WanFed Test (#2481)

* Adding support for Enterprise and other improvement on the Customizing Vault Version for WanFed Test
This is the extension of the PR -
#2043

In this PR, the followings were addressed -

1. Now the vault enterprise version can be provided in the cli command.  The previous PR only addressed Vault OSS.
2. Two flags “-no-cleanup-wan-fed” and “test-duration” were introduced to not to cleanup the test environment after successful setup to give it time to do manual testing for features/to reproduce customer issues.  Default is 1 hour.
3. This was tested in Kind environment and it works fine.  The following was taken out to use the “use-kind” option for WanFed test.

    //if cfg.UseKind {
    //  t.Skipf("Skipping this test because it's currently flaky on kind")
    //}

* Fix indentation

* Fix unit test for deleting gateway w/ consul services

* Remove redundant service deregistration code

* Exit loop early once registration is found for service

* Fix import blocking

* Set status on pods added to test

* Apply suggestions from code review

* Reduce count of test gateways to 10 from 100

---------

Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com>
Co-authored-by: Sarah Alsmiller <sarah.alsmiller@hashicorp.com>

Changes proposed in this PR:
-
-

How I've tested this PR:

How I expect reviewers to test this PR:

Checklist:
- [ ] Tests added
- [ ] CHANGELOG entry added
  > HashiCorp engineers only, community PRs should not add a changelog entry.
  > Entries should use present tense (e.g. Add support for...)

* Removing the changes in vault_namespaces_test.go

* Introducing new flag no-cleanup

* Removed "go 1.20" from go.work file

* cfg.USEKind check is added back

* Removed previousy added "Test Duration" flag

* Some changes

* Some changes
* added make target for checking for hashicorppreview

* added check to prepare-release make target
This is meant to solve for recurrent timeouts in several steps,
particularly `golangci-lint-control-plane` and `golang-ci-lint-cli`.

An accompanying change in `consul-k8s-workflows` should disable caching
until the (unclear) root of the issue can be resolved, or we can disable
or clear cache in a more targeted way that solves for these cases.
* Fix TestAPIGateway_GatewayClassConfig
* Remove stray files from bad merge
Support restricted PSA enforcement in a basic setup. This is enough to get a basic setup with ACLs and TLS working and an acceptance test passing (but does not update every component).

On OpenShift, we have the option to set the security context or not. If the security context is unset, then it is set automatically by OpenShift SCCs. However, we prefer to set the security context to avoid useless warnings on OpenShift and to reduce the config difference between OpenShift and plain Kube. By default, OpenShift namespaces have the audit and warn PSA labels set to restricted, so we receive pod security warnings when deploying Consul to OpenShift even though the pods will be able to run.

Helm chart changes:

* Add a helper to the helm chart to define a "restricted" container security context (when pod security policies are not enabled)
* Update the following container securityContexts to use the "restricted" settings (not exhaustive)

  - gateway-cleanup-job.yaml
  - gateway-resources-job.yaml
  - gossip-encryption-autogenerate-job.yaml
  - server-acl-init-cleanup-job.yaml - only if `.Values.server.containerSecurityContext.server.acl-init` is unset
  - server-acl-init-job.yaml - only if `.Values.server.containerSecurityContext.server.acl-init` is unset
  - server-statefulset.yaml:
     - the locality-init container receives the restricted context
     - the consul container receives the restricted context only if `.Values.server.containerSecurityContext.server` is unset
  - tls-init-cleanup-job.yaml - only if `.Values.server.containerSecurityContext.server.tls-init` is unset
  - tls-init-job.yaml - only if `.Values.server.containerSecurityContext.server.tls-init` is unset
  - webhook-cert-manager-deployment.yaml

Acceptance test changes:

* When `-enable-openshift` and `-enable-cni` are set, configure the CNI
  settings correctly for OpenShift.
* Add the `-enable-restricted-psa-enforcement` test flag. When this is set,
  the tests assume the Consul namespace has restricted PSA enforcement enabled.
  The tests will deploy the CNI (if enabled) into the `kube-system` namespace.
  Compatible test cases will deploy applications outside of the Consul namespace.
* Update the ConnectHelper to configure the NetworkAttachmentDefinition
  required to be compatible with the CNI on OpenShift.
* Add fixtures for static-client and static-server for OpenShift. This
  is necessary because the deployment configs must reference the network
  attachment definition when using the CNI on OpenShift.
* Update tests in the `acceptance/tests/connect` directory to either
  run or skip based on -enable-cni and -enable-openshift
security: Upgrade Go and net/http

Upgrade to Go 1.20.6 and `net/http` 1.12.0 to resolve CVE-2023-29406.
The consul client always logs into the local datacenter
* Add support for requestTimeout in Service Resolver spec
* preserve serviceresolvers.yaml
Preserving yaml from main, only adding requesttimeout property.
* update generated.deepcopy.go
* Use latest controller-gen to generate CRDs
---------

Co-authored-by: Ashwin Venkatesh <ashwin.what@gmail.com>
… ms (#2656)

increase timeout for acl replication to 60 seconds and poll every 500 ms
@hc-github-team-consul-core hc-github-team-consul-core force-pushed the backport/multiline-args/uniformly-grand-snapper branch from c5b0570 to 227c5d3 Compare July 26, 2023 22:23
@hc-github-team-consul-core hc-github-team-consul-core enabled auto-merge (squash) July 26, 2023 22:23
@hc-github-team-consul-core hc-github-team-consul-core force-pushed the backport/multiline-args/uniformly-grand-snapper branch from dcac0b1 to 96e2f17 Compare July 26, 2023 22:23
auto-merge was automatically disabled July 26, 2023 22:28

Pull request was closed

@nathancoleman
Copy link
Copy Markdown
Member

Re-triggering due to odd inclusion of so many commits

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.