Backport of Add TCP+TLS Healthchecks into release/1.16.x

.changelog/13023.txt

@@ -0,0 +1,3 @@
+```release-note:improvement
+ui: the topology view now properly displays services with mixed connect and non-connect instances.
+```

.changelog/17075.txt

@@ -0,0 +1,3 @@
+```release-note:improvement
+agent: remove agent cache dependency from service mesh leaf certificate management
+```

.changelog/17160.txt

@@ -0,0 +1,3 @@
+```release-note:bug
+Fix a bug that wrongly trims domains when there is an overlap with DC name.
+```

.changelog/17483.txt

@@ -0,0 +1,3 @@
+```release-note:bug
+peering: Fix a bug that caused server agents to continue cleaning up peering resources even after loss of leadership.
+```

.changelog/17546.txt

@@ -0,0 +1,3 @@
+```release-note:improvement
+connect: update supported envoy versions to 1.23.10, 1.24.8, 1.25.7, 1.26.2
+```

.changelog/17565.txt

@@ -0,0 +1,3 @@
+```release-note:feature
+reloadable config: Made enable_debug config reloadable and enable pprof command to work when config toggles to true
+```

.changelog/17582.txt

@@ -0,0 +1,3 @@
+```release-note:feature
+cli: `consul operator raft list-peers` command shows the number of commits each follower is trailing the leader by to aid in troubleshooting.
+```

.changelog/17596.txt

@@ -0,0 +1,3 @@
+```release-note:improvement
+ debug: change default setting of consul debug command. now default duration is 5ms and default log level is 'TRACE'
+ ```

.changelog/17609.txt

@@ -0,0 +1,4 @@
+```release-note:bug
+gateways: Fixed a bug in API gateways where binding a route that only targets a service imported from a peer results
+in the programmed gateway having no routes.
+```

.changelog/17631.txt

@@ -0,0 +1,3 @@
+```release-note:bug
+gateways: Fixed a bug where API gateways were not being taken into account in determining xDS rate limits.
+```

.changelog/17719.txt

@@ -0,0 +1,3 @@
+```release-note:security
+Bump Dockerfile base image to `alpine:3.18`. 
+ ```

.changelog/17739.txt

@@ -0,0 +1,3 @@
+```release-note:bug
+http: fixed API endpoint `PUT /acl/token/:AccessorID` (update token), no longer requires `AccessorID` in the request body. Web UI can now update tokens.
+ ```

.changelog/17754.txt

@@ -0,0 +1,3 @@
+```release-note:feature
+ui: consul version is displayed in nodes list with filtering and sorting based on versions
+```

.changelog/17755.txt

@@ -0,0 +1,3 @@
+```release-note:improvement
+mesh: Stop jwt providers referenced by intentions from being deleted.
+```

.changelog/17757.txt

@@ -0,0 +1,3 @@
+```release-note:improvement
+connect: Improve transparent proxy support for virtual services and failovers.
+```

.changelog/17759.txt

@@ -0,0 +1,3 @@
+```release-note:improvement
+extensions: Improve validation and error feedback for `property-override` builtin Envoy extension
+```

.changelog/17775.txt

@@ -0,0 +1,3 @@
+```release-note:bug
+connect: Fix issue where changes to service exports were not reflected in proxies.
+```

.changelog/17780.txt

@@ -0,0 +1,3 @@
+```release-note:feature
+cli: `consul watch` command uses `-filter` expression to filter response from checks, services, nodes, and service.
+```

.changelog/17846.txt

@@ -0,0 +1,3 @@
+```release-note:bug
+connect/ca: Fixes a bug preventing CA configuration updates in secondary datacenters
+```

.changelog/17885.txt

@@ -0,0 +1,2 @@
+```release-note:bug
+ca: Fixed a bug where the Vault provider was not passing the configured role param for AWS auth

.changelog/17888.txt

@@ -0,0 +1,3 @@
+```release-note:improvement
+connect: Add capture group labels from Envoy cluster FQDNs to Envoy exported metric labels
+```

.changelog/17894.txt

@@ -0,0 +1,3 @@
+```release-note:bug
+connect: Fix incorrect protocol config merging for transparent proxy implicit upstreams.
+```

.changelog/17911.txt

@@ -0,0 +1,4 @@
+```release-note:bug
+gateway: Fixes a bug where envoy would silently reject RSA keys that are smaller than 2048 bits,
+we now reject those earlier in the process when we validate the certificate.
+```

.changelog/17939.txt

@@ -0,0 +1,4 @@
+```release-note:improvement
+http: GET API `operator/usage` endpoint now returns node count
+cli: `consul operator usage` command now returns node count
+```

.changelog/17978.txt

@@ -0,0 +1,3 @@
+```release-note:improvement
+mesh: Expose remote jwks cluster configuration through jwt-provider config entry
+```

.changelog/18011.txt

@@ -0,0 +1,4 @@
+```release-note:bug
+connect: Removes the default health check from the `consul connect envoy` command when starting an API Gateway.
+This health check would always fail.
+```

.changelog/18024.txt

@@ -0,0 +1,3 @@
+```release-note:bug
+connect: fix a bug with Envoy potentially starting with incomplete configuration by not waiting enough for initial xDS configuration.
+```

.changelog/18068.txt

@@ -0,0 +1,3 @@
+```release-note:bug
+xds: Prevent partial application of non-Required Envoy extensions in the case of failure.
+```

.changelog/18080.txt

@@ -0,0 +1,3 @@
+```release-note:improvement
+Fix some typos in metrics docs
+```

.changelog/18112.txt

@@ -0,0 +1,3 @@
+```release-note:bug
+ca: Fixes a Vault CA provider bug where updating RootPKIPath but not IntermediatePKIPath would not renew leaf signing certificates
+```

.changelog/18140.txt

@@ -0,0 +1,3 @@
+```release-note:improvement
+hcp: Removes requirement for HCP to provide a management token
+```

.changelog/18150.txt

@@ -0,0 +1,3 @@
+```release-note:improvement
+xds: Explicitly enable WebSocket connection upgrades in HTTP connection manager
+```

.changelog/18168.txt

@@ -0,0 +1,3 @@
+```release-note:improvement
+hcp: Add dynamic configuration support for the export of server metrics to HCP.
+```

.changelog/18184.txt

@@ -0,0 +1,3 @@
+```release-note:bug
+api: Fix client deserialization errors by marking new Enterprise-only prepared query fields as omit empty
+```

.changelog/18186.txt

@@ -0,0 +1,3 @@
+```release-note:security
+Upgrade golang.org/x/net to address [CVE-2023-29406](https://nvd.nist.gov/vuln/detail/CVE-2023-29406)
+```

.changelog/18190.txt

@@ -0,0 +1,5 @@
+```release-note:security
+Upgrade to use Go 1.20.6.
+This resolves [CVE-2023-29406](https://github.com/advisories/GHSA-f8f7-69v5-w4vx)(`net/http`) for uses of the standard library. 
+A separate change updates dependencies on `golang.org/x/net` to use `0.12.0`.
+```

.changelog/18223.txt

@@ -0,0 +1,3 @@
+```release-note:feature
+cli: `consul members` command uses `-filter` expression to filter members based on bexpr.
+```

.changelog/18291.txt

@@ -0,0 +1,3 @@
+```release-note:bug
+api-gateway: fix race condition in proxy config generation when Consul is notified of the bound-api-gateway config entry before it is notified of the api-gateway config entry.
+```

.changelog/18302.txt

@@ -0,0 +1,4 @@
+```release-note:bug
+snapshot: fix access denied and handle is invalid when we call snapshot save on windows - skip sync() for folders in windows in
+https://github.com/rboyer/safeio/pull/3
+```

.changelog/18303.txt

@@ -0,0 +1,3 @@
+```release-note:improvement
+connect: update supported envoy versions to 1.23.12, 1.24.10, 1.25.9, 1.26.4
+```

.changelog/18319.txt

@@ -0,0 +1,6 @@
+```release-note:improvement
+acl: added builtin ACL policy that provides global read-only access (builtin/global-read-only)
+```
+```release-note:improvement
+acl: allow for a single slash character in policy names
+```

.changelog/18325.txt

@@ -0,0 +1,3 @@
+```release-note:bug
+mesh: **(Enterprise Only)** Require that `jwt-provider` config entries are created in the `default` namespace.
+```

.changelog/18358.txt

@@ -0,0 +1,7 @@
+```release-note:security
+Upgrade to use Go 1.20.7.
+This resolves vulnerability [CVE-2023-29409](https://nvd.nist.gov/vuln/detail/CVE-2023-29409)(`crypto/tls`).
+```
+```release-note:security
+Update `golang.org/x/net` to v0.13.0 to address [CVE-2023-3978](https://nvd.nist.gov/vuln/detail/CVE-2023-3978).
+```

.changelog/18437.txt

@@ -0,0 +1,3 @@
+```release-note:bug
+Inherit locality from services when registering sidecar proxies.
+```

.changelog/18464.txt

@@ -0,0 +1,3 @@
+```release-note:bug
+UI : Nodes list view was breaking for synthetic-nodes. Fix handles non existence of consul-version meta for node.
+```

.changelog/18558.txt

@@ -0,0 +1,3 @@
+```release-note:bug
+check: prevent go routine leakage when existing Defercheck of same check id is not nil
+```

.changelog/18584.txt

@@ -0,0 +1,3 @@
+```release-note:improvement
+Reduce the frequency of metric exports from Consul to HCP from every 10s to every 1m
+```

.changelog/18617.txt

@@ -0,0 +1,4 @@
+```release-note:improvement
+log: Currently consul logs files like this consul-{timestamp}.log. This change makes sure that there is always
+consul.log file with the latest logs in it.
+```

.changelog/18625.txt

@@ -0,0 +1,5 @@
+```release-note:improvement
+Adds flag -append-filename (which works on values version, dc, node and status) to consul snapshot save command.
+Adding the flag -append-filename version,dc,node,status will add consul version, consul datacenter, node name and leader/follower
+(status) in the file name given in the snapshot save command before the file extension.
+```

.changelog/18636.txt

@@ -0,0 +1,3 @@
+```release-note:bug
+connect: Fix issue where Envoy endpoints would not populate correctly after a snapshot restore.
+```

.changelog/_5517.txt

@@ -0,0 +1,3 @@
+```release-note:bug
+namespaces: **(Enterprise only)** fixes a bug where agent health checks stop syncing for all services on a node if the namespace of any service has been removed from the server.
+```

.changelog/_5614.txt

@@ -0,0 +1,4 @@
+```release-note:bug
+namespaces: **(Enterprise only)** fixes a bug where namespaces are stuck in a deferred deletion state indefinitely under some conditions.
+Also fixes the Consul query metadata present in the HTTP headers of the namespace read and list endpoints.
+```

.changelog/_5669.txt

@@ -0,0 +1,3 @@
+```release-note:improvement
+audit-logging: **(Enterprise only)** enable error response and request body logging
+```

.changelog/_5740.txt

@@ -0,0 +1,3 @@
+```release-note:feature
+api: (Enterprise only) Add `POST /v1/operator/audit-hash` endpoint to calculate the hash of the data used by the audit log hash function and salt.
+```

.changelog/_5750.txt

@@ -0,0 +1,3 @@
+```release-note:feature
+cli: (Enterprise only) Add a new `consul operator audit hash` command to retrieve and compare the hash of the data used by the audit log hash function and salt.
+```

.changelog/_5805.txt

@@ -0,0 +1,3 @@
+```release-note:security
+audit-logging: **(Enterprise only)** limit `v1/operator/audit-hash` endpoint to ACL token with `operator:read` privileges.
+```

.github/CODEOWNERS

@@ -8,3 +8,34 @@
 # release configuration
 /.release/                              @hashicorp/release-engineering @hashicorp/github-consul-core
 /.github/workflows/build.yml            @hashicorp/release-engineering @hashicorp/github-consul-core
+
+
+# Staff Engineer Review (protocol buffer definitions)
+/proto-public/   @hashicorp/consul-core-staff
+/proto/          @hashicorp/consul-core-staff
+
+# Staff Engineer Review (v1 architecture shared components)
+/agent/cache/                  @hashicorp/consul-core-staff
+/agent/consul/fsm/             @hashicorp/consul-core-staff
+/agent/consul/leader*.go       @hashicorp/consul-core-staff
+/agent/consul/server*.go       @hashicorp/consul-core-staff
+/agent/consul/state/           @hashicorp/consul-core-staff
+/agent/consul/stream/          @hashicorp/consul-core-staff
+/agent/submatview/             @hashicorp/consul-core-staff
+/agent/blockingquery/          @hashicorp/consul-core-staff
+
+# Staff Engineer Review (raft/autopilot)
+/agent/consul/autopilotevents/  @hashicorp/consul-core-staff
+/agent/consul/autopilot*.go     @hashicorp/consul-core-staff
+
+# Staff Engineer Review (v2 architecture shared components)
+/internal/controller/                   @hashicorp/consul-core-staff
+/internal/resource/                     @hashicorp/consul-core-staff
+/internal/storage/                      @hashicorp/consul-core-staff
+/agent/consul/controller/               @hashicorp/consul-core-staff
+/agent/grpc-external/services/resource/ @hashicorp/consul-core-staff
+
+# Staff Engineer Review (v1 security)
+/acl/                   @hashicorp/consul-core-staff
+/agent/xds/rbac*.go     @hashicorp/consul-core-staff
+/agent/xds/jwt*.go      @hashicorp/consul-core-staff

.github/scripts/filter_changed_files_go_test.sh

@@ -0,0 +1,33 @@
+#!/bin/bash
+
+# Get the list of changed files
+files_to_check=$(git diff --name-only origin/$GITHUB_BASE_REF)
+
+# Define the directories to check
+skipped_directories=("docs/" "ui/" "website/" "grafana/")
+
+# Initialize a variable to track directories outside the skipped ones
+other_directories=""
+trigger_ci=false
+
+# Loop through the changed files and find directories/files outside the skipped ones
+for file_to_check in $files_to_check; do
+	file_is_skipped=false
+	for dir in "${skipped_directories[@]}"; do
+		if [[ "$file_to_check" == "$dir"* ]] || [[ "$file_to_check" == *.md && "$dir" == *"/" ]]; then
+			file_is_skipped=true
+			break
+		fi
+	done
+	if [ "$file_is_skipped" = "false" ]; then
+		other_directories+="$(dirname "$file_to_check")\n"
+		trigger_ci=true
+		echo "Non doc file(s) changed - triggered ci: $trigger_ci"
+		echo -e $other_directories
+		echo "trigger-ci=$trigger_ci" >>"$GITHUB_OUTPUT"
+		exit 0 ## if file is outside of the skipped_directory exit script
+	fi
+done
+
+echo "Only doc file(s) changed - triggered ci: $trigger_ci"
+echo "trigger-ci=$trigger_ci" >>"$GITHUB_OUTPUT"

.github/scripts/get_runner_classes.sh

@@ -13,7 +13,7 @@ case "$GITHUB_REPOSITORY" in
         echo "compute-small=['self-hosted', 'linux', 'small']" >> "$GITHUB_OUTPUT"
         echo "compute-medium=['self-hosted', 'linux', 'medium']" >> "$GITHUB_OUTPUT"
         echo "compute-large=['self-hosted', 'linux', 'large']" >> "$GITHUB_OUTPUT"
-        # m5d.8xlarge is equivalent to our xl custom runner in OSS
+        # m5d.8xlarge is equivalent to our xl custom runner in CE
         echo "compute-xl=['self-hosted', 'ondemand', 'linux', 'type=m5d.8xlarge']" >> "$GITHUB_OUTPUT"
         ;;
     *)

.github/scripts/license_checker.sh

@@ -0,0 +1,16 @@
+#!/bin/bash
+# Copyright (c) HashiCorp, Inc.
+# SPDX-License-Identifier: BUSL-1.1
+
+
+busl_files=$(grep -r 'SPDX-License-Identifier: BUSL' . --exclude-dir .github)
+
+# If we do not find a file in .changelog/, we fail the check
+if [ -n "$busl_files" ]; then
+    echo "Found BUSL occurrences in the PR branch! (See NET-5258 for details)"
+    echo -n "$busl_files"
+    exit 1
+else
+    echo "Did not find any occurrences of BUSL in the PR branch"
+    exit 0
+fi

.github/workflows/backport-assistant.yml

@@ -40,4 +40,4 @@ jobs:
           curl -s -H "Authorization: token ${{ secrets.PR_COMMENT_TOKEN }}" \
             -X POST \
             -d "{ \"body\": \"${github_message}\"}" \
-            "https://api.github.com/repos/${GITHUB_REPOSITORY}/pull/${{ github.event.pull_request.number }}/comments"
+            "https://api.github.com/repos/${GITHUB_REPOSITORY}/issues/${{ github.event.pull_request.number }}/comments"

.github/workflows/build-artifacts.yml

@@ -80,7 +80,7 @@ jobs:
     - name: Set up Docker Buildx
       uses: docker/setup-buildx-action@f03ac48505955848960e80bbb68046aa35c7b9e7 # pin@v2.4.1
 
-    # NOTE: conditional specific logic as we store secrets in Vault in ENT and use GHA secrets in OSS.
+    # NOTE: conditional specific logic as we store secrets in Vault in ENT and use GHA secrets in CE.
     - name: Login to Docker Hub
       uses: docker/login-action@f4ef78c080cd8ba55a85445d5b36e214a81df20a # pin@v2.1.0
       with: