Conversation
* Split out VerdictFormatStore effect Also did some related cleanup around orphan instances * make format * changelog * Cleanup imports Co-authored-by: Matthias Fischmann <mf@zerobuzz.net>
* Turn legalhold requests into polysemy action * Add runFederatedConcurrentlyEither This is useful in order to handle federation errors when making multiple concurrent requests. * Introduce `runFederatedConcurrentlyEither` Also remove usage of `Galley0` in Galley.API.Query. * Unify IntraM and FederationM into an App monad Also remove Galley0 completely. * Remove HasFederatorConfig instance of Galley * Add ZAuthLocalUser combinator to Servant routes This is to extract the ZUser header as a value of type `Local UserId`. The domain is extracted from the servant context. * Turn most of the ZUser in Galley into ZLocalUser * Remove MonadReader instance of Galley * Remove MonadLogger instance of Galley * Convert Aws.enqueue to a TeamStore action * Remove (almost) all explicit uses of IO * Define Galley as a synonym for Sem * Remove uses of liftSem * Replace Galley with Sem * Remove unnecessary uses of `Input (Local ())` * Remove use of MaybeT in Action * Make galley build without polysemy-plugin * Replace ClientState Reader effect with Input * Replace all Reader effects with Input * Remove redundant constraints * Rewrite CSV streaming using Final IO
The old version of prometheus operator chart is no longer compatible with latest K8s versions. The chart has also since moved to kube-prometheus-stack on the prometheus-community helm chart reposistory from the helm-stable repository. The kube-prometheus-stack helm chart also allows creating dashboard using labels on configmaps, so now we don't have to refer to non-pinned versions of the dashboard definitions. Instead each dashboard now becomes a configmap with a specific label and grafana discovers it. This commit also updates the dashboards to use new ways of addressing the pods which came with the move from old version of prometheus-operator chart to the kube-prometheus-stack chart. Along with these changes, the dashboards now also use the qualified endpoints and federation endpoints to display information about prekey claims and message sending stats.
* Handle Galley errors in federation RPCs Return errors as part of the response of the leave-conversation RPC. This commit also simplifies the response type considerably, by limiting constructors to one per possible error type. It also replaces error responses in the public API handler with `CanThrow`. * Test propagation of leave-conv RPC failures * Test failure cases of leave-conversation RPC
* Add documentation of the changelog.d process. * Reference changelog.d-docs in PR template. Co-authored-by: jschaul <jschaul@users.noreply.github.com>
* Upgrade polysemy * changelog * Use polysemy-mocks fix PR here: akshaymankar/polysemy-mocks#3 * Changes due to: cabal update && ./tools/convert-to-cabal/generate.sh * Use polysemy-mocks 0.2 * Retry CI * Hi CI Co-authored-by: Paolo Capriotti <paolo@capriotti.io> Co-authored-by: Marko Dimjašević <marko.dimjasevic@wire.com> Co-authored-by: Akshay Mankar <akshay@wire.com>
* Now and AssIDStore interpreters * Add AReqIDStore.Mem * add boolTTL helper * Add BindCookieStore.Mem * Add SAMLUserStore.Mem * Make format * Add DefaultSsoCode.Mem * Add ScimExternalIdStore.Mem * Add ScimTokenStore.Mem * Add ScimUserTimesStore.Mem * make format * Changelog * Pull out VerdictFormatStore * Remove stale comment * Also emit internal state * Hi CI
* Use minio helm chart from charts.min.io instead of helm.min.io * Add note about change in options for fake-aws-s3 * charts/fake-aws-s3: Dynamically figure out consoleAdmin creds * charts/fake-aws: Remove dependecy from minio.defaultBucket Assumes that first bucket is the one required by cargohold * charts/fake-aws-s3: Install minio in standalone mode * charts/fake-aws-s3: Lower the memory request for minio node * fake-aws-s3: Use minio-client to recreate any missing users/buckets
Co-authored-by: jschaul <jschaul@users.noreply.github.com>
This format is intended as an improvement over the JSON format. It tries to improve in these ways: - Explicitly encode log level as a key in the main JSON object logged - Encode field values as key-value pairs in the main JSON object Encoding logs like this makes it easy for external tools like fluent-bit and elasticsearch to understand the log level and the fields and index them as such without requiring bonanza to be present while processing logs.
* charts/kibana: Use helm chart from Elastic and auto-configure index-patterns * charts/elasticsearch-ephemeral: Bump default version to 6.8.18 This is required for oldest maintained kibana helm chart to work. * charts/fluent-bit: Use chart from Fluent and configure appropriately This chart is much more opaque than the unmaintained chart, so whole config has to be written as multi-line strings. Default for `inputs` and `filters` seems to be fine, but the `outputs` and `customParsers` need to be configured * Makefile: Also serve and publish kibana and fluent-bit charts
* Change federation API to be based on HTTP2
This is a preliminary commit that removes gRPC completely from
wire-api-federation, and assumes that federator is going to be accepting and
forwarding HTTP2 requests directly. Failures are going to be propagated using
normal HTTP error codes, and using Wai.Error values as responses.
The federation API is still following the same conventions as before, with the
exception that now we require that every endpoint consists of a single path
segment, which is the name of the RPC. For this reason, the `on-user-deleted`
endpoints in brig and galley have been updated to single-segment endpoints.
- Removed gRPC dependency.
- Removed explicit "/federation" prefix from federation APIs.
- `clientRoutes` is now a polymorphic value that works for both services.
- Brig and Galley API modules can now be imported without clashes.
- Federator client has been completely rewritten, and now implements HTTP2
client functionality on top of the http2 library. The low-level http2 client
functionality that establishes a connection supports both plaintext and TLS,
and will also be used to implement the outward service of federator.
- Federator client errors have been restructured and documented. They are
organised into three layers: low-level connection/TLS errors, federator client
errors, and high-level errors thrown by the application code.
- The origin domain header type in Servant is now a special combinator that
works just like a header on the server side, but is removed from the resulting
client type. Federator client knows to add the origin domain header on every
request, whether or not the origin domain is present in the API type.
* Rewrite federator services as HTTP applications
This is a re-implementation of the inward and outward federator services as
simple Wai applications, with the following interface:
* federation client / outward service path
/rpc/:domain/:component/:rpc
* inward ingress path
/federation/:component/:rpc
The origin domain is passed as a header directly in the form expected by the
federation API in brig and galley (i.e. a `Wire-Origin-Domain` header).
Server-to-server authentication is not affected, and the client certificate is
received by the inward service as a header, exacly as it was before.
- Removed gRPC dependency.
- Both the inward and outward services are now simple Wai applications that
serve a single route each, and forward the request appropriately, without
looking at the body.
- Removed error layer from the main application monad.
- Renamed `LookupError` to `DiscoveryFailure` for consistency.
- All polysemy error effects are now converted to Wai responses with a JSON
`Wai.Error` as body.
- Restored and simplified the federator mock server. Its functionality is
similar to before the change, but the gRPC type for federated requests has
been replaced by a simpler custom type `FederatedRequest`, which is used only
by the mock machinery itself.
- The internal Polysemy effects used in federator have been re-organised
slightly. For example, the `Remote` effect never fails by itself, and
`RemoteError` can be thrown by its interpretation, instead. Also, error
handling and logging is performed in a single place (`Federator.Response`) for
both services.
- The canonical `Remote` effect interpretation is now using the low-level
HTTP2 client functionality of Federator client to forward requests to a remote
federator.
- Validation has been simplified. In particular, path sanitisation has been
removed, since the RPC path is now a normal HTTP path, which is assumed to
have already been validated and sanitised by the HTTP server (and probably all
the intermediate proxies and load balancers).
- The federator client tests that were residing in wire-api-federation before
have now been moved to the federator package, since they depend on the mock
federator code.
* Brig: follow changes in federator
Adapted brig to the changes in federator and the corresponding mock
machinery. This is mostly about following type changes, and installing
the federation API at its correct "/federation" prefix, since the prefix
has now been removed from the Servant API types.
* Galley: follow changes in federator
Adapted galley to the changes in federator and the corresponding mock
machinery. This is mostly about following type changes, and installing
the federation API at its correct "/federation" prefix, since the prefix
has now been removed from the Servant API types.
* Replace gRPC with HTTP in ingress configuration
* Makefile improvements
* Add CHANGELOG entry
* Follow http2 upstream preface-race branch
* HTTP2 client: test parsing of response header
* Remove all remaining mentions of gRPC
- Delete obsolete python federation client
- Delete out-of-date federation documentation
- Change mentions of gRPC to HTTP2 where appropriate
* Restore federation-not-available label
* Turn all immediate federator errors into 403
* Map remote federator error responses
When the remote federator returns an error response, fail by returning a
533 error which is specific for this situation. This makes it possible
to distinguish local and remote federation failures, and makes sure that
a 5xx error is always returned for remote ones.
* Convert local 403 federator errors to 500
If the local federator returns 403, that indicates an issue in federator
itself, or in the communication between the service making a federated
call and the local federator. This should result in a 500 status code.
* remove commented-out code
* ExternalServer enforces RPC name format
Co-authored-by: Paolo Capriotti <paolo@capriotti.io>
Co-authored-by: Zebot <zebot@users.noreply.github.com>
Co-authored-by: Zebot <zebot@users.noreply.github.com>
* chore: [charts] Update team-settings version * Update tag to 4.3.0-v0.28.28-a2f11cf Co-authored-by: Zebot <zebot@users.noreply.github.com> Co-authored-by: Stefan Matting <stefan@wire.com>
julialongtin
approved these changes
Dec 2, 2021
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
7 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Release 2021-12-02
Release notes
Breaking change to the
fake-aws-s3(part offake-aws) helm chart. We now use minio helm chart from https://charts.min.io. The options are documented here (Use minio helm chart from charts.min.io instead of helm.min.io #1944)Before running the upgrade, the operators must use
kubectl edit deployment fake-aws-s3and explicitly setspec.template.spec.containers[0].serviceAccountandspec.template.spec.containers[0].serviceAccountNameto null. (Use minio helm chart from charts.min.io instead of helm.min.io #1944)Upgrade team-settings version to 4.3.0-v0.28.28-a2f11cf (Upgrade webapp/team-settings: changelog entries for #1835 and #1836 #1856)
Upgrade webapp version to 2021-12-02-production.0-v0.28.29-0-ec2fa00 (Update webapp version in Helm chart [skip ci] #1954)
Features
Bug fixes and other updates
Documentation
Internal changes
(Upgrade polysemy to v1.7.0.0 #1932)
Federation changes