[feat] proxy implementation

[toddself]

Summary

This is an x-pack plugin to implement a request proxy system for kibana.

This allows parts of kibana to register as the responsible instance for a certain
resource and then allow other kibana instances to discover this information
and proxy requests for that resource back to the responsible instance.

This initial implementation is based around the needs for the code plugin, but
the idea is make this generic enough for use in other parts of the system.

There are a few pending issues:

• Blocked by [feat] create additional http servers #36804. was reverted in revert PR 36804 'Create additional HTTP servers' #42333
• Implement settings for
  • loop interval time
  • instance time out threshold
  • port to use for separate HTTP instance
• remove resources from resource map for instances that have failed liveness checks

Checklist

Use strikethroughs to remove checklist items you don't feel are applicable to this PR.

• This was checked for cross-browser compatibility, including a check against IE11
• Any text added follows EUI's writing guidelines, uses sentence case text and includes i18n support
• Documentation was added for features that require explanation or tutorials
• Unit or functional tests were updated or added to match the most common scenarios
• This was checked for keyboard-only and screenreader accessibility

For maintainers

• This was checked for breaking API changes and was labeled appropriately
• This includes a feature addition or change that requires a release note and was labeled appropriately

DevDoc

This PR implements an internal proxy mechanism for a collection of Kibana nodes living behind the same load balancer, with a shared mapping for resources stored in the backing Elasticsearch store.

The proxy allows a plugin developer to register resources with specific nodes, and then forward requests for those resources to the responsible nodes, so that the data may be returned to the user requesting it.

The end-user must provide SSL certificates and keys for the server and client to communicate via Mutual TLS authorization, securing any internal data from easily leaking out. If the SSL certificates are self-signed, the signing certificate authority will also need to be provided. Information on generating the keys, certificates and authorities may be found in the Elasticsearch encrypted communications guide. The proxy will throw an error on start if the TLS configuration has not been provided.

This introduces several new configuration options under the xpack.proxy path:

• updateInterval - the frequency (in ms) that the nodes should verify their availability
• timeoutThreshold - the maximum length of time (in ms) that a node can go without checking in
• port - the port number that the proxy server should be listening on
• maxRetry - the number of times the proxy should attempt to request the remote resource before giving up
• requestBackoff - how long the proxy should wait before re-requesting an unavailable resource

The following configuration options are the same as from server.ssl:

• cert - the TLS certificate to use
• key - the key for this TLS certificate
• ca - the certificate authority used to generate this certificate
• cipherSuites - the cipher suites to use
• supportedProtocols - the protocols supported by the proxy

When instantiated, proxy.setup() returns:

export interface ProxyServiceSetup {
  httpSetup: Omit<HttpServiceSetup, 'createNewServer'>;
}

And proxy.start() returns:

export interface ProxyServiceStart {
  assignResource: (
    resource: string,
    type: string,
    state: RouteState,
    node?: string
  ) => Promise<void>;
  unassignResource: (resource: string) => Promise<void>;
  proxyResource: (resource: string) => (req: KibanaRequest) => Promise<any>;
  proxyRequest: (req: KibanaRequest, resource: string) => Promise<any>;
  getAllocation: () =>Promise<RoutingTable>;
}

[elasticmachine]

Pinging @elastic/kibana-platform

[mshustov]

probably this PR is not finished yet, so I stop commenting 😅

[toddself]

@restrry @dover @spacedragon this is ready for a final review. i'm starting to write tests for it.

[zfy0701]

unless you plan to do it later, otherwise I think last week we agree on implement based on optimistic concurrent control, which means we need to take version query parameter to the request and properly handle it if the update fail

[elasticmachine]

💔 Build Failed

• continuous-integration/kibana-ci/pull-request

[elasticmachine]

💔 Build Failed

• continuous-integration/kibana-ci/pull-request

[spalger]

Closing this for now