-
Notifications
You must be signed in to change notification settings - Fork 255
Add RCS Strong Verification Documentation #3865
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from 3 commits
9ecc495
1375e5c
b3a7349
15ad32e
78c8b8b
935929a
09e40f6
1fe67ae
ac8fbb4
967bd9d
fe4d391
52b4ce4
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -26,6 +26,7 @@ To add a remote cluster using API key authentication: | |
| 2. [Establish trust with a remote cluster](#remote-clusters-security-api-key) | ||
| 3. [Connect to a remote cluster](#remote-clusters-connect-api-key) | ||
| 4. [Configure roles and users](#remote-clusters-privileges-api-key) | ||
| 5. (Optional) [Configure remote cluster strong verification](#remote-cluster-strong-verification) for additional security | ||
|
|
||
| If you run into any issues, refer to [Troubleshooting](/troubleshoot/elasticsearch/remote-clusters.md). | ||
|
|
||
|
|
@@ -424,3 +425,80 @@ POST /_security/user/cross-search-user | |
| ``` | ||
|
|
||
| Note that you only need to create this user on the local cluster. | ||
|
|
||
|
|
||
| ## Remote cluster strong verification [remote-cluster-strong-verification] | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I see that the improved functionlity is named "strong verification", while we (I?) generally use "strong identity verification" elsewhere. Was there a discussion about that?
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. No, this is an oversight from me. I'll update it! Thanks for reviewing! |
||
| preview::[] | ||
|
jfreden marked this conversation as resolved.
Outdated
|
||
|
|
||
| Cross-cluster API keys can be configured with strong verification to provide an additional layer of security. To enable this feature, | ||
| the cross-cluster API key is created with a certificate identity pattern. The local cluster is then required to sign each request | ||
| to the remote cluster with its private key and provide a trusted certificate whose subject matches the certificate identity | ||
| pattern configured on the cross-cluster API key. | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This paragraph could be clearer about which cluster each component lives on, and it should mention how trust is established. The phrase "provide a trusted certificate" doesn't explain that the remote cluster validates trust based on its configured certificate authorities. Something like this: "Cross-cluster API keys can be configured with strong verification to provide an additional layer of security. To enable this feature, a cross-cluster API key is created on the remote cluster with a certificate identity pattern that specifies which certificates are allowed to use it. The local cluster must then sign each request with its private key and include a certificate whose subject Distinguished Name (DN) matches the pattern. The remote cluster validates both that the certificate is trusted by its configured certificate authorities and that the certificate's subject matches the API key's identity pattern." This makes it explicit that
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Also consider mentioning that this is configured per-connection (each remote cluster alias can have different signing certs) and per-API-key (each API key can have different certificate identity patterns). This helps users understand the granularity of the feature. |
||
|
|
||
| ### How strong verification works [_how_strong_verification_works] | ||
|
|
||
| When a local cluster makes a request to a remote cluster using a cross-cluster API key: | ||
|
|
||
| 1. The local cluster signs the request headers with its configured private key and sends the signature and certificate chain as header | ||
| in the request to the remote cluster | ||
| 2. The remote cluster verifies that the API key is valid | ||
| 3. If the API key has a certificate identity pattern configured, the remote cluster extracts the Distinguished Name (DN) from the | ||
| certificate chain's leaf certificate and matches it against the certificate identity pattern | ||
| 4. The remote cluster validates that the provided certificate chain is trusted | ||
| 5. The remote cluster validates the signature and checks that the certificate is not expired | ||
|
jfreden marked this conversation as resolved.
Outdated
|
||
|
|
||
| If any of these validation steps fail, the request is rejected. | ||
|
|
||
| ### Configure strong verification [_configure_strong_verification] | ||
|
|
||
| To use certificate identity validation, the local and remote clusters must be configured to sign request headers and to verify request | ||
| headers. | ||
|
|
||
| #### On the local cluster [_certificate_identity_local_cluster] | ||
|
|
||
| The local cluster must be configured to sign cross-cluster requests with a certificate-private key pair. You can generate a signing | ||
| certificate using `elasticsearch-certutil` or use an existing certificate. | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Wondering if this should link back to the section above that shows how to use There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Also wondering: for ECH if a customer wants to use their own certs, they would need to use the bundles feature. Where would we document instructions for that?
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Yes, good point! I think that should be specifically for the ECH documentation. I'll investigate where we should add that. Thanks!
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. @gmjehovich , @jfreden , I've been working lately in all docs related with remote clusters, for all deployment types (ECH, ECE, ECK, self-managed). Feel free to ping me in private and we can review where to add information related with this PR (Strong verification configuration) for other deployment types, and I could do that in another PR. |
||
|
|
||
| ```yaml | ||
| cluster.remote.my_remote_cluster.signing.certificate: "path/to/signing/certificate.crt" | ||
| cluster.remote.my_remote_cluster.signing.key: "path/to/signing/key.key" | ||
| ``` | ||
|
|
||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I would explicitly mention that the private key can be encrypted and the password must be stored securely in the keystore. People may be wondering otherwise. |
||
| ::::{note} | ||
| Replace my_remote_cluster with your remote cluster alias and the paths with actual paths | ||
|
jfreden marked this conversation as resolved.
Outdated
|
||
| :::: | ||
|
|
||
| #### On the remote cluster [_certificate_identity_remote_cluster] | ||
|
|
||
| The remote cluster must be configured with a certificate authority that trusts the certificate that was used to sign the request headers. | ||
|
|
||
| ```yaml | ||
| cluster.remote.signing.certificate_authorities: "path/to/signing/certificate_authorities.crt" | ||
| ``` | ||
|
|
||
| When creating a cross-cluster API key on the remote cluster, specify a `certificate_identity` pattern that matches the Distinguished | ||
| Name (DN) of the local cluster's certificate. Use the [Create Cross-Cluster API key](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-security-create-cross-cluster-api-key) API: | ||
|
|
||
| ```console | ||
| POST /_security/cross_cluster/api_key | ||
| { | ||
| "name": "my-cross-cluster-api-key", | ||
| "access": { | ||
| "search": [ | ||
| { | ||
| "names": ["logs-*"] | ||
| } | ||
| ] | ||
| }, | ||
| "certificate_identity": "CN=local-cluster.example.com,O=Example Corp,C=US" | ||
| } | ||
| ``` | ||
|
|
||
| The `certificate_identity` field supports regular expressions. For example: | ||
|
|
||
| * `"CN=.*.example.com,O=Example Corp,C=US"` matches any certificate with a CN ending in "example.com" | ||
| * `"CN=local-cluster.*,O=Example Corp,C=US"` matches any certificate with a CN starting with "local-cluster" | ||
| * `"CN=.*"` matches any certificate (not recommended for production) | ||
|
jfreden marked this conversation as resolved.
Outdated
|
||
|
|
||
| To verify strong verification is active, check the Elasticsearch logs on the remote cluster for certificate validation messages, or | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Wondering if this should be more specific by including the actual output/examples from logs
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Looks like there isn't one right now, my bad. Thinking more about this, having an info log for every successful request will be too verbose so I'll just remove this. |
||
| attempt a connection without proper signing configured (which should fail with a certificate validation error). | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There are a lot more settings specified in your reference PR. how does someone know from this procedure when to use those additional settings?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good point, I wasn't sure if it made sense to include all settings. This is the most basic configuration to keep it simple, would it make sense to link the other settings here you think?
This is a setup the mirrors our SSL configuration so I could either copy some of the instructions from there or link that too? WDYT?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Including just some settings in the configuration guide should be ok, but probably we should include all settings we want users to know in the Elasticsearch reference docs. Somewhere like here: https://www.elastic.co/docs/reference/elasticsearch/configuration-reference/remote-clusters, or here: https://www.elastic.co/docs/reference/elasticsearch/configuration-reference/networking-settings#remote-cluster-network-settings (we have settings in 2 places).
We have an opened internal issue to review these settings as some of them (remote_cluster_client) are missing in the reference docs.
For your consideration.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Update: I see you have created elastic/elasticsearch#137822! So I'd just wait until the Elasticsearch PR is merged first and then in this PR just link to the reference doc for reference about all available settings, with something like: