diff --git a/docs/reference/index.asciidoc b/docs/reference/index.asciidoc index 563993f2133f5..f3676244b3a21 100644 --- a/docs/reference/index.asciidoc +++ b/docs/reference/index.asciidoc @@ -54,7 +54,7 @@ include::data-rollup-transform.asciidoc[] include::high-availability.asciidoc[] -include::security/index.asciidoc[] +include::{xes-repo-dir}/security/index.asciidoc[] include::{xes-repo-dir}/watcher/index.asciidoc[] diff --git a/docs/reference/security/index.asciidoc b/docs/reference/security/index.asciidoc deleted file mode 100644 index ed11b5916cb2c..0000000000000 --- a/docs/reference/security/index.asciidoc +++ /dev/null @@ -1,18 +0,0 @@ -[[secure-cluster]] -= Secure a cluster - -[partintro] --- -The {stack-security-features} enable you to easily secure a cluster. You can -password-protect your data as well as implement more advanced security -measures such as encrypting communications, role-based access control, -IP filtering, and auditing. - -* <> -* <> - --- - -include::overview.asciidoc[] - -include::{xes-repo-dir}/security/configuring-es.asciidoc[] diff --git a/x-pack/docs/en/security/auditing/event-types.asciidoc b/x-pack/docs/en/security/auditing/event-types.asciidoc index 19947e40b5553..dfa0c72b5e2d9 100644 --- a/x-pack/docs/en/security/auditing/event-types.asciidoc +++ b/x-pack/docs/en/security/auditing/event-types.asciidoc @@ -18,7 +18,7 @@ The following is a list of the events that can be generated: realm type. | `access_denied` | | | Logged when an authenticated user attempts to execute an action they do not have the necessary - <> to perform. + <> to perform. | `access_granted` | | | Logged when an authenticated user attempts to execute an action they have the necessary privilege to perform. When the `system_access_granted` event is included, all system @@ -28,7 +28,7 @@ The following is a list of the events that can be generated: another user that they have the necessary privileges to do. | `run_as_denied` | | | Logged when an authenticated user attempts to <> another user action they do not have the necessary - <> to do so. + <> to do so. | `tampered_request` | | | Logged when the {security-features} detect that the request has been tampered with. Typically relates to `search/scroll` requests when the scroll ID is believed to have been diff --git a/x-pack/docs/en/security/authentication/index.asciidoc b/x-pack/docs/en/security/authentication/index.asciidoc index 298376e291a3c..8e0fdb8f4a9df 100644 --- a/x-pack/docs/en/security/authentication/index.asciidoc +++ b/x-pack/docs/en/security/authentication/index.asciidoc @@ -11,13 +11,8 @@ include::native-realm.asciidoc[] include::pki-realm.asciidoc[] include::saml-realm.asciidoc[] include::kerberos-realm.asciidoc[] - -include::{xes-repo-dir}/security/authentication/custom-realm.asciidoc[] - -include::{xes-repo-dir}/security/authentication/anonymous-access.asciidoc[] - -include::{xes-repo-dir}/security/authentication/user-cache.asciidoc[] - -include::{xes-repo-dir}/security/authentication/saml-guide.asciidoc[] - -include::{xes-repo-dir}/security/authentication/oidc-guide.asciidoc[] \ No newline at end of file +include::custom-realm.asciidoc[] +include::anonymous-access.asciidoc[] +include::user-cache.asciidoc[] +include::saml-guide.asciidoc[] +include::oidc-guide.asciidoc[] \ No newline at end of file diff --git a/x-pack/docs/en/security/authorization/index.asciidoc b/x-pack/docs/en/security/authorization/index.asciidoc index 7f63565ca0137..e9c91b96531e5 100644 --- a/x-pack/docs/en/security/authorization/index.asciidoc +++ b/x-pack/docs/en/security/authorization/index.asciidoc @@ -1,24 +1,13 @@ include::overview.asciidoc[] - include::built-in-roles.asciidoc[] - -include::{xes-repo-dir}/security/authorization/managing-roles.asciidoc[] - +include::managing-roles.asciidoc[] include::privileges.asciidoc[] - include::document-level-security.asciidoc[] - include::field-level-security.asciidoc[] - -include::{xes-repo-dir}/security/authorization/alias-privileges.asciidoc[] - -include::{xes-repo-dir}/security/authorization/mapping-roles.asciidoc[] - -include::{xes-repo-dir}/security/authorization/field-and-document-access-control.asciidoc[] - -include::{xes-repo-dir}/security/authorization/run-as-privilege.asciidoc[] - +include::alias-privileges.asciidoc[] +include::mapping-roles.asciidoc[] +include::field-and-document-access-control.asciidoc[] +include::run-as-privilege.asciidoc[] include::configuring-authorization-delegation.asciidoc[] - -include::{xes-repo-dir}/security/authorization/custom-authorization.asciidoc[] +include::custom-authorization.asciidoc[] diff --git a/x-pack/docs/en/security/ccs-clients-integrations.asciidoc b/x-pack/docs/en/security/ccs-clients-integrations/index.asciidoc similarity index 80% rename from x-pack/docs/en/security/ccs-clients-integrations.asciidoc rename to x-pack/docs/en/security/ccs-clients-integrations/index.asciidoc index e5a477e3c153c..2691ad7042085 100644 --- a/x-pack/docs/en/security/ccs-clients-integrations.asciidoc +++ b/x-pack/docs/en/security/ccs-clients-integrations/index.asciidoc @@ -32,14 +32,9 @@ be secured as well, or at least communicate with the cluster in a secured way: * {kibana-ref}/secure-reporting.html[Reporting] * {winlogbeat-ref}/securing-beats.html[Winlogbeat] -include::ccs-clients-integrations/cross-cluster.asciidoc[] - -include::ccs-clients-integrations/java.asciidoc[] - -include::ccs-clients-integrations/http.asciidoc[] - -include::ccs-clients-integrations/hadoop.asciidoc[] - -include::ccs-clients-integrations/beats.asciidoc[] - -include::ccs-clients-integrations/monitoring.asciidoc[] +include::cross-cluster.asciidoc[] +include::java.asciidoc[] +include::http.asciidoc[] +include::hadoop.asciidoc[] +include::beats.asciidoc[] +include::monitoring.asciidoc[] diff --git a/x-pack/docs/en/security/ccs-clients-integrations/monitoring.asciidoc b/x-pack/docs/en/security/ccs-clients-integrations/monitoring.asciidoc index 37c7e38f651bd..45d6296948d15 100644 --- a/x-pack/docs/en/security/ccs-clients-integrations/monitoring.asciidoc +++ b/x-pack/docs/en/security/ccs-clients-integrations/monitoring.asciidoc @@ -1,7 +1,7 @@ [[secure-monitoring]] === Monitoring and security -The <> consist of two components: +The {stack} {monitor-features} consist of two components: an agent that you install on on each {es} and Logstash node, and a Monitoring UI in {kib}. The monitoring agent collects and indexes metrics from the nodes and you visualize the data through the Monitoring dashboards in {kib}. The agent diff --git a/x-pack/docs/en/security/configuring-es.asciidoc b/x-pack/docs/en/security/configuring-es.asciidoc index ea42b971a7688..7beb72e475250 100644 --- a/x-pack/docs/en/security/configuring-es.asciidoc +++ b/x-pack/docs/en/security/configuring-es.asciidoc @@ -139,13 +139,13 @@ Events are logged to a dedicated `_audit.json` file in To walk through the configuration of {security-features} in {es}, {kib}, {ls}, and {metricbeat}, see {stack-ov}/security-getting-started.html[Getting started with security]. -include::{es-repo-dir}/security/securing-communications/securing-elasticsearch.asciidoc[] +include::securing-communications/securing-elasticsearch.asciidoc[] -include::{es-repo-dir}/security/securing-communications/configuring-tls-docker.asciidoc[] +include::securing-communications/configuring-tls-docker.asciidoc[] -include::{es-repo-dir}/security/securing-communications/enabling-cipher-suites.asciidoc[] +include::securing-communications/enabling-cipher-suites.asciidoc[] -include::{es-repo-dir}/security/securing-communications/separating-node-client-traffic.asciidoc[] +include::securing-communications/separating-node-client-traffic.asciidoc[] include::authentication/configuring-active-directory-realm.asciidoc[] include::authentication/configuring-file-realm.asciidoc[] @@ -156,6 +156,6 @@ include::authentication/configuring-saml-realm.asciidoc[] include::authentication/configuring-kerberos-realm.asciidoc[] -include::{es-repo-dir}/security/reference/files.asciidoc[] +include::reference/files.asciidoc[] include::fips-140-compliance.asciidoc[] diff --git a/x-pack/docs/en/security/get-started-builtin-users.asciidoc b/x-pack/docs/en/security/get-started-builtin-users.asciidoc new file mode 100644 index 0000000000000..766c3263d756d --- /dev/null +++ b/x-pack/docs/en/security/get-started-builtin-users.asciidoc @@ -0,0 +1,33 @@ +// tag::create-users[] +There are <> that you can use for specific +administrative purposes: `apm_system`, `beats_system`, `elastic`, `kibana`, +`logstash_system`, and `remote_monitoring_user`. + +// end::create-users[] + +Before you can use them, you must set their passwords: + +. Restart {es}. For example, if you installed {es} with a `.tar.gz` package, run +the following command from the {es} directory: ++ +-- +["source","sh",subs="attributes,callouts"] +---------------------------------------------------------------------- +./bin/elasticsearch +---------------------------------------------------------------------- + +See {ref}/starting-elasticsearch.html[Starting {es}]. +-- + +. Set the built-in users' passwords. ++ +-- +// tag::create-users[] +Run the following command from the {es} directory: + +["source","sh",subs="attributes,callouts"] +---------------------------------------------------------------------- +./bin/elasticsearch-setup-passwords interactive +---------------------------------------------------------------------- +// end::create-users[] +-- diff --git a/x-pack/docs/en/security/get-started-enable-security.asciidoc b/x-pack/docs/en/security/get-started-enable-security.asciidoc new file mode 100644 index 0000000000000..bbe2999fc6753 --- /dev/null +++ b/x-pack/docs/en/security/get-started-enable-security.asciidoc @@ -0,0 +1,35 @@ +When you use the basic and trial licenses, the {es} {security-features} are +disabled by default. To enable them: + +. Stop {kib}. The method for starting and stopping {kib} varies depending on +how you installed it. For example, if you installed {kib} from an archive +distribution (`.tar.gz` or `.zip`), stop it by entering `Ctrl-C` on the command +line. See {kibana-ref}/start-stop.html[Starting and stopping {kib}]. + +. Stop {es}. For example, if you installed {es} from an archive distribution, +enter `Ctrl-C` on the command line. See +{ref}/stopping-elasticsearch.html[Stopping {es}]. + +. Add the `xpack.security.enabled` setting to the +`ES_PATH_CONF/elasticsearch.yml` file. ++ +-- +TIP: The `ES_PATH_CONF` environment variable contains the path for the {es} +configuration files. If you installed {es} using archive distributions (`zip` or +`tar.gz`), it defaults to `ES_HOME/config`. If you used package distributions +(Debian or RPM), it defaults to `/etc/elasticsearch`. For more information, see +{ref}/settings.html[Configuring {es}]. + +For example, add the following setting: + +[source,yaml] +---- +xpack.security.enabled: true +---- + +TIP: If you have a basic or trial license, the default value for this setting is +`false`. If you have a gold or higher license, the default value is `true`. +Therefore, it is a good idea to explicitly add this setting to avoid confusion +about whether {security-features} are enabled. + +-- diff --git a/x-pack/docs/en/security/get-started-kibana-users.asciidoc b/x-pack/docs/en/security/get-started-kibana-users.asciidoc new file mode 100644 index 0000000000000..2d06f670cdcba --- /dev/null +++ b/x-pack/docs/en/security/get-started-kibana-users.asciidoc @@ -0,0 +1,62 @@ +When the {es} {security-features} are enabled, users must log in to {kib} +with a valid user ID and password. + +{kib} also performs some tasks under the covers that require use of the +built-in `kibana` user. + +. Configure {kib} to use the built-in `kibana` user and the password that you +created: + +** If you don't mind having passwords visible in your configuration file, +uncomment and update the following settings in the `kibana.yml` file in your +{kib} directory: ++ +-- +TIP: If you installed {kib} using archive distributions (`zip` or +`tar.gz`), the `kibana.yml` configuration file is in `KIBANA_HOME/config`. If +you used package distributions (Debian or RPM), it's in `/etc/kibana`. For more +information, see {kibana-ref}/settings.html[Configuring {kib}]. + +For example, add the following settings: + +[source,yaml] +---- +elasticsearch.username: "kibana" +elasticsearch.password: "your_password" +---- + +Specify the password that you set with the `elasticsearch-setup-passwords` +command then save your changes to the file. +-- + +** If you prefer not to put your user ID and password in the `kibana.yml` file, +store them in a keystore instead. Run the following commands to create the {kib} +keystore and add the secure settings: ++ +-- +// tag::store-kibana-user[] +["source","sh",subs="attributes,callouts"] +---------------------------------------------------------------------- +./bin/kibana-keystore create +./bin/kibana-keystore add elasticsearch.username +./bin/kibana-keystore add elasticsearch.password +---------------------------------------------------------------------- + +When prompted, specify the `kibana` built-in user and its password for these +setting values. The settings are automatically applied when you start {kib}. +To learn more, see {kibana-ref}/secure-settings.html[Secure settings]. +// end::store-kibana-user[] +-- + +. Restart {kib}. For example, if you installed +{kib} with a `.tar.gz` package, run the following command from the {kib} +directory: ++ +-- +["source","sh",subs="attributes,callouts"] +---------------------------------------------------------------------- +./bin/kibana +---------------------------------------------------------------------- + +See {kibana-ref}/start-stop.html[Starting and stopping {kib}]. +-- \ No newline at end of file diff --git a/x-pack/docs/en/security/get-started-security.asciidoc b/x-pack/docs/en/security/get-started-security.asciidoc new file mode 100644 index 0000000000000..7fad5039881a1 --- /dev/null +++ b/x-pack/docs/en/security/get-started-security.asciidoc @@ -0,0 +1,378 @@ +[role="xpack"] +[testenv="basic"] +[[security-getting-started]] +== Tutorial: Getting started with security + +In this tutorial, you learn how to secure a cluster by configuring users and +roles in {es}, {kib}, {ls}, and {metricbeat}. + +[float] +[[get-started-security-prerequisites]] +=== Before you begin + +. Install and configure {es}, {kib}, {ls}, and {metricbeat} as described in +{stack-gs}/get-started-elastic-stack.html[Getting started with the {stack}]. ++ +-- +IMPORTANT: To complete this tutorial, you must install the default {es} and +{kib} packages, which include role-based access control (RBAC) and native +authentication {security-features}. When you install these products, they apply +basic licenses with no expiration dates. All of the subsequent steps in this +tutorial assume that you are using a basic license. For more information, see +{subscriptions} and +{stack-ov}/license-management.html[License management]. + +-- + +. Stop {ls}. The method for starting and stopping {ls} varies depending on whether +you are running it from the command line or running it as a service. For example, +if you are running {ls} from the command line, you can stop it by entering +`Ctrl-C`. See {logstash-ref}/shutdown.html[Shutting down {ls}]. + +. Stop {metricbeat}. For example, enter `Ctrl-C` on the command line where it is +running. + +. Launch the {kib} web interface by pointing your browser to port 5601. For +example, http://127.0.0.1:5601[http://127.0.0.1:5601]. + +[role="xpack"] +[[get-started-enable-security]] +=== Enable {es} {security-features} + +include::get-started-enable-security.asciidoc[] + +. Enable single-node discovery in the `ES_PATH_CONF/elasticsearch.yml` file. ++ +-- +This tutorial involves a single node cluster, but if you had multiple +nodes, you would enable {es} {security-features} on every node in the cluster +and configure Transport Layer Security (TLS) for internode-communication, which +is beyond the scope of this tutorial. By enabling single-node discovery, we are +postponing the configuration of TLS. For example, add the following setting: + +[source,yaml] +---- +discovery.type: single-node +---- + +For more information, see +{ref}/bootstrap-checks.html#single-node-discovery[Single-node discovery]. +-- + +When you enable {es} {security-features}, basic authentication is enabled by +default. To communicate with the cluster, you must specify a username and +password. Unless you <>, all requests +that don't include a user name and password are rejected. + +[role="xpack"] +[[get-started-built-in-users]] +=== Create passwords for built-in users + +include::get-started-builtin-users.asciidoc[] + +You need these built-in users in subsequent steps, so choose passwords that you +can remember! + +NOTE: This tutorial does not use the built-in `apm_system`, `logstash_system`, +`beats_system`, and `remote_monitoring_user` users, which are typically +associated with monitoring. For more information, see +{logstash-ref}/ls-security.html#ls-monitoring-user[Configuring credentials for {ls} monitoring] +and {metricbeat-ref}/monitoring.html[Monitoring {metricbeat}]. + +[role="xpack"] +[[get-started-kibana-user]] +=== Add the built-in user to {kib} + +include::get-started-kibana-users.asciidoc[] + +[role="xpack"] +[[get-started-authentication]] +=== Configure authentication + +Now that you've set up the built-in users, you need to decide how you want to +manage all the other users. + +The {stack} _authenticates_ users to ensure that they are valid. The +authentication process is handled by _realms_. You can use one or more built-in +realms, such as the native, file, LDAP, PKI, Active Directory, SAML, or Kerberos +realms. Alternatively, you can create your own custom realms. In this tutorial, +we'll use a native realm. + +In general, you configure realms by adding `xpack.security.authc.realms` +settings in the `elasticsearch.yml` file. However, the native realm is available +by default when no other realms are configured. Therefore, you don't need to do +any extra configuration steps in this tutorial. You can jump straight to +creating users! + +If you want to learn more about authentication and realms, see +<>. + +[role="xpack"] +[[get-started-users]] +=== Create users + +Let's create two users in the native realm. + +. Log in to {kib} with the `elastic` built-in user. + +. Go to the *Management / Security / Users* page: ++ +-- +[role="screenshot"] +image::security/images/management-builtin-users.jpg["User management screenshot in Kibana"] + +In this example, you can see a list of built-in users. +-- + +. Click *Create new user*. For example, create a user for yourself: ++ +-- +[role="screenshot"] +image::security/images/create-user.jpg["Creating a user in Kibana"] + +You'll notice that when you create a user, you can assign it a role. Don't +choose a role yet--we'll come back to that in subsequent steps. +-- + +. Click *Create new user* and create a `logstash_internal` user. ++ +-- +In {stack-gs}/get-started-elastic-stack.html[Getting started with the {stack}], +you configured {ls} to listen for {metricbeat} +input and to send the events to {es}. You therefore need to create a user +that {ls} can use to communicate with {es}. For example: + +[role="screenshot"] +image::security/images/create-logstash-user.jpg["Creating a {ls} user in {kib}"] +-- + +[role="xpack"] +[[get-started-roles]] +=== Assign roles + +By default, all users can change their own passwords, get information about +themselves, and run the `authenticate` API. If you want them to do more than +that, you need to give them one or more _roles_. + +Each role defines a specific set of actions (such as read, create, or delete) +that can be performed on specific secured resources (such as indices, aliases, +documents, fields, or clusters). To help you get up and running, there are +built-in roles. + +Go to the *Management / Security / Roles* page to see them: + +[role="screenshot"] +image::security/images/management-roles.jpg["Role management screenshot in Kibana"] + +Select a role to see more information about its privileges. For example, select +the `kibana_system` role to see its list of cluster and index privileges. To +learn more, see <>. + +Let's assign the `kibana_user` role to your user. Go back to the +*Management / Security / Users* page and select your user. Add the `kibana_user` +role and save the change. For example: + +[role="screenshot"] +image::security/images/assign-role.jpg["Assigning a role to a user in Kibana"] + +This user now has access to all features in {kib}. For more information about granting +access to Kibana see {kibana-ref}/xpack-security-authorization.html[Kibana Authorization]. + +If you completed all of the steps in +{stack-gs}/get-started-elastic-stack.html[Getting started with the {stack}], you should +have {metricbeat} data stored in {es}. Let's create two roles that grant +different levels of access to that data. + +Go to the *Management / Security / Roles* page and click *Create role*. + +Create a `metricbeat_reader` role that has `read` and `view_index_metadata` +privileges on the `metricbeat-*` indices: + +[role="screenshot"] +image::security/images/create-reader-role.jpg["Creating a role in Kibana"] + +Create a `metricbeat_writer` role that has `manage_index_templates` and `monitor` +cluster privileges, as well as `write`, `delete`, and `create_index` privileges +on the `metricbeat-*` indices: + +[role="screenshot"] +image::security/images/create-writer-role.jpg["Creating another role in Kibana"] + +Now go back to the *Management / Security / Users* page and assign these roles +to the appropriate users. Assign the `metricbeat_reader` role to your personal +user. Assign the `metricbeat_writer` role to the `logstash_internal` user. + +The list of users should now contain all of the built-in users as well as the +two you created. It should also show the appropriate roles for your users: + +[role="screenshot"] +image::security/images/management-users.jpg["User management screenshot in Kibana"] + +If you want to learn more about authorization and roles, see <>. + +[role="xpack"] +[[get-started-logstash-user]] +=== Add user information in {ls} + +In order for {ls} to send data successfully to {es}, you must configure its +authentication credentials in the {ls} configuration file. + +. Configure {ls} to use the `logstash_internal` user and the password that you +created: + +** If you don't mind having passwords visible in your configuration file, add +the following `user` and `password` settings in the `demo-metrics-pipeline.conf` +file in your {ls} directory: ++ +-- +[source,ruby] +---- +... + +output { + elasticsearch { + hosts => "localhost:9200" + manage_template => false + index => "%{[@metadata][beat]}-%{[@metadata][version]}-%{+YYYY.MM.dd}" + user => "logstash_internal" <1> + password => "your_password" <2> + } +} +---- +<1> Specify the `logstash_internal` user that you created earlier in this tutorial. +<2> Specify the password that you chose for this user ID. +-- + +** If you prefer not to put your user ID and password in the configuration file, +store them in a keystore instead. ++ +-- +Run the following commands to create the {ls} +keystore and add the secure settings: + +["source","sh",subs="attributes,callouts"] +---------------------------------------------------------------------- +set +o history +export LOGSTASH_KEYSTORE_PASS=mypassword <1> +set -o history +./bin/logstash-keystore create +./bin/logstash-keystore add ES_USER +./bin/logstash-keystore add ES_PWD +---------------------------------------------------------------------- +<1> You can optionally protect access to the {ls} keystore by storing a password +in an environment variable called `LOGSTASH_KEYSTORE_PASS`. For more information, +see {logstash-ref}/keystore.html#keystore-password[Keystore password]. + +When prompted, specify the `logstash_internal` user and its password for the +`ES_USER` and `ES_PWD` values. + +NOTE: The {ls} keystore differs from the {kib} keystore. Whereas the {kib} +keystore enables you to store `kibana.yml` settings by name, the {ls} keystore +enables you to create arbitrary names that you can reference in the {ls} +configuration. To learn more, see +{logstash-ref}/keystore.html[Secrets keystore for secure settings]. + +You can now use these `ES_USER` and `ES_PWD` keys in your configuration +file. For example, add the `user` and `password` settings in the +`demo-metrics-pipeline.conf` file as follows: + +[source,ruby] +---- +... + +output { + elasticsearch { + hosts => "localhost:9200" + manage_template => false + index => "%{[@metadata][beat]}-%{[@metadata][version]}-%{+YYYY.MM.dd}" + user => "${ES_USER}" + password => "${ES_PWD}" + } +} +---- +-- + +. Start {ls} by using the appropriate method for your environment. ++ +-- +For example, to +run {ls} from a command line, go to the {ls} directory and enter the following +command: + +["source","sh",subs="attributes,callouts"] +---------------------------------------------------------------------- +./bin/logstash -f demo-metrics-pipeline.conf +---------------------------------------------------------------------- + +To start {ls} as a service, see +{logstash-ref}/running-logstash.html[Running {ls} as a service on Debian or RPM]. +-- + +. If you were connecting directly from {metricbeat} to {es}, you would need +to configure authentication credentials for the {es} output in the {metricbeat} +configuration file. In +{stack-gs}/get-started-elastic-stack.html[Getting started with the {stack}], +however, you configured +{metricbeat} to send the data to {ls} for additional parsing, so no extra +settings are required in {metricbeat}. For more information, see +{metricbeat-ref}/securing-metricbeat.html[Securing {metricbeat}]. + +. Start {metricbeat} by using the appropriate method for your environment. ++ +-- +For example, on macOS, run the following command from the {metricbeat} directory: + +["source","sh",subs="attributes,callouts"] +---------------------------------------------------------------------- +./metricbeat -e +---------------------------------------------------------------------- + +For more methods, see {metricbeat-ref}/metricbeat-starting.html[Starting {metricbeat}]. +-- + +Wait a few minutes for new data to be sent from {metricbeat} to {ls} and {es}. + +[role="xpack"] +[[get-started-verify-users]] +=== View system metrics in {kib} + +Log in to {kib} with the user ID that has `metricbeat_reader` and `kibana_user` +roles (for example, `jdoe`). + +These roles enable the user to see the system metrics in {kib} (for example, on +the *Discover* page or in the +http://localhost:5601/app/kibana#/dashboard/Metricbeat-system-overview[{metricbeat} system overview dashboard]). + +[float] +[[gs-security-nextsteps]] +=== What's next? + +Congratulations! You've successfully set up authentication and authorization by +using the native realm. You learned how to create user IDs and roles that +prevent unauthorized access to the {stack}. + +Later, when you're ready to increase the number of nodes in your cluster, you'll +want to encrypt communications across the {stack}. To learn how, read +<>. + +For more detailed information about securing the {stack}, see: + +* {ref}/configuring-security.html[Configuring security in {es}]. Encrypt +inter-node communications, set passwords for the built-in users, and manage your +users and roles. + +* {kibana-ref}/using-kibana-with-security.html[Configuring security in {kib}]. +Set the authentication credentials in {kib} and encrypt communications between +the browser and the {kib} server. + +* {logstash-ref}/ls-security.html[Configuring security in Logstash]. Set the +authentication credentials for Logstash and encrypt communications between +Logstash and {es}. + +* <>. Configure authentication +credentials and encrypt connections to {es}. + +* Configuring the Java transport client to use encrypted communications. + +* {hadoop-ref}/security.html[Configuring {es} for Apache Hadoop to use secured transport]. + diff --git a/x-pack/docs/en/security/how-security-works.asciidoc b/x-pack/docs/en/security/how-security-works.asciidoc new file mode 100644 index 0000000000000..e05991cac3b7d --- /dev/null +++ b/x-pack/docs/en/security/how-security-works.asciidoc @@ -0,0 +1,50 @@ +[role="xpack"] +[[how-security-works]] +== How security works + +An Elasticsearch cluster is typically made out of many moving parts. There are +the Elasticsearch nodes that form the cluster and often Logstash instances, +Kibana instances, Beats agents, and clients all communicating with the cluster. +It should not come as a surprise that securing such clusters has many facets and +layers. + +The {stack-security-features} provide the means to secure the Elastic cluster +on several levels: + + * <> + * <> + * Node/client authentication and channel encryption + * Auditing + +[float] +=== Node/client authentication and channel encryption + +The {security-features} support configuring SSL/TLS for securing the +communication channels to, from and within the cluster. This support accounts for: + + * Encryption of data transmitted over the wires + * Certificate based node authentication - preventing unauthorized nodes/clients + from establishing a connection with the cluster. + +For more information, see <>. + +The {security-features} also enable you to <> +which can be seen as a light mechanism for node/client authentication. With IP +filtering, you can restrict the nodes and clients that can connect to the +cluster based on their IP addresses. The IP filters configuration provides +whitelisting and blacklisting of IPs, subnets and DNS domains. + + +[float] +=== Auditing +When dealing with any secure system, it is critical to have a audit trail +mechanism set in place. Audit trails log various activities/events that occur in +the system, enabling you to analyze and back track past events when things go +wrong (e.g. security breach). + +The {security-features} provide such audit trail functionality for all nodes in +the cluster. You can configure the audit level which accounts for the type of +events that are logged. These events include failed authentication attempts, +user access denied, node connection denied, and more. + +For more information on auditing see <>. diff --git a/x-pack/docs/en/security/index.asciidoc b/x-pack/docs/en/security/index.asciidoc new file mode 100644 index 0000000000000..294519debd217 --- /dev/null +++ b/x-pack/docs/en/security/index.asciidoc @@ -0,0 +1,53 @@ +[[secure-cluster]] += Secure a cluster + +[partintro] +-- +The {stack-security-features} enable you to easily secure a cluster. You can +password-protect your data as well as implement more advanced security +measures such as encrypting communications, role-based access control, +IP filtering, and auditing. + +* <> +* <> +* <> +* <> +* <> +* <> +* <> +* <> +* <> +* <> +* <> +* <> +* <> +* <> + +-- + +include::overview.asciidoc[] + +include::configuring-es.asciidoc[] + +include::how-security-works.asciidoc[] + +include::authentication/index.asciidoc[] + +include::authorization/index.asciidoc[] + +include::auditing/index.asciidoc[] + +include::securing-communications/index.asciidoc[] + +include::using-ip-filtering.asciidoc[] + +include::ccs-clients-integrations/index.asciidoc[] + +include::get-started-security.asciidoc[] + +include::securing-communications/tutorial-tls-intro.asciidoc[] + +include::troubleshooting.asciidoc[] + +include::limitations.asciidoc[] + diff --git a/x-pack/docs/en/security/limitations.asciidoc b/x-pack/docs/en/security/limitations.asciidoc new file mode 100644 index 0000000000000..d3cd6e589e2bb --- /dev/null +++ b/x-pack/docs/en/security/limitations.asciidoc @@ -0,0 +1,93 @@ +[role="xpack"] +[[security-limitations]] +== Security limitations +[subs="attributes"] +++++ +Limitations +++++ + +[float] +=== Plugins + +Elasticsearch's plugin infrastructure is extremely flexible in terms of what can +be extended. While it opens up Elasticsearch to a wide variety of (often custom) +additional functionality, when it comes to security, this high extensibility level +comes at a cost. We have no control over the third-party plugins' code (open +source or not) and therefore we cannot guarantee their compliance with +{stack-security-features}. For this reason, third-party plugins are not +officially supported on clusters with {security-features} enabled. + +[float] +=== Changes in index wildcard behavior + +Elasticsearch clusters with the {security-features} enabled apply the `/_all` +wildcard, and all other wildcards, to the indices that the current user has +privileges for, not the set of all indices on the cluster. + +[float] +=== Multi document APIs + +Multi get and multi term vectors API throw IndexNotFoundException when trying to access non existing indices that the user is +not authorized for. By doing that they leak information regarding the fact that the index doesn't exist, while the user is not +authorized to know anything about those indices. + +[float] +=== Filtered index aliases + +Aliases containing filters are not a secure way to restrict access to individual +documents, due to the limitations described in +<>. +The {stack-security-features} provide a secure way to restrict access to +documents through the +<> feature. + +[float] +=== Field and document level security limitations + +When a user's role enables document or field level security for an index: + +* The user cannot perform write operations: +** The update API isn't supported. +** Update requests included in bulk requests aren't supported. +* The request cache is disabled for search requests. + +When a user's role enables document level security for an index: + +* Document level security isn't applied for APIs that aren't document based. + An example is the field stats API. +* Document level security doesn't affect global index statistics that relevancy + scoring uses. So this means that scores are computed without taking the role + query into account. Note that documents not matching with the role query are + never returned. +* The `has_child` and `has_parent` queries aren't supported as query in the + role definition. The `has_child` and `has_parent` queries can be used in the + search API with document level security enabled. +* Any query that makes remote calls to fetch data to query by isn't supported. + The following queries aren't supported: +** The `terms` query with terms lookup isn't supported. +** The `geo_shape` query with indexed shapes isn't supported. +** The `percolate` query isn't supported. +* If suggesters are specified and document level security is enabled then + the specified suggesters are ignored. +* A search request cannot be profiled if document level security is enabled. + +[float] +[[alias-limitations]] +=== Index and field names can be leaked when using aliases + +Calling certain Elasticsearch APIs on an alias can potentially leak information +about indices that the user isn't authorized to access. For example, when you get +the mappings for an alias with the `_mapping` API, the response includes the +index name and mappings for each index that the alias applies to. + +Until this limitation is addressed, avoid index and field names that contain +confidential or sensitive information. + +[float] +=== LDAP realm + +The <> does not currently support the discovery of nested +LDAP Groups. For example, if a user is a member of `group_1` and `group_1` is a +member of `group_2`, only `group_1` will be discovered. However, the +<> *does* support transitive +group membership. diff --git a/docs/reference/security/overview.asciidoc b/x-pack/docs/en/security/overview.asciidoc similarity index 100% rename from docs/reference/security/overview.asciidoc rename to x-pack/docs/en/security/overview.asciidoc diff --git a/docs/reference/security/reference/files.asciidoc b/x-pack/docs/en/security/reference/files.asciidoc similarity index 100% rename from docs/reference/security/reference/files.asciidoc rename to x-pack/docs/en/security/reference/files.asciidoc diff --git a/docs/reference/security/securing-communications/configuring-tls-docker.asciidoc b/x-pack/docs/en/security/securing-communications/configuring-tls-docker.asciidoc similarity index 100% rename from docs/reference/security/securing-communications/configuring-tls-docker.asciidoc rename to x-pack/docs/en/security/securing-communications/configuring-tls-docker.asciidoc diff --git a/docs/reference/security/securing-communications/enabling-cipher-suites.asciidoc b/x-pack/docs/en/security/securing-communications/enabling-cipher-suites.asciidoc similarity index 96% rename from docs/reference/security/securing-communications/enabling-cipher-suites.asciidoc rename to x-pack/docs/en/security/securing-communications/enabling-cipher-suites.asciidoc index 51d5e5f6de650..4e51f5e43ff24 100644 --- a/docs/reference/security/securing-communications/enabling-cipher-suites.asciidoc +++ b/x-pack/docs/en/security/securing-communications/enabling-cipher-suites.asciidoc @@ -1,6 +1,6 @@ [role="xpack"] [[ciphers]] -=== Enabling Cipher Suites for Stronger Encryption +=== Enabling cipher suites for stronger encryption The TLS and SSL protocols use a cipher suite that determines the strength of encryption used to protect the data. You may want to increase the strength of diff --git a/x-pack/docs/en/security/securing-communications.asciidoc b/x-pack/docs/en/security/securing-communications/index.asciidoc similarity index 67% rename from x-pack/docs/en/security/securing-communications.asciidoc rename to x-pack/docs/en/security/securing-communications/index.asciidoc index 2ccea2c53659f..90989901b15eb 100644 --- a/x-pack/docs/en/security/securing-communications.asciidoc +++ b/x-pack/docs/en/security/securing-communications/index.asciidoc @@ -17,14 +17,5 @@ This section shows how to: The authentication of new nodes helps prevent a rogue node from joining the cluster and receiving data through replication. -include::{es-repo-dir}/security/securing-communications/setting-up-ssl.asciidoc[] +include::setting-up-ssl.asciidoc[] -[[ciphers]] -=== Enabling cipher suites for stronger encryption - -See {ref}/ciphers.html[Enabling Cipher Suites for Stronger Encryption]. - -[[separating-node-client-traffic]] -=== Separating node-to-node and client traffic - -See {ref}/separating-node-client-traffic.html[Separating node-to-node and client traffic]. diff --git a/docs/reference/security/securing-communications/node-certificates.asciidoc b/x-pack/docs/en/security/securing-communications/node-certificates.asciidoc similarity index 100% rename from docs/reference/security/securing-communications/node-certificates.asciidoc rename to x-pack/docs/en/security/securing-communications/node-certificates.asciidoc diff --git a/docs/reference/security/securing-communications/securing-elasticsearch.asciidoc b/x-pack/docs/en/security/securing-communications/securing-elasticsearch.asciidoc similarity index 100% rename from docs/reference/security/securing-communications/securing-elasticsearch.asciidoc rename to x-pack/docs/en/security/securing-communications/securing-elasticsearch.asciidoc diff --git a/docs/reference/security/securing-communications/separating-node-client-traffic.asciidoc b/x-pack/docs/en/security/securing-communications/separating-node-client-traffic.asciidoc similarity index 100% rename from docs/reference/security/securing-communications/separating-node-client-traffic.asciidoc rename to x-pack/docs/en/security/securing-communications/separating-node-client-traffic.asciidoc diff --git a/docs/reference/security/securing-communications/setting-up-ssl.asciidoc b/x-pack/docs/en/security/securing-communications/setting-up-ssl.asciidoc similarity index 100% rename from docs/reference/security/securing-communications/setting-up-ssl.asciidoc rename to x-pack/docs/en/security/securing-communications/setting-up-ssl.asciidoc diff --git a/docs/reference/security/securing-communications/tls-ad.asciidoc b/x-pack/docs/en/security/securing-communications/tls-ad.asciidoc similarity index 100% rename from docs/reference/security/securing-communications/tls-ad.asciidoc rename to x-pack/docs/en/security/securing-communications/tls-ad.asciidoc diff --git a/docs/reference/security/securing-communications/tls-http.asciidoc b/x-pack/docs/en/security/securing-communications/tls-http.asciidoc similarity index 100% rename from docs/reference/security/securing-communications/tls-http.asciidoc rename to x-pack/docs/en/security/securing-communications/tls-http.asciidoc diff --git a/docs/reference/security/securing-communications/tls-ldap.asciidoc b/x-pack/docs/en/security/securing-communications/tls-ldap.asciidoc similarity index 100% rename from docs/reference/security/securing-communications/tls-ldap.asciidoc rename to x-pack/docs/en/security/securing-communications/tls-ldap.asciidoc diff --git a/docs/reference/security/securing-communications/tls-transport.asciidoc b/x-pack/docs/en/security/securing-communications/tls-transport.asciidoc similarity index 100% rename from docs/reference/security/securing-communications/tls-transport.asciidoc rename to x-pack/docs/en/security/securing-communications/tls-transport.asciidoc diff --git a/x-pack/docs/en/security/securing-communications/tutorial-tls-internode.asciidoc b/x-pack/docs/en/security/securing-communications/tutorial-tls-internode.asciidoc index d32edb7eea956..b22ac07e09dcd 100644 --- a/x-pack/docs/en/security/securing-communications/tutorial-tls-internode.asciidoc +++ b/x-pack/docs/en/security/securing-communications/tutorial-tls-internode.asciidoc @@ -152,7 +152,7 @@ command from the {es} directory: NOTE: If you already configured passwords for these users in other tutorials, you can skip this step. -include::{stack-repo-dir}/security/get-started-builtin-users.asciidoc[tag=create-users] +include:{xes-repo-dir}/security/get-started-builtin-users.asciidoc[tag=create-users] After you setup the password for the `kibana` built-in user, <>. @@ -160,7 +160,7 @@ After you setup the password for the `kibana` built-in user, For example, run the following commands to create the {kib} keystore and add the `kibana` built-in user and its password in secure settings: -include::{stack-repo-dir}/security/get-started-kibana-users.asciidoc[tag=store-kibana-user] +include::{xes-repo-dir}/security/get-started-kibana-users.asciidoc[tag=store-kibana-user] -- . Start {kib}. diff --git a/x-pack/docs/en/security/securing-communications/tutorial-tls-intro.asciidoc b/x-pack/docs/en/security/securing-communications/tutorial-tls-intro.asciidoc index 31bed2f3a0eaf..9cb1b9f21631f 100644 --- a/x-pack/docs/en/security/securing-communications/tutorial-tls-intro.asciidoc +++ b/x-pack/docs/en/security/securing-communications/tutorial-tls-intro.asciidoc @@ -40,7 +40,7 @@ IMPORTANT: To complete this tutorial, you must install the default {es} and When you install these products, they apply basic licenses with no expiration dates. All of the subsequent steps in this tutorial assume that you are using a basic license. For more information, see {subscriptions} and -<>. +{stack-ov}/license-management.html[License management]. include::tutorial-tls-certificates.asciidoc[] include::tutorial-tls-internode.asciidoc[] diff --git a/x-pack/docs/en/security/troubleshooting.asciidoc b/x-pack/docs/en/security/troubleshooting.asciidoc new file mode 100644 index 0000000000000..3141270839933 --- /dev/null +++ b/x-pack/docs/en/security/troubleshooting.asciidoc @@ -0,0 +1,782 @@ +[role="xpack"] +[[security-troubleshooting]] +== Troubleshooting security +++++ +Troubleshooting +++++ + +Use the information in this section to troubleshoot common problems and find +answers for frequently asked questions. + +* <> +* <> +* <> +* <> +* <> +* <> +* <> +* <> +* <> +* <> +* <> +* <> + + +For issues that you cannot fix yourself … we’re here to help. +If you are an existing Elastic customer with a support contract, please create +a ticket in the +https://support.elastic.co/customers/s/login/[Elastic Support portal]. +Or post in the https://discuss.elastic.co/[Elastic forum]. + +[[security-trb-settings]] +=== Some settings are not returned via the nodes settings API + +*Symptoms:* + +* When you use the {ref}/cluster-nodes-info.html[nodes info API] to retrieve +settings for a node, some information is missing. + +*Resolution:* + +This is intentional. Some of the settings are considered to be highly +sensitive: all `ssl` settings, ldap `bind_dn`, and `bind_password`. +For this reason, we filter these settings and do not expose them via +the nodes info API rest endpoint. You can also define additional +sensitive settings that should be hidden using the +`xpack.security.hide_settings` setting. For example, this snippet +hides the `url` settings of the `ldap1` realm and all settings of the +`ad1` realm. + +[source, yaml] +------------------------------------------ +xpack.security.hide_settings: xpack.security.authc.realms.ldap1.url, +xpack.security.authc.realms.ad1.* +------------------------------------------ + +[[security-trb-roles]] +=== Authorization exceptions + +*Symptoms:* + +* I configured the appropriate roles and the users, but I still get an +authorization exception. +* I can authenticate to LDAP, but I still get an authorization exception. + + +*Resolution:* + +. Verify that the role names associated with the users match the roles defined +in the `roles.yml` file. You can use the `elasticsearch-users` tool to list all +the users. Any unknown roles are marked with `*`. ++ +-- +[source, shell] +------------------------------------------ +bin/elasticsearch-users list +rdeniro : admin +alpacino : power_user +jacknich : monitoring,unknown_role* <1> +------------------------------------------ +<1> `unknown_role` was not found in `roles.yml` + +For more information about this command, see the +{ref}/users-command.html[`elasticsearch-users` command]. +-- + +. If you are authenticating to LDAP, a number of configuration options can cause +this error. ++ +-- +|====================== +|_group identification_ | + +Groups are located by either an LDAP search or by the "memberOf" attribute on +the user. Also, If subtree search is turned off, it will search only one +level deep. See the <> for all the options. +There are many options here and sticking to the defaults will not work for all +scenarios. + +| _group to role mapping_| + +Either the `role_mapping.yml` file or the location for this file could be +misconfigured. For more information, see {ref}/security-files.html[Security files]. + +|_role definition_| + +The role definition might be missing or invalid. + +|====================== + +To help track down these possibilities, add the following lines to the end of +the `log4j2.properties` configuration file in the `ES_PATH_CONF`: + +[source,properties] +---------------- +logger.authc.name = org.elasticsearch.xpack.security.authc +logger.authc.level = DEBUG +---------------- + +A successful authentication should produce debug statements that list groups and +role mappings. +-- + +[[security-trb-extraargs]] +=== Users command fails due to extra arguments + +*Symptoms:* + +* The `elasticsearch-users` command fails with the following message: +`ERROR: extra arguments [...] were provided`. + +*Resolution:* + +This error occurs when the `elasticsearch-users` tool is parsing the input and +finds unexpected arguments. This can happen when there are special characters +used in some of the arguments. For example, on Windows systems the `,` character +is considered a parameter separator; in other words `-r role1,role2` is +translated to `-r role1 role2` and the `elasticsearch-users` tool only +recognizes `role1` as an expected parameter. The solution here is to quote the +parameter: `-r "role1,role2"`. + +For more information about this command, see +{ref}/users-command.html[`elasticsearch-users` command]. + +[[trouble-shoot-active-directory]] +=== Users are frequently locked out of Active Directory + +*Symptoms:* + +* Certain users are being frequently locked out of Active Directory. + +*Resolution:* + +Check your realm configuration; realms are checked serially, one after another. +If your Active Directory realm is being checked before other realms and there +are usernames that appear in both Active Directory and another realm, a valid +login for one realm might be causing failed login attempts in another realm. + +For example, if `UserA` exists in both Active Directory and a file realm, and +the Active Directory realm is checked first and file is checked second, an +attempt to authenticate as `UserA` in the file realm would first attempt to +authenticate against Active Directory and fail, before successfully +authenticating against the `file` realm. Because authentication is verified on +each request, the Active Directory realm would be checked - and fail - on each +request for `UserA` in the `file` realm. In this case, while the authentication +request completed successfully, the account on Active Directory would have +received several failed login attempts, and that account might become +temporarily locked out. Plan the order of your realms accordingly. + +Also note that it is not typically necessary to define multiple Active Directory +realms to handle domain controller failures. When using Microsoft DNS, the DNS +entry for the domain should always point to an available domain controller. + + +[[trb-security-maccurl]] +=== Certificate verification fails for curl on Mac + +*Symptoms:* + +* `curl` on the Mac returns a certificate verification error even when the +`--cacert` option is used. + + +*Resolution:* + +Apple's integration of `curl` with their keychain technology disables the +`--cacert` option. +See http://curl.haxx.se/mail/archive-2013-10/0036.html for more information. + +You can use another tool, such as `wget`, to test certificates. Alternately, you +can add the certificate for the signing certificate authority MacOS system +keychain, using a procedure similar to the one detailed at the +http://support.apple.com/kb/PH14003[Apple knowledge base]. Be sure to add the +signing CA's certificate and not the server's certificate. + + +[[trb-security-sslhandshake]] +=== SSLHandshakeException causes connections to fail + +*Symptoms:* + +* A `SSLHandshakeException` causes a connection to a node to fail and indicates +that there is a configuration issue. Some of the common exceptions are shown +below with tips on how to resolve these issues. + + +*Resolution:* + +`java.security.cert.CertificateException: No name matching node01.example.com found`:: ++ +-- +Indicates that a client connection was made to `node01.example.com` but the +certificate returned did not contain the name `node01.example.com`. In most +cases, the issue can be resolved by ensuring the name is specified during +certificate creation. For more information, see <>. Another scenario is +when the environment does not wish to use DNS names in certificates at all. In +this scenario, all settings in `elasticsearch.yml` should only use IP addresses +including the `network.publish_host` setting. +-- + +`java.security.cert.CertificateException: No subject alternative names present`:: ++ +-- +Indicates that a client connection was made to an IP address but the returned +certificate did not contain any `SubjectAlternativeName` entries. IP addresses +are only used for hostname verification if they are specified as a +`SubjectAlternativeName` during certificate creation. If the intent was to use +IP addresses for hostname verification, then the certificate will need to be +regenerated with the appropriate IP address. See <>. +-- + +`javax.net.ssl.SSLHandshakeException: null cert chain` and `javax.net.ssl.SSLException: Received fatal alert: bad_certificate`:: ++ +-- +The `SSLHandshakeException` indicates that a self-signed certificate was +returned by the client that is not trusted as it cannot be found in the +`truststore` or `keystore`. This `SSLException` is seen on the client side of +the connection. +-- + +`sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target` and `javax.net.ssl.SSLException: Received fatal alert: certificate_unknown`:: ++ +-- +This `SunCertPathBuilderException` indicates that a certificate was returned +during the handshake that is not trusted. This message is seen on the client +side of the connection. The `SSLException` is seen on the server side of the +connection. The CA certificate that signed the returned certificate was not +found in the `keystore` or `truststore` and needs to be added to trust this +certificate. +-- + +`javax.net.ssl.SSLHandshakeException: Invalid ECDH ServerKeyExchange signature`:: ++ +-- +The `Invalid ECDH ServerKeyExchange signature` can indicate that a key and a corresponding certificate don't match and are +causing the handshake to fail. +Verify the contents of each of the files you are using for your configured certificate authorities, certificates and keys. In particular, check that the key and certificate belong to the same key pair. +-- + +[[trb-security-ssl]] +=== Common SSL/TLS exceptions + +*Symptoms:* + +* You might see some exceptions related to SSL/TLS in your logs. Some of the +common exceptions are shown below with tips on how to resolve these issues. + + + + +*Resolution:* + +`WARN: received plaintext http traffic on a https channel, closing connection`:: ++ +-- +Indicates that there was an incoming plaintext http request. This typically +occurs when an external applications attempts to make an unencrypted call to the +REST interface. Please ensure that all applications are using `https` when +calling the REST interface with SSL enabled. +-- + +`org.elasticsearch.common.netty.handler.ssl.NotSslRecordException: not an SSL/TLS record:`:: ++ +-- +Indicates that there was incoming plaintext traffic on an SSL connection. This +typically occurs when a node is not configured to use encrypted communication +and tries to connect to nodes that are using encrypted communication. Please +verify that all nodes are using the same setting for +`xpack.security.transport.ssl.enabled`. + +For more information about this setting, see +{ref}/security-settings.html[Security Settings in {es}]. +-- + +`java.io.StreamCorruptedException: invalid internal transport message format, got`:: ++ +-- +Indicates an issue with data received on the transport interface in an unknown +format. This can happen when a node with encrypted communication enabled +connects to a node that has encrypted communication disabled. Please verify that +all nodes are using the same setting for `xpack.security.transport.ssl.enabled`. + +For more information about this setting, see +{ref}/security-settings.html[Security Settings in {es}]. +-- + +`java.lang.IllegalArgumentException: empty text`:: ++ +-- +This exception is typically seen when a `https` request is made to a node that +is not using `https`. If `https` is desired, please ensure the following setting +is in `elasticsearch.yml`: + +[source,yaml] +---------------- +xpack.security.http.ssl.enabled: true +---------------- + +For more information about this setting, see +{ref}/security-settings.html[Security Settings in {es}]. +-- + +`ERROR: unsupported ciphers [...] were requested but cannot be used in this JVM`:: ++ +-- +This error occurs when a SSL/TLS cipher suite is specified that cannot supported +by the JVM that {es} is running in. Security tries to use the specified cipher +suites that are supported by this JVM. This error can occur when using the +Security defaults as some distributions of OpenJDK do not enable the PKCS11 +provider by default. In this case, we recommend consulting your JVM +documentation for details on how to enable the PKCS11 provider. + +Another common source of this error is requesting cipher suites that use +encrypting with a key length greater than 128 bits when running on an Oracle JDK. +In this case, you must install the +<>. +-- + +[[trb-security-kerberos]] +=== Common Kerberos exceptions + +*Symptoms:* + +* User authentication fails due to either GSS negotiation failure +or a service login failure (either on the server or in the {es} http client). +Some of the common exceptions are listed below with some tips to help resolve +them. + +*Resolution:* + +`Failure unspecified at GSS-API level (Mechanism level: Checksum failed)`:: ++ +-- + +When you see this error message on the HTTP client side, then it may be +related to an incorrect password. + +When you see this error message in the {es} server logs, then it may be +related to the {es} service keytab. The keytab file is present but it failed +to log in as the user. Please check the keytab expiry. Also check whether the +keytab contain up-to-date credentials; if not, replace them. + +You can use tools like `klist` or `ktab` to list principals inside +the keytab and validate them. You can use `kinit` to see if you can acquire +initial tickets using the keytab. Please check the tools and their documentation +in your Kerberos environment. + +Kerberos depends on proper hostname resolution, so please check your DNS infrastructure. +Incorrect DNS setup, DNS SRV records or configuration for KDC servers in `krb5.conf` +can cause problems with hostname resolution. + +-- + +`Failure unspecified at GSS-API level (Mechanism level: Request is a replay (34))`:: + +`Failure unspecified at GSS-API level (Mechanism level: Clock skew too great (37))`:: ++ +-- + +To prevent replay attacks, Kerberos V5 sets a maximum tolerance for computer +clock synchronization and it is typically 5 minutes. Please check whether +the time on the machines within the domain is in sync. + +-- + +`gss_init_sec_context() failed: An unsupported mechanism was requested`:: + +`No credential found for: 1.2.840.113554.1.2.2 usage: Accept`:: ++ +-- + +You would usually see this error message on the client side when using `curl` to +test {es} Kerberos setup. For example, these messages occur when you are using +an old version of curl on the client and therefore Kerberos Spnego support is missing. +The Kerberos realm in {es} only supports Spengo mechanism (Oid 1.3.6.1.5.5.2); +it does not yet support Kerberos mechanism (Oid 1.2.840.113554.1.2.2). + +Make sure that: + +* You have installed curl version 7.49 or above as older versions of curl have +known Kerberos bugs. + +* The curl installed on your machine has `GSS-API`, `Kerberos` and `SPNEGO` +features listed when you invoke command `curl -V`. If not, you will need to +compile `curl` version with this support. + +To download latest curl version visit https://curl.haxx.se/download.html + +-- + +As Kerberos logs are often cryptic in nature and many things can go wrong +as it depends on external services like DNS and NTP. You might +have to enable additional debug logs to determine the root cause of the issue. + +{es} uses a JAAS (Java Authentication and Authorization Service) Kerberos login +module to provide Kerberos support. To enable debug logs on {es} for the login +module use following Kerberos realm setting: +[source,yaml] +---------------- +xpack.security.authc.realms..krb.debug: true +---------------- + +For detailed information, see {ref}/security-settings.html#ref-kerberos-settings[Kerberos realm settings]. + +Sometimes you may need to go deeper to understand the problem during SPNEGO +GSS context negotiation or look at the Kerberos message exchange. To enable +Kerberos/SPNEGO debug logging on JVM, add following JVM system properties: + +`-Dsun.security.krb5.debug=true` + +`-Dsun.security.spnego.debug=true` + +For more information about JVM system properties, see {ref}/jvm-options.html[configuring JVM options]. + +[[trb-security-saml]] +=== Common SAML issues + +Some of the common SAML problems are shown below with tips on how to resolve +these issues. + +. *Symptoms:* ++ +-- +Authentication in {kib} fails and the following error is printed in the {es} +logs: + +.... +Cannot find any matching realm for [SamlPrepareAuthenticationRequest{realmName=saml1, +assertionConsumerServiceURL=https://my.kibana.url/api/security/v1/saml}] +.... + +*Resolution:* + +In order to initiate a SAML authentication, {kib} needs to know which SAML realm +it should use from the ones that are configured in {es}. You can use the +`xpack.security.authc.saml.reaml` setting to explicitly set the SAML realm name +in {kib}. It must match the name of the SAML realm that is configured in {es}. + +If you get an error like the one above, it possibly means that the value of +`xpack.security.authc.saml.reaml` in your {kib} configuration is wrong. Verify +that it matches the name of the configured realm in {es}, which is the string +after `xpack.security.authc.realms.saml.` in your {es} configuration. + +-- + +. *Symptoms:* ++ +-- +Authentication in {kib} fails and the following error is printed in the +{es} logs: + +.... +Authentication to realm saml1 failed - Provided SAML response is not valid for realm +saml/saml1 (Caused by ElasticsearchSecurityException[Conditions [https://some-url-here...] +do not match required audience [https://my.kibana.url]]) +.... + +*Resolution:* + +We received a SAML response that is addressed to another SAML Service Provider. +This usually means that the configured SAML Service Provider Entity ID in +`elasticsearch.yml` (`sp.entity_id`) does not match what has been configured as +the SAML Service Provider Entity ID in the SAML Identity Provider documentation. + +To resolve this issue, ensure that both the saml realm in {es} and the IdP are +configured with the same string for the SAML Entity ID of the Service Provider. + +TIP: These strings are compared as case-sensitive strings and not as +canonicalized URLs even when the values are URL-like. Be mindful of trailing +slashes, port numbers, etc. + +-- + +. *Symptoms:* ++ +-- +Authentication in {kib} fails and the following error is printed in the +{es} logs: + +.... +Cannot find metadata for entity [your:entity.id] in [metadata.xml] +.... + +*Resolution:* + +We could not find the metadata for the SAML Entity ID `your:entity.id` in the +configured metadata file (`metadata.xml`). + +.. Ensure that the `metadata.xml` file you are using is indeed the one provided +by your SAML Identity Provider. +.. Ensure that the `metadata.xml` file contains one element +as follows: `] +for action [cluster:admin/xpack/security/saml/authenticate] +.... + +*Resolution:* + +This error indicates that {es} failed to process the incoming SAML +authentication message. Since the message can't be processed, {es} is not aware +of who the to-be authenticated user is and the `` +placeholder is used instead. To diagnose the _actual_ problem, you must check +the {es} logs for further details. +-- + +. *Symptoms:* ++ +-- +Authentication in {kib} fails and the following error is printed in the +{es} logs: + +.... +Authentication to realm my-saml-realm failed - +Provided SAML response is not valid for realm saml/my-saml-realm +(Caused by ElasticsearchSecurityException[SAML Response is not a 'success' response: + The SAML IdP did not grant the request. It indicated that the Elastic Stack side sent + something invalid (urn:oasis:names:tc:SAML:2.0:status:Requester). Specific status code which might + indicate what the issue is: [urn:oasis:names:tc:SAML:2.0:status:InvalidNameIDPolicy]] +) +.... + +*Resolution:* + +This means that the SAML Identity Provider failed to authenticate the user and +sent a SAML Response to the Service Provider ({stack}) indicating this failure. +The message will convey whether the SAML Identity Provider thinks that the problem +is with the Service Provider ({stack}) or with the Identity Provider itself and +the specific status code that follows is extremely useful as it usually indicates +the underlying issue. The list of specific error codes is defined in the +https://docs.oasis-open.org/security/saml/v2.0/saml-core-2.0-os.pdf[SAML 2.0 Core specification - Section 3.2.2.2] +and the most commonly encountered ones are: + +. `urn:oasis:names:tc:SAML:2.0:status:AuthnFailed`: The SAML Identity Provider failed to + authenticate the user. There is not much to troubleshoot on the {stack} side for this status, the logs of + the SAML Identity Provider will hopefully offer much more information. +. `urn:oasis:names:tc:SAML:2.0:status:InvalidNameIDPolicy`: The SAML Identity Provider cannot support + releasing a NameID with the requested format. When creating SAML Authentication Requests, {es} sets + the NameIDPolicy element of the Authentication request with the appropriate value. This is controlled + by the {ref}/security-settings.html#ref-saml-settings[`nameid_format`] configuration parameter in + `elasticsearch.yml`, which if not set defaults to `urn:oasis:names:tc:SAML:2.0:nameid-format:transient`. + This instructs the Identity Provider to return a NameID with that specific format in the SAML Response. If + the SAML Identity Provider cannot grant that request, for example because it is configured to release a + NameID format with `urn:oasis:names:tc:SAML:2.0:nameid-format:persistent` format instead, it returns this error + indicating an invalid NameID policy. This issue can be resolved by adjusting `nameid_format` to match the format + the SAML Identity Provider can return or by setting it to `urn:oasis:names:tc:SAML:2.0:nameid-format:unspecified` + so that the Identity Provider is allowed to return any format it wants. +-- + +. *Symptoms:* ++ +-- +Authentication in {kib} fails and the following error is printed in the +{es} logs: + +.... +The XML Signature of this SAML message cannot be validated. Please verify that the saml +realm uses the correct SAMLmetadata file/URL for this Identity Provider +.... + +*Resolution:* + +This means that {es} failed to validate the digital signature of the SAML +message that the Identity Provider sent. {es} uses the public key of the +Identity Provider that is included in the SAML metadata, in order to validate +the signature that the IdP has created using its corresponding private key. +Failure to do so, can have a number of causes: + +.. As the error message indicates, the most common cause is that the wrong +metadata file is used and as such the public key it contains doesn't correspond +to the private key the Identity Provider uses. +.. The configuration of the Identity Provider has changed or the key has been +rotated and the metadata file that {es} is using has not been updated. +.. The SAML Response has been altered in transit and the signature cannot be +validated even though the correct key is used. + +NOTE: The private keys and public keys and self-signed X.509 certificates that +are used in SAML for digital signatures as described above have no relation to +the keys and certificates that are used for TLS either on the transport or the +http layer. A failure such as the one described above has nothing to do with +your `xpack.ssl` related configuration. + +-- + +. *Symptoms:* ++ +-- +Users are unable to login with a local username and password in {kib} because +SAML is enabled. + +*Resolution:* + +If you want your users to be able to use local credentials to authenticate to +{kib} in addition to using the SAML realm for Single Sign-On, you must enable +the `basic` `authProvider` in {kib}. The process is documented in the +<> +-- + +*Logging:* + +Very detailed trace logging can be enabled specifically for the SAML realm by +setting the following transient setting: + +[source, shell] +----------------------------------------------- +PUT /_cluster/settings +{ + "transient": { + "logger.org.elasticsearch.xpack.security.authc.saml": "trace" + } +} +----------------------------------------------- + + +Alternatively, you can add the following lines to the end of the +`log4j2.properties` configuration file in the `ES_PATH_CONF`: + +[source,properties] +---------------- +logger.saml.name = org.elasticsearch.xpack.security.authc.saml +logger.saml.level = TRACE +---------------- + +[[trb-security-internalserver]] +=== Internal Server Error in Kibana + +*Symptoms:* + +* In 5.1.1, an `UnhandledPromiseRejectionWarning` occurs and {kib} displays an +Internal Server Error. +//TBD: Is the same true for later releases? + +*Resolution:* + +If the Security plugin is enabled in {es} but disabled in {kib}, you must +still set `elasticsearch.username` and `elasticsearch.password` in `kibana.yml`. +Otherwise, {kib} cannot connect to {es}. + + +[[trb-security-setup]] +=== Setup-passwords command fails due to connection failure + +The {ref}/setup-passwords.html[elasticsearch-setup-passwords command] sets +passwords for the built-in users by sending user management API requests. If +your cluster uses SSL/TLS for the HTTP (REST) interface, the command attempts to +establish a connection with the HTTPS protocol. If the connection attempt fails, +the command fails. + +*Symptoms:* + +. {es} is running HTTPS, but the command fails to detect it and returns the +following errors: ++ +-- +[source, shell] +------------------------------------------ +Cannot connect to elasticsearch node. +java.net.SocketException: Unexpected end of file from server +... +ERROR: Failed to connect to elasticsearch at +http://127.0.0.1:9200/_security/_authenticate?pretty. +Is the URL correct and elasticsearch running? +------------------------------------------ +-- + +. SSL/TLS is configured, but trust cannot be established. The command returns +the following errors: ++ +-- +[source, shell] +------------------------------------------ +SSL connection to +https://127.0.0.1:9200/_security/_authenticate?pretty +failed: sun.security.validator.ValidatorException: +PKIX path building failed: +sun.security.provider.certpath.SunCertPathBuilderException: +unable to find valid certification path to requested target +Please check the elasticsearch SSL settings under +xpack.security.http.ssl. +... +ERROR: Failed to establish SSL connection to elasticsearch at +https://127.0.0.1:9200/_security/_authenticate?pretty. +------------------------------------------ +-- + +. The command fails because hostname verification fails, which results in the +following errors: ++ +-- +[source, shell] +------------------------------------------ +SSL connection to +https://idp.localhost.test:9200/_security/_authenticate?pretty +failed: java.security.cert.CertificateException: +No subject alternative DNS name matching +elasticsearch.example.com found. +Please check the elasticsearch SSL settings under +xpack.security.http.ssl. +... +ERROR: Failed to establish SSL connection to elasticsearch at +https://elasticsearch.example.com:9200/_security/_authenticate?pretty. +------------------------------------------ +-- + +*Resolution:* + +. If your cluster uses TLS/SSL for the HTTP interface but the +`elasticsearch-setup-passwords` command attempts to establish a non-secure +connection, use the `--url` command option to explicitly specify an HTTPS URL. +Alternatively, set the `xpack.security.http.ssl.enabled` setting to `true`. + +. If the command does not trust the {es} server, verify that you configured the +`xpack.security.http.ssl.certificate_authorities` setting or the +`xpack.security.http.ssl.truststore.path` setting. + +. If hostname verification fails, you can disable this verification by setting +`xpack.security.http.ssl.verification_mode` to `certificate`. + +For more information about these settings, see +{ref}/security-settings.html[Security Settings in {es}]. + +[[trb-security-path]] +=== Failures due to relocation of the configuration files + +*Symptoms:* + +* Active Directory or LDAP realms might stop working after upgrading to {es} 6.3 +or later releases. In 6.4 or later releases, you might see messages in the {es} +log that indicate a config file is in a deprecated location. + +*Resolution:* + +By default, in 6.2 and earlier releases, the security configuration files are +located in the `ES_PATH_CONF/x-pack` directory, where `ES_PATH_CONF` is an +environment variable that defines the location of the +{ref}/settings.html#config-files-location[config directory]. + +In 6.3 and later releases, the config directory no longer contains an `x-pack` +directory. The files that were stored in this folder, such as the +`log4j2.properties`, `role_mapping.yml`, `roles.yml`, `users`, and `users_roles` +files, now exist directly in the config directory. + +IMPORTANT: If you upgraded to 6.3 or later releases, your old security +configuration files still exist in an `x-pack` folder. That file path is +deprecated, however, and you should move your files out of that folder. + +In 6.3 and later releases, settings such as `files.role_mapping` default to +`ES_PATH_CONF/role_mapping.yml`. If you do not want to use the default locations, +you must update the settings appropriately. See +{ref}/security-settings.html[Security settings in {es}]. +