Skip to content

[@kbn/es] Add a predefined list of Elasticsearch secure settings to be added into keystore. Re-enable OIDC tests.#42239

Merged
azasypkin merged 5 commits intoelastic:masterfrom
azasypkin:issue-36959-kbn-es-secure
Aug 1, 2019
Merged

[@kbn/es] Add a predefined list of Elasticsearch secure settings to be added into keystore. Re-enable OIDC tests.#42239
azasypkin merged 5 commits intoelastic:masterfrom
azasypkin:issue-36959-kbn-es-secure

Conversation

@azasypkin
Copy link
Contributor

@azasypkin azasypkin commented Jul 30, 2019

This PR adds a predefined list of Elasticsearch settings that should be added into keystore instead of config or set of command line arguments.

Additionally I tweaked @kbn/test a bit to pass esArgs into installSource/Snapshot and re-enabled OIDC tests that rely on this functionality (and hence implicitly test it).


See that OIDC tests are run on the CI now


Fixes: #36959

…e added into keystore. Re-enable OIDC tests.
@azasypkin azasypkin added test dev Team:Operations Kibana-Operations Team Team:Security Platform Security: Auth, Users, Roles, Spaces, Audit Logging, etc t// release_note:skip Skip the PR/issue when compiling release notes v7.4.0 v7.3.1 labels Jul 30, 2019
@elasticmachine
Copy link
Contributor

Pinging @elastic/kibana-operations

@elasticmachine
Copy link
Contributor

Pinging @elastic/kibana-security

log: this._log,
}).reduce((acc, cur) => acc.concat(['-E', cur]), []);
const args = parseSettings(
extractConfigFiles(options.esArgs || [], installPath, { log: this._log }),
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

note: I'm using parseSettngs here to not touch extractConfigFiles.

await configureKeystore(installPath, password, log, bundledJDK);
await configureKeystore(
installPath,
password,
Copy link
Contributor Author

@azasypkin azasypkin Jul 30, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

note: decided to keep password separate from secureSettings even though the code that adds secure settings could be reused for password as well since are kind of different things. What do you think?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I actually really like the idea of combing them and just manually passing [['bootstrap.plassword', password], ...parseSettings(esArgs, { filter: SettingsFilter.SecureOnly })] to configureKeystore

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You got it!

});

for (const [secureSettingName, secureSettingValue] of secureSettings) {
log.info(`setting secure setting [${secureSettingName}] to %s`, chalk.bold(secureSettingValue));
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

note: setting .... setting not sure how that sounds 🤷‍♂️ Couldn't find a better verb.

Copy link
Member

@tylersmalley tylersmalley Jul 31, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit, but we could keep this consistent with the previous output:

Suggested change
log.info(`setting secure setting [${secureSettingName}] to %s`, chalk.bold(secureSettingValue));
log.info(`setting secure setting %s to %s`, chalk.bold(secureSettingName), chalk.bold(secureSettingValue));

* List of the patterns for the settings names that are supposed to be secure and stored in the keystore.
*/
const SECURE_SETTINGS_LIST = [
/^xpack\.security\.authc\.realms\.oidc\.[a-zA-Z0-9_]+\.rp\.client_secret$/,
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

note: didn't realize it before, but realm name is a custom string, so had to resort to reqular expression here.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's fine - I was thinking we could also just match rp.client_secret but this is better.

export enum SettingsFilter {
All = 'all',
SecureOnly = 'secure-only',
NonSecureOnly = 'non-secure-only',
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

note: NonSecureOnly is kind of misnomer, also was considering ExcludeSecure instead, but don't have a strong opinion.

for (const rawSettingNameValuePair of rawSettingNameValuePairs) {
const [settingName, settingValue] = rawSettingNameValuePair.split('=');

const includeSetting =
Copy link
Contributor Author

@azasypkin azasypkin Jul 30, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

note: if I understand JS correctly isSecureSetting should never be called twice, so there shouldn't be overhead (unless a filter is undefined/arbitrary string that shouldn't be allowed in TS, but can be set in JS though).

* under the License.
*/

require('../src/setup_node_env');
Copy link
Contributor Author

@azasypkin azasypkin Jul 30, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

note: not sure if it's the correct solution - but I wanted to use TypeScript in kbn/es. Let me know if there is a better solution or if you prefer me to switch to JavaScript instead.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's fine, as long as it doesn't have to be transpiled.

@@ -0,0 +1,63 @@
/*
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

note: I see we call these settings as esArgs, but I picked the word settings as it's used in Secure Settings.

@elasticmachine

This comment has been minimized.

@elasticmachine
Copy link
Contributor

💚 Build Succeeded

Copy link
Member

@tylersmalley tylersmalley left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, just a couple of small nits I could live without.

@elasticmachine
Copy link
Contributor

💚 Build Succeeded

@azasypkin azasypkin merged commit 1a103c0 into elastic:master Aug 1, 2019
@azasypkin azasypkin deleted the issue-36959-kbn-es-secure branch August 1, 2019 07:14
@azasypkin
Copy link
Contributor Author

azasypkin commented Aug 1, 2019

7.3/7.3.1: 837bcdd
7.x/7.4.0: 7ded0b6

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backported dev release_note:skip Skip the PR/issue when compiling release notes Team:Operations Kibana-Operations Team Team:Security Platform Security: Auth, Users, Roles, Spaces, Audit Logging, etc t// test v7.3.1 v7.4.0

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants