[Security Solution] Allow to specify custom servers and security for the result OpenAPI bundle#189348
Conversation
servers and security for the result bundleservers and security for the result OpenAPI bundle
e3d3681 to
d22c1d7
Compare
|
Pinging @elastic/security-detections-response (Team:Detections and Resp) |
|
Pinging @elastic/security-solution (Team: SecuritySolution) |
|
Pinging @elastic/security-detection-rule-management (Team:Detection Rule Management) |
xcrzx
left a comment
There was a problem hiding this comment.
LGTM 👍 Left just a couple of nits.
There was a problem hiding this comment.
Should these two bundler calls also be wrapped in an IIFE?
There was a problem hiding this comment.
It looks the same as in bundler, tet's deduplicate this logic
There was a problem hiding this comment.
Is this condition relevant? It doesn't look like anything other than an object can be returned from readFile.
There was a problem hiding this comment.
Generally JSON/YAML files may contain an array, primitive values or an empty string. These files are invalid OpenAPI documents. Even not all objects are valid OpenAPI documents but that validation is omitted here since it looks for specific keys in that object.
There was a problem hiding this comment.
Shouldn't version be overridable by the prototypeDoc.info.version?
There was a problem hiding this comment.
bundle utility groups OpenAPI specs by info.version. blankOasDocumentFactory constructs a blank OpenAPI document with a desired info.vesion.
merge utility on the other hand should produce result bundles with specified version.
2b555f2 to
9fe2267
Compare
💛 Build succeeded, but was flaky
Failed CI StepsMetrics [docs]Unknown metric groupsESLint disabled in files
Total ESLint disabled count
History
To update your PR or re-run it, just comment with: cc @maximpn |
…for the result OpenAPI bundle (elastic#189348) **Resolves:** elastic#189269 **Resolves:** elastic#189270 This PR adds an ability to specify OpenAPI `servers` and security requirements (`security`) to be used in the result bundle. `servers` and/or `security` in the source OpenAPI specs are be dropped when custom `servers` and/or `security` provided. Kibana is usually deployed at a single access point and manages authentication in a central way. That way it's much more convenient to have control on what `servers` and `security` are present in the result bundles. It will help to avoid conflicts, duplicates and update them in centralized way. This PR extends OpenAPI bundler configuration options with `prototypeDocument`. "Prototype" in the name means it's a prototype for the result. The bundler uses certain properties from that prototype OpenAPI document to add them to the result OpenAPI bundle. The following properties are used - `info` representing OpenAPI Info object (former `options.specInfo`) - `servers` OpenAPI Server Object Array - `security` + `components.securitySchemes` OpenAPI Security Requirement Object Array + OpenAPI Security Schemes Object (validation checks that both fields are set otherwise an error is thrown) For convenience `prototypeDocument` could be specified as a string path to a file containing prototype OpenAPI document. `prototypeDocument` can be specified for `bundle` and `merge` utilities like the following **bundle** ```js const { bundle } = require('@kbn/openapi-bundler'); (async () => { await bundle({ sourceGlob: 'source/glob/*.yaml', outputFilePath: 'output/bundle.yaml, options: { prototypeDocument: { info: { title: 'Some title', description: 'Some description', }, servers: [{ url: 'https://{kibana_url}', variables: { kibana_url: { default: 'localhost:5601', } } }], security: [{ ApiKeyAuth: [] }], components: { securitySchemes: { ApiKeyAuth: { type: 'apiKey', in: 'header', name: 'Authorization', } } } }, }, }); ``` **bundle** with external prototype document ```js const { bundle } = require('@kbn/openapi-bundler'); (async () => { await bundle({ sourceGlob: 'source/glob/*.yaml', outputFilePath: 'output/bundle.yaml, options: { prototypeDocument: 'path/to/prototype_document.yaml',, }, }); ``` **merge** ```js const { merge } = require('@kbn/openapi-bundler'); (async () => { await merge({ sourceGlobs: [ 'absolute/path/to/file.yaml`, 'some/glob/*.schema.yaml', ], outputFilePath: 'output/file/path/bundle.yaml', options: { prototypeDocument: { info: { title: 'Some title', description: 'Some description', }, servers: [{ url: 'https://{kibana_url}', variables: { kibana_url: { default: 'localhost:5601', } } }], security: [{ ApiKeyAuth: [] }], components: { securitySchemes: { ApiKeyAuth: { type: 'apiKey', in: 'header', name: 'Authorization', } } } }, }, }); })(); ``` **merge** with external prototype document ```js const { merge } = require('@kbn/openapi-bundler'); (async () => { await merge({ sourceGlobs: [ 'absolute/path/to/file.yaml`, 'some/glob/*.schema.yaml', ], outputFilePath: 'output/file/path/bundle.yaml', options: { prototypeDocument: 'path/to/prototype_document.yaml', }, }); })(); ``` The result bundles will contain specified `servers` and `security` while source `servers` and `security` will be dropped.
…189262) (#190541) # Backport This will backport the bundler aspects of the following commits from `main` to `8.15`: - [[HTTP/OAS] Merge OpenAPI specs by using `kbn-openapi-bundler` (#189262)](#189262) - #189621 - #189482 - #189348 - #189472 - #188110 - #188812 <!--- Backport version: 8.9.8 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sqren/backport) <!--BACKPORT [{"author":{"name":"Maxim Palenov","email":"maxim.palenov@elastic.co"},"sourceCommit":{"committedDate":"2024-08-13T10:45:35Z","message":"[HTTP/OAS] Merge OpenAPI specs by using `kbn-openapi-bundler` (#189262)\n\n**Addresses:** https://github.com/elastic/kibana/issues/186356\r\n**Relates to:** https://github.com/elastic/kibana/issues/184428\r\n\r\n## Summary\r\n\r\nThis PR adds a merging JS script based on the utility implemented in #186356. Resulted OpenAPI bundle as committed in `oas_docs/output/kibana.serverless.bundled.yaml`.\r\n\r\n## Details\r\n\r\nhttps://github.com//pull/188110 implements and exposes `merge` utility design to merge source OpenAPI specs without processing. It's has only a programmatic API. To merge OpenAPI specs it's required to add a JS script like below\r\n\r\n```js\r\nconst { merge } = require('@kbn/openapi-bundler');\r\n\r\n(async () => {\r\n await merge({\r\n sourceGlobs: [/* a list of source globs goes here */],\r\n outputFilePath: 'path/to/the/output/file.yaml',\r\n });\r\n})();\r\n```\r\n\r\nThe JS script added in this PR includes source OpenAPI specs presented in `oas_docs/makefile` plus Security Solution OpenAPI specs based on https://github.com/elastic/kibana/issues/184428.\r\n\r\n**To run** the script use the following command from Kibana root folder\r\n\r\n```bash\r\nnode ./oas_docs/scripts/merge_serverless_oas.js \r\n```\r\n\r\n## Known linting issues with Security Solution OpenAPI specs\r\n\r\nRunning Spectral OpenAPI linter on the result bundle shows a number of errors caused by `no-$ref-siblings` rule. This caused by the current code generator implementation which requires `default` property to be set next to `$ref` though it's not correct for OpenAPI `3.0.3` while it's allowed in `3.1`. It seems that Bump.sh handles such cases properly though by properly showing a default value.\r\n\r\nWe need to analyze the problem and decide if/when we should fix it.\r\n\r\nThe rest of warnings look fixable and will be addressed in the next stage after setting up linter rules.\r\n\r\n## Next steps\r\n\r\nSince `@kbn/openapi-bundler` package is tailored specifically for Kibana it should replace Redocly currently used to merge OpenAPI specs. It also means `oas_docs/makefile` should be superseded by JS script(s) using `merge` utility form `@kbn/openapi-bundler` package.\r\n\r\n`@kbn/openapi-bundler` SHOULD NOT replace OpenAPI linters since it doesn't perform thorough linting. It's good if we continue adopting `spectral-cli` for linting purposes.","sha":"7a2e7bef9644eaf04fd820d16b8ee137b3c00f2b","branchLabelMapping":{"^v8.16.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["Team:Core","release_note:skip","backport:skip","docs","Team: SecuritySolution","Feature:OAS","v8.16.0"],"number":189262,"url":"https://github.com/elastic/kibana/pull/189262","mergeCommit":{"message":"[HTTP/OAS] Merge OpenAPI specs by using `kbn-openapi-bundler` (#189262)\n\n**Addresses:** https://github.com/elastic/kibana/issues/186356\r\n**Relates to:** https://github.com/elastic/kibana/issues/184428\r\n\r\n## Summary\r\n\r\nThis PR adds a merging JS script based on the utility implemented in #186356. Resulted OpenAPI bundle as committed in `oas_docs/output/kibana.serverless.bundled.yaml`.\r\n\r\n## Details\r\n\r\nhttps://github.com//pull/188110 implements and exposes `merge` utility design to merge source OpenAPI specs without processing. It's has only a programmatic API. To merge OpenAPI specs it's required to add a JS script like below\r\n\r\n```js\r\nconst { merge } = require('@kbn/openapi-bundler');\r\n\r\n(async () => {\r\n await merge({\r\n sourceGlobs: [/* a list of source globs goes here */],\r\n outputFilePath: 'path/to/the/output/file.yaml',\r\n });\r\n})();\r\n```\r\n\r\nThe JS script added in this PR includes source OpenAPI specs presented in `oas_docs/makefile` plus Security Solution OpenAPI specs based on https://github.com/elastic/kibana/issues/184428.\r\n\r\n**To run** the script use the following command from Kibana root folder\r\n\r\n```bash\r\nnode ./oas_docs/scripts/merge_serverless_oas.js \r\n```\r\n\r\n## Known linting issues with Security Solution OpenAPI specs\r\n\r\nRunning Spectral OpenAPI linter on the result bundle shows a number of errors caused by `no-$ref-siblings` rule. This caused by the current code generator implementation which requires `default` property to be set next to `$ref` though it's not correct for OpenAPI `3.0.3` while it's allowed in `3.1`. It seems that Bump.sh handles such cases properly though by properly showing a default value.\r\n\r\nWe need to analyze the problem and decide if/when we should fix it.\r\n\r\nThe rest of warnings look fixable and will be addressed in the next stage after setting up linter rules.\r\n\r\n## Next steps\r\n\r\nSince `@kbn/openapi-bundler` package is tailored specifically for Kibana it should replace Redocly currently used to merge OpenAPI specs. It also means `oas_docs/makefile` should be superseded by JS script(s) using `merge` utility form `@kbn/openapi-bundler` package.\r\n\r\n`@kbn/openapi-bundler` SHOULD NOT replace OpenAPI linters since it doesn't perform thorough linting. It's good if we continue adopting `spectral-cli` for linting purposes.","sha":"7a2e7bef9644eaf04fd820d16b8ee137b3c00f2b"}},"sourceBranch":"main","suggestedTargetBranches":[],"targetPullRequestStates":[{"branch":"main","label":"v8.16.0","labelRegex":"^v8.16.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/189262","number":189262,"mergeCommit":{"message":"[HTTP/OAS] Merge OpenAPI specs by using `kbn-openapi-bundler` (#189262)\n\n**Addresses:** https://github.com/elastic/kibana/issues/186356\r\n**Relates to:** https://github.com/elastic/kibana/issues/184428\r\n\r\n## Summary\r\n\r\nThis PR adds a merging JS script based on the utility implemented in #186356. Resulted OpenAPI bundle as committed in `oas_docs/output/kibana.serverless.bundled.yaml`.\r\n\r\n## Details\r\n\r\nhttps://github.com//pull/188110 implements and exposes `merge` utility design to merge source OpenAPI specs without processing. It's has only a programmatic API. To merge OpenAPI specs it's required to add a JS script like below\r\n\r\n```js\r\nconst { merge } = require('@kbn/openapi-bundler');\r\n\r\n(async () => {\r\n await merge({\r\n sourceGlobs: [/* a list of source globs goes here */],\r\n outputFilePath: 'path/to/the/output/file.yaml',\r\n });\r\n})();\r\n```\r\n\r\nThe JS script added in this PR includes source OpenAPI specs presented in `oas_docs/makefile` plus Security Solution OpenAPI specs based on https://github.com/elastic/kibana/issues/184428.\r\n\r\n**To run** the script use the following command from Kibana root folder\r\n\r\n```bash\r\nnode ./oas_docs/scripts/merge_serverless_oas.js \r\n```\r\n\r\n## Known linting issues with Security Solution OpenAPI specs\r\n\r\nRunning Spectral OpenAPI linter on the result bundle shows a number of errors caused by `no-$ref-siblings` rule. This caused by the current code generator implementation which requires `default` property to be set next to `$ref` though it's not correct for OpenAPI `3.0.3` while it's allowed in `3.1`. It seems that Bump.sh handles such cases properly though by properly showing a default value.\r\n\r\nWe need to analyze the problem and decide if/when we should fix it.\r\n\r\nThe rest of warnings look fixable and will be addressed in the next stage after setting up linter rules.\r\n\r\n## Next steps\r\n\r\nSince `@kbn/openapi-bundler` package is tailored specifically for Kibana it should replace Redocly currently used to merge OpenAPI specs. It also means `oas_docs/makefile` should be superseded by JS script(s) using `merge` utility form `@kbn/openapi-bundler` package.\r\n\r\n`@kbn/openapi-bundler` SHOULD NOT replace OpenAPI linters since it doesn't perform thorough linting. It's good if we continue adopting `spectral-cli` for linting purposes.","sha":"7a2e7bef9644eaf04fd820d16b8ee137b3c00f2b"}}]}] BACKPORT--> --------- Co-authored-by: Maxim Palenov <maxim.palenov@elastic.co>
Resolves: #189269
Resolves: #189270
Summary
This PR adds an ability to specify OpenAPI
serversand security requirements (security) to be used in the result bundle.serversand/orsecurityin the source OpenAPI specs are be dropped when customserversand/orsecurityprovided.Details
Kibana is usually deployed at a single access point and manages authentication in a central way. That way it's much more convenient to have control on what
serversandsecurityare present in the result bundles. It will help to avoid conflicts, duplicates and update them in centralized way.This PR extends OpenAPI bundler configuration options with
prototypeDocument. "Prototype" in the name means it's a prototype for the result. The bundler uses certain properties from that prototype OpenAPI document to add them to the result OpenAPI bundle. The following properties are usedinforepresenting OpenAPI Info object (formeroptions.specInfo)serversOpenAPI Server Object Arraysecurity+components.securitySchemesOpenAPI Security Requirement Object Array + OpenAPI Security Schemes Object (validation checks that both fields are set otherwise an error is thrown)For convenience
prototypeDocumentcould be specified as a string path to a file containing prototype OpenAPI document.How to test?
prototypeDocumentcan be specified forbundleandmergeutilities like the followingbundle
bundle with external prototype document
merge
merge with external prototype document
The result bundles will contain specified
serversandsecuritywhile sourceserversandsecuritywill be dropped.