diff --git a/packages/kbn-openapi-bundler/README.md b/packages/kbn-openapi-bundler/README.md index cfe165c49c447..8844806c6461c 100644 --- a/packages/kbn-openapi-bundler/README.md +++ b/packages/kbn-openapi-bundler/README.md @@ -1,20 +1,24 @@ # OpenAPI Specs Bundler for Kibana -`@kbn/openapi-bundler` is a tool for transforming multiple OpenAPI specification files (source specs) into a bundled specification file(s) (target spec). The number of resulting bundles depends on a number of versions -used in the OpenAPI specification files. The package can be used for API documentation generation purposes. This approach allows you to: - -- Abstract away the knowledge of where you keep your OpenAPI specs, how many specs are there, and how to find them. Consumer should only know where result files (bundles) are located. -- Omit internal API endpoints from the bundle. -- Omit API endpoints that are hidden behind a feature flag and haven't been released yet. -- Omit parts of schemas that are hidden behind a feature flag (e.g. a new property added to an existing response schema). -- Omit custom OpenAPI attributes from the bundle, such as `x-codegen-enabled`, `x-internal`, `x-modify` and `x-labels`. -- Include only dedicated OpenAPI operation objects (a.k.a HTTP verbs) into the result bundle by labeling them via `x-labels` - and using `includeLabels` bundler option, e.g. produce separate ESS and Serverless bundles -- Transform the target schema according to the custom OpenAPI attributes, such as `x-modify`. -- Resolve references, inline some of them and merge `allOf` object schemas for better readability. The bundled file contains only local references and paths. -- Group OpenAPI specs by version (OpenAPI's `info.version`) and produce a separate bundle for each group - -## Getting started +This packages provides tooling for manipulating OpenAPI endpoint specifications. It has two tools exposes + +- **OpenAPI bundler** is a tool for transforming multiple OpenAPI specification files (source specs) into a bundled specification file(s) (target spec). The number of resulting bundles depends on a number of versions + used in the OpenAPI specification files. The package can be used for API documentation generation purposes. This approach allows you to: + + - Abstract away the knowledge of where you keep your OpenAPI specs, how many specs are there, and how to find them. Consumer should only know where result files (bundles) are located. + - Omit internal API endpoints from the bundle. + - Omit API endpoints that are hidden behind a feature flag and haven't been released yet. + - Omit parts of schemas that are hidden behind a feature flag (e.g. a new property added to an existing response schema). + - Omit custom OpenAPI attributes from the bundle, such as `x-codegen-enabled`, `x-internal`, `x-modify` and `x-labels`. + - Include only dedicated OpenAPI operation objects (a.k.a HTTP verbs) into the result bundle by labeling them via `x-labels` + and using `includeLabels` bundler option, e.g. produce separate ESS and Serverless bundles + - Transform the target schema according to the custom OpenAPI attributes, such as `x-modify`. + - Resolve references, inline some of them and merge `allOf` object schemas for better readability. The bundled file contains only local references and paths. + - Group OpenAPI specs by version (OpenAPI's `info.version`) and produce a separate bundle for each group + +- **OpenAPI merger** is a tool for merging multiple OpenAPI specification files. It's useful to merge already processed specification files to produce a result bundle. **OpenAPI bundler** uses the merger under the hood to merge bundled OpenAPI specification files. Exposed externally merger is a wrapper of the bundler's merger but extended with an ability to parse JSON files and forced to produce a single result file. + +## Getting started with OpenAPI bundling To let this package help you with bundling your OpenAPI specifications you should have OpenAPI specification describing your API endpoint request and response schemas along with common types used in your API. Refer [@kbn/openapi-generator](../kbn-openapi-generator/README.md) and [OpenAPI 3.0.3](https://swagger.io/specification/v3/) (support for [OpenAPI 3.1.0](https://swagger.io/specification/) is planned to be added later) for more details. @@ -163,6 +167,48 @@ components: securitySchemes: ... ``` +## Getting started with OpenAPI merger + +To let this package help you with merging OpenAPI specifications you should have valid OpenAPI specifications version `3.0.x`. OpenAPI `3.1` is not supported currently. + +Currently package supports only programmatic API. As the next step you need to create a JavaScript script file like below + +```ts +require('../../src/setup_node_env'); +const { resolve } = require('path'); +const { merge } = require('@kbn/openapi-bundler'); +const { REPO_ROOT } = require('@kbn/repo-info'); + +(async () => { + await merge({ + sourceGlobs: [ + `${REPO_ROOT}/my/path/to/spec1.json`, + `${REPO_ROOT}/my/path/to/spec2.yml`, + `${REPO_ROOT}/my/path/to/spec3.yaml`, + ], + outputFilePath: `${REPO_ROOT}/oas_docs/bundle.serverless.yaml`, + mergedSpecInfo: { + title: 'Kibana Serverless', + version: '1.0.0', + }, + }); +})(); +``` + +Finally you should be able to run OpenAPI merger via + +```bash +node ./path/to/the/script.js +``` + +or it could be added to a `package.json` and run via `yarn`. + +After running the script it will log different information and write a merged OpenAPI specification to a the provided path. + +### Caveats + +Merger shows an error when it's unable to merge some OpenAPI specifications. There is a possibility that references with the same name are defined in two or more files or there are endpoints of different versions and different parameters. Additionally top level `$ref` in path items, path item's `requestBody` and each response in `responses` aren't supported. + ## Multiple API versions declared via OpenAPI's `info.version` Serverless brought necessity for versioned HTTP API endpoints. We started with a single `2023-10-31` version. In some point diff --git a/packages/kbn-openapi-bundler/index.ts b/packages/kbn-openapi-bundler/index.ts index badd58def955e..b070eadf89710 100644 --- a/packages/kbn-openapi-bundler/index.ts +++ b/packages/kbn-openapi-bundler/index.ts @@ -7,3 +7,4 @@ */ export * from './src/openapi_bundler'; +export * from './src/openapi_merger'; diff --git a/packages/kbn-openapi-bundler/src/bundler/bundle_document.ts b/packages/kbn-openapi-bundler/src/bundler/bundle_document.ts index ca53afffdf010..3b827a15c90f0 100644 --- a/packages/kbn-openapi-bundler/src/bundler/bundle_document.ts +++ b/packages/kbn-openapi-bundler/src/bundler/bundle_document.ts @@ -26,6 +26,7 @@ import { import { createIncludeLabelsProcessor } from './process_document/document_processors/include_labels'; import { BundleRefProcessor } from './process_document/document_processors/bundle_refs'; import { RemoveUnusedComponentsProcessor } from './process_document/document_processors/remove_unused_components'; +import { insertRefByPointer } from '../utils/insert_by_json_pointer'; export class SkipException extends Error { constructor(public documentPath: string, message: string) { @@ -33,10 +34,6 @@ export class SkipException extends Error { } } -export interface BundledDocument extends ResolvedDocument { - bundledRefs: ResolvedRef[]; -} - interface BundleDocumentOptions { includeLabels?: string[]; } @@ -58,7 +55,7 @@ interface BundleDocumentOptions { export async function bundleDocument( absoluteDocumentPath: string, options?: BundleDocumentOptions -): Promise { +): Promise { if (!isAbsolute(absoluteDocumentPath)) { throw new Error( `bundleDocument expects an absolute document path but got "${absoluteDocumentPath}"` @@ -114,7 +111,9 @@ export async function bundleDocument( ); } - return { ...resolvedDocument, bundledRefs: Array.from(bundleRefsProcessor.getBundledRefs()) }; + injectBundledRefs(resolvedDocument, bundleRefsProcessor.getBundledRefs()); + + return resolvedDocument; } interface MaybeObjectWithPaths { @@ -128,3 +127,12 @@ function hasPaths(document: MaybeObjectWithPaths): boolean { Object.keys(document.paths).length > 0 ); } + +function injectBundledRefs( + resolvedDocument: ResolvedDocument, + refs: IterableIterator +): void { + for (const ref of refs) { + insertRefByPointer(ref.pointer, ref.refNode, resolvedDocument.document); + } +} diff --git a/packages/kbn-openapi-bundler/src/bundler/merge_documents/enrich_with_version_mime_param.ts b/packages/kbn-openapi-bundler/src/bundler/merge_documents/enrich_with_version_mime_param.ts new file mode 100644 index 0000000000000..d3d7c6e44885e --- /dev/null +++ b/packages/kbn-openapi-bundler/src/bundler/merge_documents/enrich_with_version_mime_param.ts @@ -0,0 +1,82 @@ +/* + * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one + * or more contributor license agreements. Licensed under the Elastic License + * 2.0 and the Server Side Public License, v 1; you may not use this file except + * in compliance with, at your election, the Elastic License 2.0 or the Server + * Side Public License, v 1. + */ + +import { OpenAPIV3 } from 'openapi-types'; +import { ResolvedDocument } from '../ref_resolver/resolved_document'; +import { isRefNode } from '../process_document'; +import { getOasDocumentVersion } from '../../utils/get_oas_document_version'; +import { KNOWN_HTTP_METHODS } from './http_methods'; + +export function enrichWithVersionMimeParam(resolvedDocuments: ResolvedDocument[]): void { + for (const resolvedDocument of resolvedDocuments) { + const version = getOasDocumentVersion(resolvedDocument); + const paths = resolvedDocument.document.paths as OpenAPIV3.PathsObject; + + for (const path of Object.keys(paths ?? {})) { + const pathItemObj = paths[path]; + + for (const httpVerb of KNOWN_HTTP_METHODS) { + const operationObj = pathItemObj?.[httpVerb]; + + if (operationObj?.requestBody && !isRefNode(operationObj.requestBody)) { + const requestBodyContent = operationObj.requestBody.content; + + enrichContentWithVersion(requestBodyContent, version); + } + + enrichCollection(operationObj?.responses ?? {}, version); + } + } + + if (resolvedDocument.document.components) { + const components = resolvedDocument.document.components as OpenAPIV3.ComponentsObject; + + if (components.requestBodies) { + enrichCollection(components.requestBodies, version); + } + + if (components.responses) { + enrichCollection(components.responses, version); + } + } + } +} + +function enrichCollection( + collection: Record< + string, + { content?: Record } | OpenAPIV3.ReferenceObject + >, + version: string +) { + for (const name of Object.keys(collection)) { + const obj = collection[name]; + + if (!obj || isRefNode(obj) || !obj.content) { + continue; + } + + enrichContentWithVersion(obj.content, version); + } +} + +function enrichContentWithVersion( + content: Record, + version: string +): void { + for (const mimeType of Object.keys(content)) { + if (mimeType.includes('; Elastic-Api-Version=')) { + continue; + } + + const mimeTypeWithVersionParam = `${mimeType}; Elastic-Api-Version=${version}`; + + content[mimeTypeWithVersionParam] = content[mimeType]; + delete content[mimeType]; + } +} diff --git a/packages/kbn-openapi-bundler/src/bundler/merge_documents/http_methods.ts b/packages/kbn-openapi-bundler/src/bundler/merge_documents/http_methods.ts new file mode 100644 index 0000000000000..4c3cbea9c53ff --- /dev/null +++ b/packages/kbn-openapi-bundler/src/bundler/merge_documents/http_methods.ts @@ -0,0 +1,20 @@ +/* + * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one + * or more contributor license agreements. Licensed under the Elastic License + * 2.0 and the Server Side Public License, v 1; you may not use this file except + * in compliance with, at your election, the Elastic License 2.0 or the Server + * Side Public License, v 1. + */ + +import { OpenAPIV3 } from 'openapi-types'; + +export const KNOWN_HTTP_METHODS = [ + OpenAPIV3.HttpMethods.HEAD, + OpenAPIV3.HttpMethods.GET, + OpenAPIV3.HttpMethods.POST, + OpenAPIV3.HttpMethods.PATCH, + OpenAPIV3.HttpMethods.PUT, + OpenAPIV3.HttpMethods.OPTIONS, + OpenAPIV3.HttpMethods.DELETE, + OpenAPIV3.HttpMethods.TRACE, +] as const; diff --git a/packages/kbn-openapi-bundler/src/bundler/merge_documents/merge_arrays.ts b/packages/kbn-openapi-bundler/src/bundler/merge_documents/merge_arrays.ts new file mode 100644 index 0000000000000..85ecf912896c4 --- /dev/null +++ b/packages/kbn-openapi-bundler/src/bundler/merge_documents/merge_arrays.ts @@ -0,0 +1,39 @@ +/* + * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one + * or more contributor license agreements. Licensed under the Elastic License + * 2.0 and the Server Side Public License, v 1; you may not use this file except + * in compliance with, at your election, the Elastic License 2.0 or the Server + * Side Public License, v 1. + */ + +/** + * Merges source arrays by merging array items and omitting duplicates. + * Duplicates checked by exacts match. + */ +export function mergeArrays(sources: Array): T[] { + const merged: T[] = []; + const seen = new Set(); + + for (const itemsSource of sources) { + for (const item of itemsSource) { + const searchableItem = toString(item); + + if (seen.has(searchableItem)) { + continue; + } + + merged.push(item); + seen.add(searchableItem); + } + } + + return merged; +} + +function toString(value: unknown): string { + try { + return JSON.stringify(value); + } catch { + throw new Error('Unable to merge arrays - encountered value is not serializable'); + } +} diff --git a/packages/kbn-openapi-bundler/src/bundler/merge_documents/merge_documents.ts b/packages/kbn-openapi-bundler/src/bundler/merge_documents/merge_documents.ts index 03275dbf3f3de..8e745c50ac679 100644 --- a/packages/kbn-openapi-bundler/src/bundler/merge_documents/merge_documents.ts +++ b/packages/kbn-openapi-bundler/src/bundler/merge_documents/merge_documents.ts @@ -8,28 +8,55 @@ import chalk from 'chalk'; import { OpenAPIV3 } from 'openapi-types'; -import { logger } from '../../logger'; -import { BundledDocument } from '../bundle_document'; +import { ResolvedDocument } from '../ref_resolver/resolved_document'; import { mergePaths } from './merge_paths'; import { mergeSharedComponents } from './merge_shared_components'; +import { mergeServers } from './merge_servers'; +import { mergeSecurityRequirements } from './merge_security_requirements'; +import { mergeTags } from './merge_tags'; +import { getOasVersion } from '../../utils/get_oas_version'; +import { getOasDocumentVersion } from '../../utils/get_oas_document_version'; +import { enrichWithVersionMimeParam } from './enrich_with_version_mime_param'; + +export interface MergeDocumentsOptions { + splitDocumentsByVersion: boolean; +} export async function mergeDocuments( - bundledDocuments: BundledDocument[], - blankOasFactory: (oasVersion: string, apiVersion: string) => OpenAPIV3.Document + resolvedDocuments: ResolvedDocument[], + blankOasDocumentFactory: (oasVersion: string, apiVersion: string) => OpenAPIV3.Document, + options: MergeDocumentsOptions ): Promise> { - const bundledDocumentsByVersion = splitByVersion(bundledDocuments); + const documentsByVersion = options.splitDocumentsByVersion + ? splitByVersion(resolvedDocuments) + : new Map([['', resolvedDocuments]]); const mergedByVersion = new Map(); - for (const [apiVersion, singleVersionBundledDocuments] of bundledDocumentsByVersion.entries()) { - const oasVersion = extractOasVersion(singleVersionBundledDocuments); - const mergedDocument = blankOasFactory(oasVersion, apiVersion); + if (!options.splitDocumentsByVersion) { + enrichWithVersionMimeParam(resolvedDocuments); + } - mergedDocument.paths = mergePaths(singleVersionBundledDocuments); - mergedDocument.components = { - // Copy components defined in the blank OpenAPI document - ...mergedDocument.components, - ...mergeSharedComponents(singleVersionBundledDocuments), - }; + for (const [apiVersion, documentsGroup] of documentsByVersion.entries()) { + validateSameOasVersion(documentsGroup); + + const oasVersion = getOasVersion(documentsGroup[0]); + const mergedDocument = blankOasDocumentFactory(oasVersion, apiVersion); + // Any shared components defined in the blank OAS like `securitySchemes` should + // preserve in the result document. Passing this document in the merge pipeline + // is the simplest way to take initial components into account. + const documentsToMerge = [ + { + absolutePath: 'MERGED OpenAPI SPEC', + document: mergedDocument as unknown as ResolvedDocument['document'], + }, + ...documentsGroup, + ]; + + mergedDocument.servers = mergeServers(documentsToMerge); + mergedDocument.paths = mergePaths(documentsToMerge); + mergedDocument.components = mergeSharedComponents(documentsToMerge); + mergedDocument.security = mergeSecurityRequirements(documentsToMerge); + mergedDocument.tags = mergeTags(documentsToMerge); mergedByVersion.set(mergedDocument.info.version, mergedDocument); } @@ -37,65 +64,35 @@ export async function mergeDocuments( return mergedByVersion; } -function splitByVersion(bundledDocuments: BundledDocument[]): Map { - const splitBundledDocuments = new Map(); +function splitByVersion(resolvedDocuments: ResolvedDocument[]): Map { + const splitBundledDocuments = new Map(); - for (const bundledDocument of bundledDocuments) { - const documentInfo = bundledDocument.document.info as OpenAPIV3.InfoObject; - - if (!documentInfo.version) { - logger.warning(`OpenAPI version is missing in ${chalk.bold(bundledDocument.absolutePath)}`); - - continue; - } - - const versionBundledDocuments = splitBundledDocuments.get(documentInfo.version); + for (const resolvedDocument of resolvedDocuments) { + const version = getOasDocumentVersion(resolvedDocument); + const versionBundledDocuments = splitBundledDocuments.get(version); if (!versionBundledDocuments) { - splitBundledDocuments.set(documentInfo.version, [bundledDocument]); + splitBundledDocuments.set(version, [resolvedDocument]); } else { - versionBundledDocuments.push(bundledDocument); + versionBundledDocuments.push(resolvedDocument); } } return splitBundledDocuments; } -function extractOasVersion(bundledDocuments: BundledDocument[]): string { - if (bundledDocuments.length === 0) { - throw new Error('Empty bundled document list'); - } +function validateSameOasVersion(resolvedDocuments: ResolvedDocument[]): void { + const firstDocumentOasVersion = getOasVersion(resolvedDocuments[0]); - const firstBundledDocument = bundledDocuments[0]; - - for (let i = 1; i < bundledDocuments.length; ++i) { - if ( - !areOasVersionsEqual( - bundledDocuments[i].document.openapi as string, - firstBundledDocument.document.openapi as string - ) - ) { + for (let i = 1; i < resolvedDocuments.length; ++i) { + if (getOasVersion(resolvedDocuments[i]) !== firstDocumentOasVersion) { throw new Error( `OpenAPI specs must use the same OpenAPI version, encountered ${chalk.blue( - bundledDocuments[i].document.openapi - )} at ${chalk.bold(bundledDocuments[i].absolutePath)} does not match ${chalk.blue( - firstBundledDocument.document.openapi - )} at ${chalk.bold(firstBundledDocument.absolutePath)}` + resolvedDocuments[i].document.openapi + )} at ${chalk.bold(resolvedDocuments[i].absolutePath)} does not match ${chalk.blue( + resolvedDocuments[0].document.openapi + )} at ${chalk.bold(resolvedDocuments[0].absolutePath)}` ); } } - - const version = firstBundledDocument.document.openapi as string; - - // Automatically promote to the recent OAS 3.0 version which is 3.0.3 - // 3.0.3 is the version used in the specification https://swagger.io/specification/v3/ - return version < '3.0.3' ? '3.0.3' : version; -} - -/** - * Tells if versions are equal by comparing only major and minor OAS version parts - */ -function areOasVersionsEqual(versionA: string, versionB: string): boolean { - // versionA.substring(0, 3) results in `3.0` or `3.1` - return versionA.substring(0, 3) === versionB.substring(0, 3); } diff --git a/packages/kbn-openapi-bundler/src/bundler/merge_documents/merge_operations.ts b/packages/kbn-openapi-bundler/src/bundler/merge_documents/merge_operations.ts new file mode 100644 index 0000000000000..c7a4ae4edbd7f --- /dev/null +++ b/packages/kbn-openapi-bundler/src/bundler/merge_documents/merge_operations.ts @@ -0,0 +1,114 @@ +/* + * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one + * or more contributor license agreements. Licensed under the Elastic License + * 2.0 and the Server Side Public License, v 1; you may not use this file except + * in compliance with, at your election, the Elastic License 2.0 or the Server + * Side Public License, v 1. + */ + +import { omit } from 'lodash'; +import deepEqual from 'fast-deep-equal'; +import { OpenAPIV3 } from 'openapi-types'; +import { KNOWN_HTTP_METHODS } from './http_methods'; +import { isRefNode } from '../process_document'; + +export function mergeOperations( + sourcePathItem: OpenAPIV3.PathItemObject, + mergedPathItem: OpenAPIV3.PathItemObject +) { + for (const httpMethod of KNOWN_HTTP_METHODS) { + const sourceOperation = sourcePathItem[httpMethod]; + const mergedOperation = mergedPathItem[httpMethod]; + + if (!sourceOperation) { + continue; + } + + if (!mergedOperation || deepEqual(sourceOperation, mergedOperation)) { + mergedPathItem[httpMethod] = sourceOperation; + continue; + } + + mergeOperation(sourceOperation, mergedOperation); + } +} + +function mergeOperation( + sourceOperation: OpenAPIV3.OperationObject, + mergedOperation: OpenAPIV3.OperationObject +) { + if ( + !deepEqual( + omit(sourceOperation, ['requestBody', 'responses']), + omit(mergedOperation, ['requestBody', 'responses']) + ) + ) { + throw new Error('Operation objects are incompatible'); + } + + mergeRequestBody(sourceOperation, mergedOperation); + mergeResponses(sourceOperation, mergedOperation); +} + +function mergeRequestBody( + sourceOperation: OpenAPIV3.OperationObject, + mergedOperation: OpenAPIV3.OperationObject +): void { + if (isRefNode(sourceOperation.requestBody)) { + throw new Error('Request body top level $ref is not supported'); + } + + if (!sourceOperation.requestBody) { + return; + } + + if (!mergedOperation.requestBody) { + mergedOperation.requestBody = sourceOperation.requestBody; + return; + } + + const mergedOperationRequestBody = mergedOperation.requestBody as OpenAPIV3.RequestBodyObject; + + if (!mergedOperationRequestBody.content) { + mergedOperationRequestBody.content = {}; + } + + for (const mimeType of Object.keys(sourceOperation.requestBody.content)) { + if (mergedOperationRequestBody.content[mimeType]) { + throw new Error(`Duplicated request MIME type "${mimeType}". Please fix the conflict.`); + } + + mergedOperationRequestBody.content[mimeType] = sourceOperation.requestBody.content[mimeType]; + } +} + +function mergeResponses( + sourceOperation: OpenAPIV3.OperationObject, + mergedOperation: OpenAPIV3.OperationObject +): void { + for (const httpStatusCode of Object.keys(sourceOperation.responses)) { + const sourceResponseObj = sourceOperation.responses[httpStatusCode]; + + if (isRefNode(sourceResponseObj)) { + throw new Error('Response object top level $ref is not supported'); + } + + if (!sourceResponseObj.content) { + continue; + } + + const mergedResponseObj = mergedOperation.responses[httpStatusCode] as OpenAPIV3.ResponseObject; + + if (!mergedResponseObj.content) { + mergedResponseObj.content = {}; + } + + for (const mimeType of Object.keys(sourceResponseObj.content)) { + if (mergedResponseObj.content[mimeType]) { + throw new Error(`Duplicated response MIME type "${mimeType}". Please fix the conflict.`); + } + + mergedResponseObj.content[mimeType] = sourceResponseObj.content[mimeType]; + } + } +} diff --git a/packages/kbn-openapi-bundler/src/bundler/merge_documents/merge_paths.ts b/packages/kbn-openapi-bundler/src/bundler/merge_documents/merge_paths.ts index 1d541b5bb513e..d1a775e07b278 100644 --- a/packages/kbn-openapi-bundler/src/bundler/merge_documents/merge_paths.ts +++ b/packages/kbn-openapi-bundler/src/bundler/merge_documents/merge_paths.ts @@ -8,12 +8,15 @@ import chalk from 'chalk'; import { OpenAPIV3 } from 'openapi-types'; -import { BundledDocument } from '../bundle_document'; +import { ResolvedDocument } from '../ref_resolver/resolved_document'; +import { isRefNode } from '../process_document'; +import { mergeOperations } from './merge_operations'; +import { mergeArrays } from './merge_arrays'; -export function mergePaths(bundledDocuments: BundledDocument[]): OpenAPIV3.PathsObject { +export function mergePaths(resolvedDocuments: ResolvedDocument[]): OpenAPIV3.PathsObject { const mergedPaths: Record = {}; - for (const { absolutePath, document } of bundledDocuments) { + for (const { absolutePath, document } of resolvedDocuments) { if (!document.paths) { continue; } @@ -28,11 +31,15 @@ export function mergePaths(bundledDocuments: BundledDocument[]): OpenAPIV3.Paths const sourcePathItem = pathsObject[path]; const mergedPathItem = mergedPaths[path]; + if (isRefNode(sourcePathItem)) { + throw new Error('Path item top level $ref is not supported'); + } + try { mergeOptionalPrimitiveValue('summary', sourcePathItem, mergedPathItem); } catch { throw new Error( - `❌ Unable to bundle ${chalk.bold(absolutePath)} since ${chalk.bold( + `❌ Unable to merge ${chalk.bold(absolutePath)} due to ${chalk.bold( `paths.${path}.summary` )}'s value ${chalk.blue( sourcePathItem.summary @@ -44,7 +51,7 @@ export function mergePaths(bundledDocuments: BundledDocument[]): OpenAPIV3.Paths mergeOptionalPrimitiveValue('description', sourcePathItem, mergedPathItem); } catch { throw new Error( - `❌ Unable to bundle ${chalk.bold(absolutePath)} since ${chalk.bold( + `❌ Unable to merge ${chalk.bold(absolutePath)} due to ${chalk.bold( `paths.${path}.description` )}'s value ${chalk.blue( sourcePathItem.description @@ -56,18 +63,20 @@ export function mergePaths(bundledDocuments: BundledDocument[]): OpenAPIV3.Paths mergeOperations(sourcePathItem, mergedPathItem); } catch (e) { throw new Error( - `❌ Unable to bundle ${chalk.bold(absolutePath)} since ${chalk.bold( - `paths.${path}.${e.message}` - )}'s definition is duplicated and differs from previously encountered.` + `❌ Unable to merge ${chalk.bold(absolutePath)} due to an error in ${chalk.bold( + `paths.${path}` + )} occurred "${e.message}".` ); } + mergePathItemServers(sourcePathItem, mergedPathItem); + try { mergeParameters(sourcePathItem, mergedPathItem); } catch (e) { throw new Error( - `❌ Unable to bundle ${chalk.bold(absolutePath)} since ${chalk.bold( - `paths.${path}.parameters.[${e.message}]` + `❌ Unable to merge ${chalk.bold(absolutePath)} since ${chalk.bold( + `paths.${path}.servers.[${e.message}]` )}'s definition is duplicated and differs from previously encountered.` ); } @@ -77,34 +86,6 @@ export function mergePaths(bundledDocuments: BundledDocument[]): OpenAPIV3.Paths return mergedPaths; } -const KNOWN_HTTP_METHODS = [ - OpenAPIV3.HttpMethods.HEAD, - OpenAPIV3.HttpMethods.GET, - OpenAPIV3.HttpMethods.POST, - OpenAPIV3.HttpMethods.PATCH, - OpenAPIV3.HttpMethods.PUT, - OpenAPIV3.HttpMethods.OPTIONS, - OpenAPIV3.HttpMethods.DELETE, - OpenAPIV3.HttpMethods.TRACE, -]; - -function mergeOperations( - sourcePathItem: OpenAPIV3.PathItemObject, - mergedPathItem: OpenAPIV3.PathItemObject -) { - for (const httpMethod of KNOWN_HTTP_METHODS) { - if (!sourcePathItem[httpMethod]) { - continue; - } - - if (mergedPathItem[httpMethod]) { - throw new Error(httpMethod); - } - - mergedPathItem[httpMethod] = sourcePathItem[httpMethod]; - } -} - function mergeOptionalPrimitiveValue( fieldName: FieldName, source: { [field in FieldName]?: unknown }, @@ -123,6 +104,21 @@ function mergeOptionalPrimitiveValue( } } +function mergePathItemServers( + sourcePathItem: OpenAPIV3.PathItemObject, + mergedPathItem: OpenAPIV3.PathItemObject +): void { + if (!sourcePathItem.servers) { + return; + } + + if (!mergedPathItem.servers) { + mergedPathItem.servers = []; + } + + mergedPathItem.servers = mergeArrays([sourcePathItem.servers, mergedPathItem.servers]); +} + function mergeParameters( sourcePathItem: OpenAPIV3.PathItemObject, mergedPathItem: OpenAPIV3.PathItemObject @@ -136,9 +132,9 @@ function mergeParameters( } for (const sourceParameter of sourcePathItem.parameters) { - if ('$ref' in sourceParameter) { + if (isRefNode(sourceParameter)) { const existing = mergedPathItem.parameters.find( - (x) => '$ref' in x && x.$ref === sourceParameter.$ref + (x) => isRefNode(x) && x.$ref === sourceParameter.$ref ); if (existing) { @@ -146,7 +142,7 @@ function mergeParameters( } } else { const existing = mergedPathItem.parameters.find( - (x) => !('$ref' in x) && x.name === sourceParameter.name && x.in === sourceParameter.in + (x) => !isRefNode(x) && x.name === sourceParameter.name && x.in === sourceParameter.in ); if (existing) { diff --git a/packages/kbn-openapi-bundler/src/bundler/merge_documents/merge_security_requirements.ts b/packages/kbn-openapi-bundler/src/bundler/merge_documents/merge_security_requirements.ts new file mode 100644 index 0000000000000..b8714b27276dc --- /dev/null +++ b/packages/kbn-openapi-bundler/src/bundler/merge_documents/merge_security_requirements.ts @@ -0,0 +1,23 @@ +/* + * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one + * or more contributor license agreements. Licensed under the Elastic License + * 2.0 and the Server Side Public License, v 1; you may not use this file except + * in compliance with, at your election, the Elastic License 2.0 or the Server + * Side Public License, v 1. + */ + +import { OpenAPIV3 } from 'openapi-types'; +import { ResolvedDocument } from '../ref_resolver/resolved_document'; +import { mergeArrays } from './merge_arrays'; + +export function mergeSecurityRequirements( + resolvedDocuments: ResolvedDocument[] +): OpenAPIV3.SecurityRequirementObject[] | undefined { + const securityArrayOfArrays = resolvedDocuments + .filter(({ document }) => Array.isArray(document.security)) + .map(({ document }) => document.security as OpenAPIV3.SecurityRequirementObject[]); + + const merged = mergeArrays(securityArrayOfArrays); + + return merged.length > 0 ? merged : undefined; +} diff --git a/packages/kbn-openapi-bundler/src/bundler/merge_documents/merge_servers.ts b/packages/kbn-openapi-bundler/src/bundler/merge_documents/merge_servers.ts new file mode 100644 index 0000000000000..673f96d2aec6e --- /dev/null +++ b/packages/kbn-openapi-bundler/src/bundler/merge_documents/merge_servers.ts @@ -0,0 +1,23 @@ +/* + * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one + * or more contributor license agreements. Licensed under the Elastic License + * 2.0 and the Server Side Public License, v 1; you may not use this file except + * in compliance with, at your election, the Elastic License 2.0 or the Server + * Side Public License, v 1. + */ + +import { OpenAPIV3 } from 'openapi-types'; +import { ResolvedDocument } from '../ref_resolver/resolved_document'; +import { mergeArrays } from './merge_arrays'; + +export function mergeServers( + resolvedDocuments: ResolvedDocument[] +): OpenAPIV3.ServerObject[] | undefined { + const serverObjArrayOfArrays = resolvedDocuments + .filter(({ document }) => Array.isArray(document.servers)) + .map(({ document }) => document.servers as OpenAPIV3.ServerObject[]); + + const merged = mergeArrays(serverObjArrayOfArrays); + + return merged.length > 0 ? merged : undefined; +} diff --git a/packages/kbn-openapi-bundler/src/bundler/merge_documents/merge_shared_components.ts b/packages/kbn-openapi-bundler/src/bundler/merge_documents/merge_shared_components.ts index e7379c05b5b6e..f38341d3e0f94 100644 --- a/packages/kbn-openapi-bundler/src/bundler/merge_documents/merge_shared_components.ts +++ b/packages/kbn-openapi-bundler/src/bundler/merge_documents/merge_shared_components.ts @@ -6,49 +6,103 @@ * Side Public License, v 1. */ -import { OpenAPIV3 } from 'openapi-types'; -import deepEqual from 'fast-deep-equal'; import chalk from 'chalk'; -import { insertRefByPointer } from '../../utils/insert_by_json_pointer'; -import { ResolvedRef } from '../ref_resolver/resolved_ref'; -import { BundledDocument } from '../bundle_document'; +import deepEqual from 'fast-deep-equal'; +import { OpenAPIV3 } from 'openapi-types'; +import { ResolvedDocument } from '../ref_resolver/resolved_document'; +import { extractByJsonPointer } from '../../utils/extract_by_json_pointer'; +import { logger } from '../../logger'; + +const MERGEABLE_COMPONENT_TYPES = [ + 'schemas', + 'responses', + 'parameters', + 'examples', + 'requestBodies', + 'headers', + 'securitySchemes', + 'links', + 'callbacks', +] as const; export function mergeSharedComponents( - bundledDocuments: BundledDocument[] + bundledDocuments: ResolvedDocument[] ): OpenAPIV3.ComponentsObject { - const componentsMap = new Map(); const mergedComponents: Record = {}; - for (const bundledDocument of bundledDocuments) { - mergeRefsToMap(bundledDocument.bundledRefs, componentsMap); - } + for (const componentsType of MERGEABLE_COMPONENT_TYPES) { + const mergedTypedComponents = mergeObjects(bundledDocuments, `/components/${componentsType}`); + + if (Object.keys(mergedTypedComponents).length === 0) { + // Nothing was merged for this components type, go to the next component type + continue; + } - for (const resolvedRef of componentsMap.values()) { - insertRefByPointer(resolvedRef.pointer, resolvedRef.refNode, mergedComponents); + mergedComponents[componentsType] = mergedTypedComponents; } return mergedComponents; } -function mergeRefsToMap(bundledRefs: ResolvedRef[], componentsMap: Map): void { - for (const bundledRef of bundledRefs) { - const existingRef = componentsMap.get(bundledRef.pointer); +function mergeObjects( + resolvedDocuments: ResolvedDocument[], + sourcePointer: string +): Record { + const merged: Record = {}; + const componentNameSourceLocationMap = new Map(); + const mergedEntityName = sourcePointer.split('/').at(-1); - if (!existingRef) { - componentsMap.set(bundledRef.pointer, bundledRef); + for (const resolvedDocument of resolvedDocuments) { + const object = extractObjectToMerge(resolvedDocument, sourcePointer); + + if (!object) { continue; } - if (deepEqual(existingRef.refNode, bundledRef.refNode)) { - continue; + for (const name of Object.keys(object)) { + const componentToAdd = object[name]; + const existingComponent = merged[name]; + + if (existingComponent) { + const existingSchemaLocation = componentNameSourceLocationMap.get(name); + + if (deepEqual(componentToAdd, existingComponent)) { + logger.warning( + `Found a duplicate component ${chalk.yellow( + `${sourcePointer}/${name}` + )} defined in ${chalk.blue(resolvedDocument.absolutePath)} and in ${chalk.magenta( + existingSchemaLocation + )}.` + ); + } else { + throw new Error( + `❌ Unable to merge documents due to conflicts in referenced ${mergedEntityName}. Component ${chalk.yellow( + `${sourcePointer}/${name}` + )} is defined in ${chalk.blue(resolvedDocument.absolutePath)} and in ${chalk.magenta( + existingSchemaLocation + )} but has not matching definitions.` + ); + } + } + + merged[name] = componentToAdd; + componentNameSourceLocationMap.set(name, resolvedDocument.absolutePath); } + } + + return merged; +} - throw new Error( - `❌ Unable to bundle documents due to conflicts in references. Schema ${chalk.yellow( - bundledRef.pointer - )} is defined in ${chalk.blue(existingRef.absolutePath)} and in ${chalk.magenta( - bundledRef.absolutePath - )} but has not matching definitions.` +function extractObjectToMerge( + resolvedDocument: ResolvedDocument, + sourcePointer: string +): Record | undefined { + try { + return extractByJsonPointer(resolvedDocument.document, sourcePointer); + } catch (e) { + logger.debug( + `JSON pointer "${sourcePointer}" is not resolvable in ${resolvedDocument.absolutePath}` ); + return; } } diff --git a/packages/kbn-openapi-bundler/src/bundler/merge_documents/merge_tags.ts b/packages/kbn-openapi-bundler/src/bundler/merge_documents/merge_tags.ts new file mode 100644 index 0000000000000..e1b8411538deb --- /dev/null +++ b/packages/kbn-openapi-bundler/src/bundler/merge_documents/merge_tags.ts @@ -0,0 +1,23 @@ +/* + * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one + * or more contributor license agreements. Licensed under the Elastic License + * 2.0 and the Server Side Public License, v 1; you may not use this file except + * in compliance with, at your election, the Elastic License 2.0 or the Server + * Side Public License, v 1. + */ + +import { OpenAPIV3 } from 'openapi-types'; +import { ResolvedDocument } from '../ref_resolver/resolved_document'; +import { mergeArrays } from './merge_arrays'; + +export function mergeTags( + resolvedDocuments: ResolvedDocument[] +): OpenAPIV3.TagObject[] | undefined { + const tagsArrayOfArrays = resolvedDocuments + .filter(({ document }) => Array.isArray(document.tags)) + .map(({ document }) => document.tags as OpenAPIV3.TagObject[]); + + const merged = mergeArrays(tagsArrayOfArrays); + + return merged.length > 0 ? merged : undefined; +} diff --git a/packages/kbn-openapi-bundler/src/bundler/process_document/process_document.ts b/packages/kbn-openapi-bundler/src/bundler/process_document/process_document.ts index 907c42ebf6ea4..7606d5ce1d02b 100644 --- a/packages/kbn-openapi-bundler/src/bundler/process_document/process_document.ts +++ b/packages/kbn-openapi-bundler/src/bundler/process_document/process_document.ts @@ -7,7 +7,6 @@ */ import { dirname } from 'path'; -import { isPlainObject } from 'lodash'; import { IRefResolver } from '../ref_resolver/ref_resolver'; import { ResolvedDocument } from '../ref_resolver/resolved_document'; import { parseRef } from '../../utils/parse_ref'; @@ -146,8 +145,8 @@ function isTraversableNode(maybeTraversableNode: unknown): boolean { return typeof maybeTraversableNode === 'object' && maybeTraversableNode !== null; } -export function isRefNode(node: DocumentNode): node is { $ref: string } { - return isPlainObject(node) && '$ref' in node; +export function isRefNode(node: unknown): node is { $ref: string } { + return isPlainObjectType(node) && '$ref' in node; } function applyEnterProcessors( diff --git a/packages/kbn-openapi-bundler/src/openapi_bundler.ts b/packages/kbn-openapi-bundler/src/openapi_bundler.ts index 856f9aaeb9b98..d4a19d2806863 100644 --- a/packages/kbn-openapi-bundler/src/openapi_bundler.ts +++ b/packages/kbn-openapi-bundler/src/openapi_bundler.ts @@ -9,14 +9,15 @@ import chalk from 'chalk'; import { isUndefined, omitBy } from 'lodash'; import { OpenAPIV3 } from 'openapi-types'; -import globby from 'globby'; -import { basename, dirname, resolve } from 'path'; -import { BundledDocument, bundleDocument, SkipException } from './bundler/bundle_document'; +import { basename, dirname } from 'path'; +import { bundleDocument, SkipException } from './bundler/bundle_document'; import { mergeDocuments } from './bundler/merge_documents'; import { removeFilesByGlob } from './utils/remove_files_by_glob'; import { logger } from './logger'; -import { writeYamlDocument } from './utils/write_yaml_document'; import { createBlankOpenApiDocument } from './bundler/merge_documents/create_blank_oas_document'; +import { writeDocuments } from './utils/write_documents'; +import { ResolvedDocument } from './bundler/ref_resolver/resolved_document'; +import { resolveGlobs } from './utils/resolve_globs'; export interface BundlerConfig { sourceGlob: string; @@ -37,8 +38,7 @@ export const bundle = async ({ logger.debug(chalk.bold(`Bundling API route schemas`)); logger.debug(`👀 Searching for source files in ${chalk.underline(sourceGlob)}`); - const sourceFilesGlob = resolve(sourceGlob); - const schemaFilePaths = await globby([sourceFilesGlob]); + const schemaFilePaths = await resolveGlobs([sourceGlob]); logger.info(`🕵️‍♀️ Found ${schemaFilePaths.length} schemas`); logSchemas(schemaFilePaths); @@ -69,7 +69,9 @@ export const bundle = async ({ isUndefined ), }); - const resultDocumentsMap = await mergeDocuments(resolvedDocuments, blankOasFactory); + const resultDocumentsMap = await mergeDocuments(resolvedDocuments, blankOasFactory, { + splitDocumentsByVersion: true, + }); await writeDocuments(resultDocumentsMap, outputFilePath); }; @@ -83,7 +85,7 @@ function logSchemas(schemaFilePaths: string[]): void { async function resolveDocuments( schemaFilePaths: string[], options?: BundleOptions -): Promise { +): Promise { const resolvedDocuments = await Promise.all( schemaFilePaths.map(async (schemaFilePath) => { try { @@ -110,9 +112,9 @@ async function resolveDocuments( } function filterOutSkippedDocuments( - documents: Array -): BundledDocument[] { - const processedDocuments: BundledDocument[] = []; + documents: Array +): ResolvedDocument[] { + const processedDocuments: ResolvedDocument[] = []; for (const document of documents) { if (!document) { @@ -124,35 +126,3 @@ function filterOutSkippedDocuments( return processedDocuments; } - -async function writeDocuments( - resultDocumentsMap: Map, - outputFilePath: string -): Promise { - for (const [version, document] of resultDocumentsMap.entries()) { - const versionedOutputFilePath = getVersionedOutputFilePath(outputFilePath, version); - - try { - await writeYamlDocument(versionedOutputFilePath, document); - - logger.success(`📖 Wrote bundled OpenAPI specs to ${chalk.bold(versionedOutputFilePath)}`); - } catch (e) { - logger.error( - `Unable to save bundled document to ${chalk.bold(versionedOutputFilePath)}: ${e.message}` - ); - } - } -} - -function getVersionedOutputFilePath(outputFilePath: string, version: string): string { - const hasVersionPlaceholder = outputFilePath.indexOf('{version}') > -1; - const snakeCasedVersion = version.replaceAll(/[^\w\d]+/g, '_'); - - if (hasVersionPlaceholder) { - return outputFilePath.replace('{version}', snakeCasedVersion); - } - - const filename = basename(outputFilePath); - - return outputFilePath.replace(filename, `${version}-${filename}`); -} diff --git a/packages/kbn-openapi-bundler/src/openapi_merger.ts b/packages/kbn-openapi-bundler/src/openapi_merger.ts new file mode 100644 index 0000000000000..c5e67fe74f221 --- /dev/null +++ b/packages/kbn-openapi-bundler/src/openapi_merger.ts @@ -0,0 +1,102 @@ +/* + * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one + * or more contributor license agreements. Licensed under the Elastic License + * 2.0 and the Server Side Public License, v 1; you may not use this file except + * in compliance with, at your election, the Elastic License 2.0 or the Server + * Side Public License, v 1. + */ + +import chalk from 'chalk'; +import { OpenAPIV3 } from 'openapi-types'; +import { basename, extname } from 'path'; +import { mergeDocuments } from './bundler/merge_documents'; +import { logger } from './logger'; +import { createBlankOpenApiDocument } from './bundler/merge_documents/create_blank_oas_document'; +import { readYamlDocument } from './utils/read_yaml_document'; +import { readJsonDocument } from './utils/read_json_document'; +import { ResolvedDocument } from './bundler/ref_resolver/resolved_document'; +import { writeDocuments } from './utils/write_documents'; +import { resolveGlobs } from './utils/resolve_globs'; + +export interface MergerConfig { + sourceGlobs: string[]; + outputFilePath: string; + mergedSpecInfo?: Partial; +} + +export const merge = async ({ + sourceGlobs, + outputFilePath = 'merged.schema.yaml', + mergedSpecInfo, +}: MergerConfig) => { + if (sourceGlobs.length < 1) { + throw new Error('As minimum one source glob is expected'); + } + + logger.debug(chalk.bold(`Merging OpenAPI specs`)); + logger.debug( + `👀 Searching for source files in ${sourceGlobs + .map((glob) => chalk.underline(glob)) + .join(', ')}` + ); + + const schemaFilePaths = await resolveGlobs(sourceGlobs); + + logger.info(`🕵️‍♀️ Found ${schemaFilePaths.length} schemas`); + logSchemas(schemaFilePaths); + + logger.debug(`Merging schemas...`); + + const resolvedDocuments = await resolveDocuments(schemaFilePaths); + + const blankOasDocumentFactory = (oasVersion: string) => + createBlankOpenApiDocument(oasVersion, { + title: 'Merged OpenAPI specs', + version: 'not specified', + ...mergedSpecInfo, + }); + const resultDocumentsMap = await mergeDocuments(resolvedDocuments, blankOasDocumentFactory, { + splitDocumentsByVersion: false, + }); + // Only one document is expected when `splitDocumentsByVersion` is set to `false` + const mergedDocument = Array.from(resultDocumentsMap.values())[0]; + + // An empty string key prevents adding a version to a file name + await writeDocuments(new Map([['', mergedDocument]]), outputFilePath); +}; + +function logSchemas(schemaFilePaths: string[]): void { + for (const filePath of schemaFilePaths) { + logger.debug(`Found OpenAPI spec ${chalk.bold(filePath)}`); + } +} + +async function resolveDocuments(schemaFilePaths: string[]): Promise { + const resolvedDocuments = await Promise.all( + schemaFilePaths.map(async (schemaFilePath) => { + const extension = extname(schemaFilePath); + + logger.debug(`Reading ${chalk.bold(basename(schemaFilePath))}`); + + switch (extension) { + case '.yaml': + case '.yml': + return { + absolutePath: schemaFilePath, + document: await readYamlDocument(schemaFilePath), + }; + + case '.json': + return { + absolutePath: schemaFilePath, + document: await readJsonDocument(schemaFilePath), + }; + + default: + throw new Error(`${extension} files are not supported`); + } + }) + ); + + return resolvedDocuments; +} diff --git a/packages/kbn-openapi-bundler/src/utils/extract_by_json_pointer.ts b/packages/kbn-openapi-bundler/src/utils/extract_by_json_pointer.ts index 15507ac938ae1..937f708e0ce87 100644 --- a/packages/kbn-openapi-bundler/src/utils/extract_by_json_pointer.ts +++ b/packages/kbn-openapi-bundler/src/utils/extract_by_json_pointer.ts @@ -23,7 +23,7 @@ import { isPlainObjectType } from './is_plain_object_type'; */ export function extractByJsonPointer(document: unknown, pointer: string): Record { if (!pointer.startsWith('/')) { - throw new Error('$ref pointer must start with a leading slash'); + throw new Error('JSON pointer must start with a leading slash'); } if (!isPlainObjectType(document)) { @@ -36,7 +36,9 @@ export function extractByJsonPointer(document: unknown, pointer: string): Record const nextTarget = target[segment]; if (!isPlainObjectType(nextTarget)) { - throw new Error(`JSON Pointer "${pointer}" is not found in "${JSON.stringify(document)}"`); + throw new Error( + `JSON Pointer "${pointer}" is not resolvable in "${JSON.stringify(document)}"` + ); } target = nextTarget; diff --git a/packages/kbn-openapi-bundler/src/utils/get_oas_document_version.ts b/packages/kbn-openapi-bundler/src/utils/get_oas_document_version.ts new file mode 100644 index 0000000000000..bbc093241919d --- /dev/null +++ b/packages/kbn-openapi-bundler/src/utils/get_oas_document_version.ts @@ -0,0 +1,27 @@ +/* + * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one + * or more contributor license agreements. Licensed under the Elastic License + * 2.0 and the Server Side Public License, v 1; you may not use this file except + * in compliance with, at your election, the Elastic License 2.0 or the Server + * Side Public License, v 1. + */ + +import chalk from 'chalk'; +import { ResolvedDocument } from '../bundler/ref_resolver/resolved_document'; +import { isPlainObjectType } from './is_plain_object_type'; + +export function getOasDocumentVersion(resolvedDocument: ResolvedDocument): string { + if (!isPlainObjectType(resolvedDocument.document.info)) { + throw new Error( + `Required info object is not found in ${chalk.yellow(resolvedDocument.absolutePath)}` + ); + } + + if (typeof resolvedDocument.document.info.version !== 'string') { + throw new Error( + `Required info.version is not a string in ${chalk.yellow(resolvedDocument.absolutePath)}` + ); + } + + return resolvedDocument.document.info.version; +} diff --git a/packages/kbn-openapi-bundler/src/utils/get_oas_version.ts b/packages/kbn-openapi-bundler/src/utils/get_oas_version.ts new file mode 100644 index 0000000000000..9ca48afb09df7 --- /dev/null +++ b/packages/kbn-openapi-bundler/src/utils/get_oas_version.ts @@ -0,0 +1,24 @@ +/* + * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one + * or more contributor license agreements. Licensed under the Elastic License + * 2.0 and the Server Side Public License, v 1; you may not use this file except + * in compliance with, at your election, the Elastic License 2.0 or the Server + * Side Public License, v 1. + */ + +import chalk from 'chalk'; +import { ResolvedDocument } from '../bundler/ref_resolver/resolved_document'; + +export function getOasVersion(resolvedDocument: ResolvedDocument): string { + if (typeof resolvedDocument.document.openapi !== 'string') { + throw new Error( + `OpenAPI version field is not a string in ${chalk.yellow(resolvedDocument.absolutePath)}` + ); + } + + const version = resolvedDocument.document.openapi; + + // Automatically promote to the recent OAS 3.0 version which is 3.0.3 + // 3.0.3 is the version used in the specification https://swagger.io/specification/v3/ + return version < '3.0.3' ? '3.0.3' : version; +} diff --git a/packages/kbn-openapi-bundler/src/utils/insert_by_json_pointer.ts b/packages/kbn-openapi-bundler/src/utils/insert_by_json_pointer.ts index 161f548ad2cf9..c7228f8d03de0 100644 --- a/packages/kbn-openapi-bundler/src/utils/insert_by_json_pointer.ts +++ b/packages/kbn-openapi-bundler/src/utils/insert_by_json_pointer.ts @@ -7,7 +7,7 @@ */ /** - * Inserts `data` into the location specified by pointer in the `document`. + * Inserts `data` into the location specified by pointer in the `targetObject`. * * @param pointer [JSON Pointer](https://datatracker.ietf.org/doc/html/rfc6901) * @param component Component data to insert @@ -16,7 +16,7 @@ export function insertRefByPointer( pointer: string, component: unknown, - componentsObject: Record + targetObject: Record ): void { if (!pointer.startsWith('/components')) { throw new Error( @@ -24,9 +24,10 @@ export function insertRefByPointer( ); } - // splitting '/components' by '/' gives ['', 'components'] which should be skipped - const segments = pointer.split('/').slice(2); - let target = componentsObject; + // splitting '/components/some/path' by '/' gives ['', 'components'...] + // where the first empty string should be skipped + const segments = pointer.split('/').slice(1); + let target = targetObject; while (segments.length > 0) { const segment = segments.shift() as string; diff --git a/packages/kbn-openapi-bundler/src/utils/read_json_document.ts b/packages/kbn-openapi-bundler/src/utils/read_json_document.ts new file mode 100644 index 0000000000000..61ce61c6df3d8 --- /dev/null +++ b/packages/kbn-openapi-bundler/src/utils/read_json_document.ts @@ -0,0 +1,16 @@ +/* + * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one + * or more contributor license agreements. Licensed under the Elastic License + * 2.0 and the Server Side Public License, v 1; you may not use this file except + * in compliance with, at your election, the Elastic License 2.0 or the Server + * Side Public License, v 1. + */ + +import fs from 'fs/promises'; + +export async function readJsonDocument(filePath: string): Promise> { + // Typing load's result to Record is optimistic as we can't be sure + // there is object inside a yaml file. We don't have this validation layer so far + // but using JSON Schemas here should mitigate this problem. + return await JSON.parse(await fs.readFile(filePath, { encoding: 'utf8' })); +} diff --git a/packages/kbn-openapi-bundler/src/utils/resolve_globs.ts b/packages/kbn-openapi-bundler/src/utils/resolve_globs.ts new file mode 100644 index 0000000000000..83788011226bb --- /dev/null +++ b/packages/kbn-openapi-bundler/src/utils/resolve_globs.ts @@ -0,0 +1,17 @@ +/* + * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one + * or more contributor license agreements. Licensed under the Elastic License + * 2.0 and the Server Side Public License, v 1; you may not use this file except + * in compliance with, at your election, the Elastic License 2.0 or the Server + * Side Public License, v 1. + */ + +import globby from 'globby'; +import { resolve } from 'path'; + +export async function resolveGlobs(globs: string[]): Promise { + const normalizedGlobs = globs.map((glob) => resolve(glob)); + const filePaths = await globby(normalizedGlobs); + + return filePaths; +} diff --git a/packages/kbn-openapi-bundler/src/utils/write_documents.ts b/packages/kbn-openapi-bundler/src/utils/write_documents.ts new file mode 100644 index 0000000000000..40311bc7d61e1 --- /dev/null +++ b/packages/kbn-openapi-bundler/src/utils/write_documents.ts @@ -0,0 +1,49 @@ +/* + * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one + * or more contributor license agreements. Licensed under the Elastic License + * 2.0 and the Server Side Public License, v 1; you may not use this file except + * in compliance with, at your election, the Elastic License 2.0 or the Server + * Side Public License, v 1. + */ + +import chalk from 'chalk'; +import { OpenAPIV3 } from 'openapi-types'; +import { basename } from 'path'; +import { logger } from '../logger'; +import { writeYamlDocument } from './write_yaml_document'; + +export async function writeDocuments( + resultDocumentsMap: Map, + outputFilePath: string +): Promise { + for (const [version, document] of resultDocumentsMap.entries()) { + const versionedOutputFilePath = getVersionedOutputFilePath(outputFilePath, version); + + try { + await writeYamlDocument(versionedOutputFilePath, document); + + logger.success(`📖 Wrote merged OpenAPI specs to ${chalk.bold(versionedOutputFilePath)}`); + } catch (e) { + logger.error( + `Unable to save merged document to ${chalk.bold(versionedOutputFilePath)}: ${e.message}` + ); + } + } +} + +function getVersionedOutputFilePath(outputFilePath: string, version: string): string { + const hasVersionPlaceholder = outputFilePath.indexOf('{version}') > -1; + const snakeCasedVersion = version.replaceAll(/[^\w\d]+/g, '_'); + + if (hasVersionPlaceholder) { + return outputFilePath.replace('{version}', snakeCasedVersion); + } + + if (version === '') { + return outputFilePath; + } + + const filename = basename(outputFilePath); + + return outputFilePath.replace(filename, `${version}-${filename}`); +} diff --git a/packages/kbn-openapi-bundler/tests/bundle_refs.test.ts b/packages/kbn-openapi-bundler/tests/bundler/bundle_refs.test.ts similarity index 99% rename from packages/kbn-openapi-bundler/tests/bundle_refs.test.ts rename to packages/kbn-openapi-bundler/tests/bundler/bundle_refs.test.ts index a75dca8f27558..5371f0e07925a 100644 --- a/packages/kbn-openapi-bundler/tests/bundle_refs.test.ts +++ b/packages/kbn-openapi-bundler/tests/bundler/bundle_refs.test.ts @@ -8,7 +8,7 @@ import { OpenAPIV3 } from 'openapi-types'; import { bundleSpecs } from './bundle_specs'; -import { createOASDocument } from './create_oas_document'; +import { createOASDocument } from '../create_oas_document'; describe('OpenAPI Bundler - bundle references', () => { it('bundles files with external references', async () => { diff --git a/packages/kbn-openapi-bundler/tests/bundle_simple_specs.test.ts b/packages/kbn-openapi-bundler/tests/bundler/bundle_simple_specs.test.ts similarity index 97% rename from packages/kbn-openapi-bundler/tests/bundle_simple_specs.test.ts rename to packages/kbn-openapi-bundler/tests/bundler/bundle_simple_specs.test.ts index 15ef0e8215908..0d723f21a0b37 100644 --- a/packages/kbn-openapi-bundler/tests/bundle_simple_specs.test.ts +++ b/packages/kbn-openapi-bundler/tests/bundler/bundle_simple_specs.test.ts @@ -7,7 +7,7 @@ */ import { bundleSpecs } from './bundle_specs'; -import { createOASDocument } from './create_oas_document'; +import { createOASDocument } from '../create_oas_document'; describe('OpenAPI Bundler - simple specs', () => { it('bundles two simple specs', async () => { diff --git a/packages/kbn-openapi-bundler/tests/bundle_specs.ts b/packages/kbn-openapi-bundler/tests/bundler/bundle_specs.ts similarity index 96% rename from packages/kbn-openapi-bundler/tests/bundle_specs.ts rename to packages/kbn-openapi-bundler/tests/bundler/bundle_specs.ts index 0eaf7d3e1a974..59d5271fc0f87 100644 --- a/packages/kbn-openapi-bundler/tests/bundle_specs.ts +++ b/packages/kbn-openapi-bundler/tests/bundler/bundle_specs.ts @@ -18,12 +18,12 @@ import { } from 'fs'; import { dump, load } from 'js-yaml'; import { OpenAPIV3 } from 'openapi-types'; -import { bundle, BundlerConfig } from '../src/openapi_bundler'; +import { bundle, BundlerConfig } from '../../src/openapi_bundler'; const ROOT_PATH = join(__dirname, '..'); // Suppress bundler logging via mocking the logger -jest.mock('../src/logger'); +jest.mock('../../src/logger'); export async function bundleSpecs( oasSpecs: Record, diff --git a/packages/kbn-openapi-bundler/tests/bundle_specs_with_multiple_modifications.test.ts b/packages/kbn-openapi-bundler/tests/bundler/bundle_specs_with_multiple_modifications.test.ts similarity index 100% rename from packages/kbn-openapi-bundler/tests/bundle_specs_with_multiple_modifications.test.ts rename to packages/kbn-openapi-bundler/tests/bundler/bundle_specs_with_multiple_modifications.test.ts diff --git a/packages/kbn-openapi-bundler/tests/circular.test.ts b/packages/kbn-openapi-bundler/tests/bundler/circular.test.ts similarity index 98% rename from packages/kbn-openapi-bundler/tests/circular.test.ts rename to packages/kbn-openapi-bundler/tests/bundler/circular.test.ts index 0d7715ace7827..b711014a4ff34 100644 --- a/packages/kbn-openapi-bundler/tests/circular.test.ts +++ b/packages/kbn-openapi-bundler/tests/bundler/circular.test.ts @@ -9,7 +9,7 @@ import { dump } from 'js-yaml'; import { OpenAPIV3 } from 'openapi-types'; import { bundleSpecs } from './bundle_specs'; -import { createOASDocument } from './create_oas_document'; +import { createOASDocument } from '../create_oas_document'; describe('OpenAPI Bundler - circular specs', () => { it('bundles recursive spec', async () => { diff --git a/packages/kbn-openapi-bundler/tests/complex_specs/common.schema.yaml b/packages/kbn-openapi-bundler/tests/bundler/complex_specs/common.schema.yaml similarity index 100% rename from packages/kbn-openapi-bundler/tests/complex_specs/common.schema.yaml rename to packages/kbn-openapi-bundler/tests/bundler/complex_specs/common.schema.yaml diff --git a/packages/kbn-openapi-bundler/tests/complex_specs/expected.yaml b/packages/kbn-openapi-bundler/tests/bundler/complex_specs/expected.yaml similarity index 100% rename from packages/kbn-openapi-bundler/tests/complex_specs/expected.yaml rename to packages/kbn-openapi-bundler/tests/bundler/complex_specs/expected.yaml diff --git a/packages/kbn-openapi-bundler/tests/complex_specs/invalid_labels.schema.yaml b/packages/kbn-openapi-bundler/tests/bundler/complex_specs/invalid_labels.schema.yaml similarity index 100% rename from packages/kbn-openapi-bundler/tests/complex_specs/invalid_labels.schema.yaml rename to packages/kbn-openapi-bundler/tests/bundler/complex_specs/invalid_labels.schema.yaml diff --git a/packages/kbn-openapi-bundler/tests/complex_specs/missing_labels.schema.yaml b/packages/kbn-openapi-bundler/tests/bundler/complex_specs/missing_labels.schema.yaml similarity index 100% rename from packages/kbn-openapi-bundler/tests/complex_specs/missing_labels.schema.yaml rename to packages/kbn-openapi-bundler/tests/bundler/complex_specs/missing_labels.schema.yaml diff --git a/packages/kbn-openapi-bundler/tests/complex_specs/spec1.schema.yaml b/packages/kbn-openapi-bundler/tests/bundler/complex_specs/spec1.schema.yaml similarity index 100% rename from packages/kbn-openapi-bundler/tests/complex_specs/spec1.schema.yaml rename to packages/kbn-openapi-bundler/tests/bundler/complex_specs/spec1.schema.yaml diff --git a/packages/kbn-openapi-bundler/tests/complex_specs/spec2.schema.yaml b/packages/kbn-openapi-bundler/tests/bundler/complex_specs/spec2.schema.yaml similarity index 100% rename from packages/kbn-openapi-bundler/tests/complex_specs/spec2.schema.yaml rename to packages/kbn-openapi-bundler/tests/bundler/complex_specs/spec2.schema.yaml diff --git a/packages/kbn-openapi-bundler/tests/complex_specs/spec3.schema.yaml b/packages/kbn-openapi-bundler/tests/bundler/complex_specs/spec3.schema.yaml similarity index 100% rename from packages/kbn-openapi-bundler/tests/complex_specs/spec3.schema.yaml rename to packages/kbn-openapi-bundler/tests/bundler/complex_specs/spec3.schema.yaml diff --git a/packages/kbn-openapi-bundler/tests/different_endpoint_versions.test.ts b/packages/kbn-openapi-bundler/tests/bundler/different_endpoint_versions.test.ts similarity index 97% rename from packages/kbn-openapi-bundler/tests/different_endpoint_versions.test.ts rename to packages/kbn-openapi-bundler/tests/bundler/different_endpoint_versions.test.ts index 956bd59f25adb..b6b3a3870e974 100644 --- a/packages/kbn-openapi-bundler/tests/different_endpoint_versions.test.ts +++ b/packages/kbn-openapi-bundler/tests/bundler/different_endpoint_versions.test.ts @@ -7,7 +7,7 @@ */ import { bundleSpecs } from './bundle_specs'; -import { createOASDocument } from './create_oas_document'; +import { createOASDocument } from '../create_oas_document'; describe('OpenAPI Bundler - different API versions', () => { it('bundles one endpoint with different versions', async () => { diff --git a/packages/kbn-openapi-bundler/tests/different_oas_versions.test.ts b/packages/kbn-openapi-bundler/tests/bundler/different_oas_versions.test.ts similarity index 96% rename from packages/kbn-openapi-bundler/tests/different_oas_versions.test.ts rename to packages/kbn-openapi-bundler/tests/bundler/different_oas_versions.test.ts index 949f4f882ff4e..8163de8b62605 100644 --- a/packages/kbn-openapi-bundler/tests/different_oas_versions.test.ts +++ b/packages/kbn-openapi-bundler/tests/bundler/different_oas_versions.test.ts @@ -7,7 +7,7 @@ */ import { bundleSpecs } from './bundle_specs'; -import { createOASDocument } from './create_oas_document'; +import { createOASDocument } from '../create_oas_document'; describe('OpenAPI Bundler - different OAS versions', () => { it('DOES NOT bundle specs with different OpenAPI versions', async () => { diff --git a/packages/kbn-openapi-bundler/tests/include_labels.test.ts b/packages/kbn-openapi-bundler/tests/bundler/include_labels.test.ts similarity index 99% rename from packages/kbn-openapi-bundler/tests/include_labels.test.ts rename to packages/kbn-openapi-bundler/tests/bundler/include_labels.test.ts index 8456afdd00c3e..f31c3e8e2606b 100644 --- a/packages/kbn-openapi-bundler/tests/include_labels.test.ts +++ b/packages/kbn-openapi-bundler/tests/bundler/include_labels.test.ts @@ -7,7 +7,7 @@ */ import { bundleSpecs } from './bundle_specs'; -import { createOASDocument } from './create_oas_document'; +import { createOASDocument } from '../create_oas_document'; describe('OpenAPI Bundler - include labeled operations', () => { it.each([ diff --git a/packages/kbn-openapi-bundler/tests/inline_ref.test.ts b/packages/kbn-openapi-bundler/tests/bundler/inline_ref.test.ts similarity index 99% rename from packages/kbn-openapi-bundler/tests/inline_ref.test.ts rename to packages/kbn-openapi-bundler/tests/bundler/inline_ref.test.ts index 913afd934de09..e220699f17c68 100644 --- a/packages/kbn-openapi-bundler/tests/inline_ref.test.ts +++ b/packages/kbn-openapi-bundler/tests/bundler/inline_ref.test.ts @@ -7,7 +7,7 @@ */ import { bundleSpecs } from './bundle_specs'; -import { createOASDocument } from './create_oas_document'; +import { createOASDocument } from '../create_oas_document'; describe('OpenAPI Bundler - inline references', () => { it('inlines local references', async () => { diff --git a/packages/kbn-openapi-bundler/tests/omit_unused_schemas.test.ts b/packages/kbn-openapi-bundler/tests/bundler/omit_unused_schemas.test.ts similarity index 98% rename from packages/kbn-openapi-bundler/tests/omit_unused_schemas.test.ts rename to packages/kbn-openapi-bundler/tests/bundler/omit_unused_schemas.test.ts index b714ba5ddf834..7fd9201194ded 100644 --- a/packages/kbn-openapi-bundler/tests/omit_unused_schemas.test.ts +++ b/packages/kbn-openapi-bundler/tests/bundler/omit_unused_schemas.test.ts @@ -7,7 +7,7 @@ */ import { bundleSpecs } from './bundle_specs'; -import { createOASDocument } from './create_oas_document'; +import { createOASDocument } from '../create_oas_document'; describe('OpenAPI Bundler - omit unused schemas', () => { it('omits unused local schema', async () => { diff --git a/packages/kbn-openapi-bundler/tests/produce_stable_bundle.test.ts b/packages/kbn-openapi-bundler/tests/bundler/produce_stable_bundle.test.ts similarity index 97% rename from packages/kbn-openapi-bundler/tests/produce_stable_bundle.test.ts rename to packages/kbn-openapi-bundler/tests/bundler/produce_stable_bundle.test.ts index 04f85272121f2..088c1e7fd1f52 100644 --- a/packages/kbn-openapi-bundler/tests/produce_stable_bundle.test.ts +++ b/packages/kbn-openapi-bundler/tests/bundler/produce_stable_bundle.test.ts @@ -8,7 +8,7 @@ import { OpenAPIV3 } from 'openapi-types'; import { bundleSpecs } from './bundle_specs'; -import { createOASDocument } from './create_oas_document'; +import { createOASDocument } from '../create_oas_document'; describe('OpenAPI Bundler - produce stable bundle', () => { it('produces stable bundle (keys are sorted)', async () => { diff --git a/packages/kbn-openapi-bundler/tests/reduce_all_of.test.ts b/packages/kbn-openapi-bundler/tests/bundler/reduce_all_of.test.ts similarity index 99% rename from packages/kbn-openapi-bundler/tests/reduce_all_of.test.ts rename to packages/kbn-openapi-bundler/tests/bundler/reduce_all_of.test.ts index 8d79f8d77ea09..71976583e42c0 100644 --- a/packages/kbn-openapi-bundler/tests/reduce_all_of.test.ts +++ b/packages/kbn-openapi-bundler/tests/bundler/reduce_all_of.test.ts @@ -7,7 +7,7 @@ */ import { bundleSpecs } from './bundle_specs'; -import { createOASDocument } from './create_oas_document'; +import { createOASDocument } from '../create_oas_document'; describe('OpenAPI Bundler - reduce allOf item', () => { it('flatten folded allOfs', async () => { diff --git a/packages/kbn-openapi-bundler/tests/remove_props.test.ts b/packages/kbn-openapi-bundler/tests/bundler/remove_props.test.ts similarity index 98% rename from packages/kbn-openapi-bundler/tests/remove_props.test.ts rename to packages/kbn-openapi-bundler/tests/bundler/remove_props.test.ts index d9ab67386f3c2..99da2a538e196 100644 --- a/packages/kbn-openapi-bundler/tests/remove_props.test.ts +++ b/packages/kbn-openapi-bundler/tests/bundler/remove_props.test.ts @@ -8,7 +8,7 @@ import { OpenAPIV3 } from 'openapi-types'; import { bundleSpecs } from './bundle_specs'; -import { createOASDocument } from './create_oas_document'; +import { createOASDocument } from '../create_oas_document'; describe('OpenAPI Bundler - remove custom x- props', () => { it('removes "x-codegen-enabled" property', async () => { diff --git a/packages/kbn-openapi-bundler/tests/skip_nodes.test.ts b/packages/kbn-openapi-bundler/tests/bundler/skip_nodes.test.ts similarity index 98% rename from packages/kbn-openapi-bundler/tests/skip_nodes.test.ts rename to packages/kbn-openapi-bundler/tests/bundler/skip_nodes.test.ts index 35b1de2b3a0bc..777cee3653a2f 100644 --- a/packages/kbn-openapi-bundler/tests/skip_nodes.test.ts +++ b/packages/kbn-openapi-bundler/tests/bundler/skip_nodes.test.ts @@ -7,7 +7,7 @@ */ import { bundleSpecs } from './bundle_specs'; -import { createOASDocument } from './create_oas_document'; +import { createOASDocument } from '../create_oas_document'; describe('OpenAPI Bundler - skip nodes like internal endpoints', () => { it('skips nodes with x-internal property', async () => { diff --git a/packages/kbn-openapi-bundler/tests/x_modify.test.ts b/packages/kbn-openapi-bundler/tests/bundler/x_modify.test.ts similarity index 99% rename from packages/kbn-openapi-bundler/tests/x_modify.test.ts rename to packages/kbn-openapi-bundler/tests/bundler/x_modify.test.ts index cdc53e8369345..68c84bd29dce0 100644 --- a/packages/kbn-openapi-bundler/tests/x_modify.test.ts +++ b/packages/kbn-openapi-bundler/tests/bundler/x_modify.test.ts @@ -7,7 +7,7 @@ */ import { bundleSpecs } from './bundle_specs'; -import { createOASDocument } from './create_oas_document'; +import { createOASDocument } from '../create_oas_document'; describe('OpenAPI Bundler - x-modify', () => { it('inlines references with x-modify property', async () => { diff --git a/packages/kbn-openapi-bundler/tests/merger/different_oas_versions.test.ts b/packages/kbn-openapi-bundler/tests/merger/different_oas_versions.test.ts new file mode 100644 index 0000000000000..83af0016236d7 --- /dev/null +++ b/packages/kbn-openapi-bundler/tests/merger/different_oas_versions.test.ts @@ -0,0 +1,68 @@ +/* + * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one + * or more contributor license agreements. Licensed under the Elastic License + * 2.0 and the Server Side Public License, v 1; you may not use this file except + * in compliance with, at your election, the Elastic License 2.0 or the Server + * Side Public License, v 1. + */ + +import { createOASDocument } from '../create_oas_document'; +import { mergeSpecs } from './merge_specs'; + +describe('OpenAPI Merger - different OpenAPI versions', () => { + it('merges specs having OpenAPI 3.0.x versions', async () => { + const spec1 = createOASDocument({ + openapi: '3.0.3', + paths: {}, + }); + const spec2 = createOASDocument({ + openapi: '3.0.0', + paths: {}, + }); + + const [mergedSpec] = Object.values( + await mergeSpecs({ + 1: spec1, + 2: spec2, + }) + ); + + expect(mergedSpec.openapi).toBe('3.0.3'); + }); + + it('throws an error when different minor OAS versions encountered', async () => { + const spec1 = createOASDocument({ + openapi: '3.0.3', + paths: {}, + }); + const spec2 = createOASDocument({ + openapi: '3.1.0', + paths: {}, + }); + + expect( + mergeSpecs({ + 1: spec1, + 2: spec2, + }) + ).rejects.toThrowError(/OpenAPI specs must use the same OpenAPI version/); + }); + + it('throws an error when different OAS 3.1.x patch versions encountered', async () => { + const spec1 = createOASDocument({ + openapi: '3.1.0', + paths: {}, + }); + const spec2 = createOASDocument({ + openapi: '3.1.1', + paths: {}, + }); + + expect( + mergeSpecs({ + 1: spec1, + 2: spec2, + }) + ).rejects.toThrowError(/OpenAPI specs must use the same OpenAPI version/); + }); +}); diff --git a/packages/kbn-openapi-bundler/tests/merger/merge_multiple_specs.test.ts b/packages/kbn-openapi-bundler/tests/merger/merge_multiple_specs.test.ts new file mode 100644 index 0000000000000..536bc56137f96 --- /dev/null +++ b/packages/kbn-openapi-bundler/tests/merger/merge_multiple_specs.test.ts @@ -0,0 +1,136 @@ +/* + * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one + * or more contributor license agreements. Licensed under the Elastic License + * 2.0 and the Server Side Public License, v 1; you may not use this file except + * in compliance with, at your election, the Elastic License 2.0 or the Server + * Side Public License, v 1. + */ + +import { createOASDocument } from '../create_oas_document'; +import { mergeSpecs } from './merge_specs'; + +describe('OpenAPI Merger - merge paths', () => { + it('merges path operations', async () => { + const spec1 = createOASDocument({ + paths: { + '/api/some_api': { + get: { + responses: { + '200': { + description: 'Successful response', + content: { + 'application/json': { + schema: { + type: 'string', + }, + }, + }, + }, + }, + }, + }, + }, + }); + const spec2 = createOASDocument({ + paths: { + '/api/some_api': { + post: { + responses: { + '200': { + description: 'Successful response', + content: { + 'application/json': { + schema: { + type: 'string', + }, + }, + }, + }, + }, + }, + }, + }, + }); + + const [mergedSpec] = Object.values( + await mergeSpecs({ + 1: spec1, + 2: spec2, + }) + ); + + expect(Object.keys(mergedSpec.paths)).toEqual(['/api/some_api']); + + expect(mergedSpec.paths).toMatchObject({ + '/api/some_api': { + get: expect.anything(), + post: expect.anything(), + }, + }); + }); + + it('merges different versions of the same endpoint', async () => { + const spec1 = createOASDocument({ + info: { + version: '2023-10-31', + }, + paths: { + '/api/some_api': { + get: { + responses: { + '200': { + description: 'Successful response', + content: { + 'application/json': { + schema: { + type: 'string', + }, + }, + }, + }, + }, + }, + }, + }, + }); + const spec2 = createOASDocument({ + info: { + version: '2024-01-01', + }, + paths: { + '/api/some_api': { + get: { + responses: { + '200': { + description: 'Successful response', + content: { + 'application/json': { + schema: { + type: 'string', + }, + }, + }, + }, + }, + }, + }, + }, + }); + + const [mergedSpec] = Object.values( + await mergeSpecs({ + 1: spec1, + 2: spec2, + }) + ); + + expect(Object.keys(mergedSpec.paths)).toEqual(['/api/some_api']); + + expect(mergedSpec.paths['/api/some_api']!.get?.responses['200']).toMatchObject({ + content: { + 'application/json; Elastic-Api-Version=2023-10-31': expect.anything(), + 'application/json; Elastic-Api-Version=2024-01-01': expect.anything(), + }, + }); + }); +}); diff --git a/packages/kbn-openapi-bundler/tests/merger/merge_specs.ts b/packages/kbn-openapi-bundler/tests/merger/merge_specs.ts new file mode 100644 index 0000000000000..743b40373053c --- /dev/null +++ b/packages/kbn-openapi-bundler/tests/merger/merge_specs.ts @@ -0,0 +1,85 @@ +/* + * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one + * or more contributor license agreements. Licensed under the Elastic License + * 2.0 and the Server Side Public License, v 1; you may not use this file except + * in compliance with, at your election, the Elastic License 2.0 or the Server + * Side Public License, v 1. + */ + +import { join } from 'path'; +import { + existsSync, + mkdirSync, + readdirSync, + readFileSync, + rmdirSync, + unlinkSync, + writeFileSync, +} from 'fs'; +import { dump, load } from 'js-yaml'; +import { OpenAPIV3 } from 'openapi-types'; +import { merge, MergerConfig } from '../../src/openapi_merger'; + +const ROOT_PATH = join(__dirname, '..'); + +// Suppress merger logging via mocking the logger +jest.mock('../../src/logger'); + +export async function mergeSpecs( + oasSpecs: Record, + mergedSpecInfo?: MergerConfig['mergedSpecInfo'] +): Promise> { + const randomStr = (Math.random() + 1).toString(36).substring(7); + const folderToMergePath = join(ROOT_PATH, 'target', 'oas-test', randomStr); + const resultFolderPath = join(ROOT_PATH, 'target', 'oas-test-merged-result', randomStr); + const mergedFilePathTemplate = join(resultFolderPath, '{version}.yaml'); + + dumpSpecs(folderToMergePath, oasSpecs); + + await mergeFolder(folderToMergePath, mergedFilePathTemplate, mergedSpecInfo); + + return readMergedSpecs(resultFolderPath); +} + +function removeFolder(folderPath: string): void { + if (existsSync(folderPath)) { + for (const fileName of readdirSync(folderPath)) { + unlinkSync(join(folderPath, fileName)); + } + + rmdirSync(folderPath); + } +} + +function dumpSpecs(folderPath: string, oasSpecs: Record): void { + removeFolder(folderPath); + mkdirSync(folderPath, { recursive: true }); + + for (const [fileName, oasSpec] of Object.entries(oasSpecs)) { + writeFileSync(join(folderPath, `${fileName}.schema.yaml`), dump(oasSpec)); + } +} + +export function readMergedSpecs(folderPath: string): Record { + const mergedSpecs: Record = {}; + + for (const fileName of readdirSync(folderPath)) { + const yaml = readFileSync(join(folderPath, fileName), { encoding: 'utf8' }); + + mergedSpecs[fileName] = load(yaml); + } + + return mergedSpecs; +} + +export async function mergeFolder( + folderToMergePath: string, + mergedFilePathTemplate: string, + mergedSpecInfo?: MergerConfig['mergedSpecInfo'] +): Promise { + await merge({ + sourceGlobs: [join(folderToMergePath, '*.schema.yaml')], + outputFilePath: mergedFilePathTemplate, + mergedSpecInfo, + }); +} diff --git a/packages/kbn-openapi-bundler/tests/merger/unresolvable_operation_conflicts.test.ts b/packages/kbn-openapi-bundler/tests/merger/unresolvable_operation_conflicts.test.ts new file mode 100644 index 0000000000000..0305b31772287 --- /dev/null +++ b/packages/kbn-openapi-bundler/tests/merger/unresolvable_operation_conflicts.test.ts @@ -0,0 +1,318 @@ +/* + * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one + * or more contributor license agreements. Licensed under the Elastic License + * 2.0 and the Server Side Public License, v 1; you may not use this file except + * in compliance with, at your election, the Elastic License 2.0 or the Server + * Side Public License, v 1. + */ + +import { createOASDocument } from '../create_oas_document'; +import { mergeSpecs } from './merge_specs'; + +describe('OpenAPI Merger - unresolvable operation object conflicts', () => { + it.each([ + [ + 'tags', + createOASDocument({ + paths: { + '/api/my/endpoint': { + get: { + tags: ['tag1'], + responses: {}, + }, + }, + }, + }), + createOASDocument({ + paths: { + '/api/my/endpoint': { + get: { + tags: ['tag2'], + responses: {}, + }, + }, + }, + }), + ], + [ + 'summary', + createOASDocument({ + paths: { + '/api/my/endpoint': { + get: { + summary: 'Summary A', + responses: {}, + }, + }, + }, + }), + createOASDocument({ + paths: { + '/api/my/endpoint': { + get: { + summary: 'Summary B', + responses: {}, + }, + }, + }, + }), + ], + [ + 'description', + createOASDocument({ + paths: { + '/api/my/endpoint': { + get: { + description: 'Description A', + responses: {}, + }, + }, + }, + }), + createOASDocument({ + paths: { + '/api/my/endpoint': { + get: { + description: 'Description B', + responses: {}, + }, + }, + }, + }), + ], + [ + 'operationId', + createOASDocument({ + paths: { + '/api/my/endpoint': { + get: { + operationId: 'EndpointA', + responses: {}, + }, + }, + }, + }), + createOASDocument({ + paths: { + '/api/my/endpoint': { + get: { + operationId: 'EndpointB', + responses: {}, + }, + }, + }, + }), + ], + [ + 'parameters', + createOASDocument({ + paths: { + '/api/my/endpoint': { + get: { + parameters: [ + { + name: 'param1', + in: 'path', + }, + ], + responses: {}, + }, + }, + }, + }), + createOASDocument({ + paths: { + '/api/my/endpoint': { + get: { + parameters: [ + { + name: 'param1', + in: 'path', + required: true, + }, + ], + responses: {}, + }, + }, + }, + }), + ], + [ + 'callbacks', + createOASDocument({ + paths: { + '/api/my/endpoint': { + get: { + callbacks: { + callback1: { + responses: {}, + }, + }, + responses: {}, + }, + }, + }, + }), + createOASDocument({ + paths: { + '/api/my/endpoint': { + get: { + callbacks: { + callback2: { + responses: {}, + }, + }, + responses: {}, + }, + }, + }, + }), + ], + [ + 'deprecation status', + createOASDocument({ + paths: { + '/api/my/endpoint': { + get: { + responses: {}, + }, + }, + }, + }), + createOASDocument({ + paths: { + '/api/my/endpoint': { + get: { + deprecated: true, + responses: {}, + }, + }, + }, + }), + ], + [ + 'security requirements', + createOASDocument({ + paths: { + '/api/my/endpoint': { + get: { + responses: {}, + }, + }, + }, + }), + createOASDocument({ + paths: { + '/api/my/endpoint': { + get: { + security: [ + { + securityRequirement: [], + }, + ], + responses: {}, + }, + }, + }, + }), + ], + [ + 'servers', + createOASDocument({ + paths: { + '/api/my/endpoint': { + get: { + responses: {}, + }, + }, + }, + }), + createOASDocument({ + paths: { + '/api/my/endpoint': { + get: { + servers: [ + { + url: '/some/url', + }, + ], + responses: {}, + }, + }, + }, + }), + ], + ])('throws an error when operations %s do not match', async (_, spec1, spec2) => { + expect( + mergeSpecs({ + 1: spec1, + 2: spec2, + }) + ).rejects.toThrowError('"Operation objects are incompatible"'); + }); + + it("throws an error when operation's request body has a top level $ref", async () => { + const spec1 = createOASDocument({ + paths: { + '/api/my/endpoint': { + get: { + responses: {}, + }, + }, + }, + }); + const spec2 = createOASDocument({ + paths: { + '/api/my/endpoint': { + get: { + requestBody: { + // SomeRequestBody definition is omitted for brivity since it's not validated by the merger + $ref: '#/components/requestBodies/SomeRequestBody', + }, + responses: {}, + }, + }, + }, + }); + + expect( + mergeSpecs({ + 1: spec1, + 2: spec2, + }) + ).rejects.toThrowError('Request body top level $ref is not supported'); + }); + + it("throws an error when one of operation's responses has a top level $ref", async () => { + const spec1 = createOASDocument({ + paths: { + '/api/my/endpoint': { + get: { + responses: {}, + }, + }, + }, + }); + const spec2 = createOASDocument({ + paths: { + '/api/my/endpoint': { + get: { + responses: { + 200: { + // SomeResponse definition is omitted for brivity since it's not validated by the merger + $ref: '#/components/responses/SomeResponse', + }, + }, + }, + }, + }, + }); + + expect( + mergeSpecs({ + 1: spec1, + 2: spec2, + }) + ).rejects.toThrowError('Response object top level $ref is not supported'); + }); +}); diff --git a/packages/kbn-openapi-bundler/tests/merger/unresolvable_path_item_conflicts.test.ts b/packages/kbn-openapi-bundler/tests/merger/unresolvable_path_item_conflicts.test.ts new file mode 100644 index 0000000000000..3488c55fccb87 --- /dev/null +++ b/packages/kbn-openapi-bundler/tests/merger/unresolvable_path_item_conflicts.test.ts @@ -0,0 +1,136 @@ +/* + * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one + * or more contributor license agreements. Licensed under the Elastic License + * 2.0 and the Server Side Public License, v 1; you may not use this file except + * in compliance with, at your election, the Elastic License 2.0 or the Server + * Side Public License, v 1. + */ + +import { createOASDocument } from '../create_oas_document'; +import { mergeSpecs } from './merge_specs'; + +describe('OpenAPI Merger - unresolvable path item object conflicts', () => { + it.each([ + [ + 'summary', + createOASDocument({ + paths: { + '/api/my/endpoint': { + summary: 'Summary A', + get: { + responses: {}, + }, + }, + }, + }), + createOASDocument({ + paths: { + '/api/my/endpoint': { + summary: 'Summary B', + get: { + responses: {}, + }, + }, + }, + }), + ], + [ + 'description', + createOASDocument({ + paths: { + '/api/my/endpoint': { + description: 'Description A', + get: { + responses: {}, + }, + }, + }, + }), + createOASDocument({ + paths: { + '/api/my/endpoint': { + description: 'Description B', + get: { + responses: {}, + }, + }, + }, + }), + ], + ])('throws an error when path items %s do not match', async (_, spec1, spec2) => { + expect( + mergeSpecs({ + 1: spec1, + 2: spec2, + }) + ).rejects.toThrowError(/value .+ doesn't match to already encountered/); + }); + + it("throws an error when path item's parameters do not match", async () => { + const spec1 = createOASDocument({ + paths: { + '/api/my/endpoint': { + parameters: [ + { + name: 'param1', + in: 'path', + }, + ], + get: { + responses: {}, + }, + }, + }, + }); + const spec2 = createOASDocument({ + paths: { + '/api/my/endpoint': { + parameters: [ + { + name: 'param1', + in: 'path', + required: true, + }, + ], + get: { + responses: {}, + }, + }, + }, + }); + + expect( + mergeSpecs({ + 1: spec1, + 2: spec2, + }) + ).rejects.toThrowError('definition is duplicated and differs from previously encountered'); + }); + + it('throws an error when path item has a top level $ref', async () => { + const spec1 = createOASDocument({ + paths: { + '/api/my/endpoint': { + get: { + responses: {}, + }, + }, + }, + }); + const spec2 = createOASDocument({ + paths: { + '/api/my/endpoint': { + // PathItemDefinition definition is omitted for brivity since it's not validated by the merger + $ref: '#/components/schemas/PathItemDefinition', + }, + }, + }); + + expect( + mergeSpecs({ + 1: spec1, + 2: spec2, + }) + ).rejects.toThrowError('Path item top level $ref is not supported'); + }); +});