[build] Create CDN assets by jbudz · Pull Request #169707 · elastic/kibana

jbudz · 2023-10-24T18:49:56Z

Adds a new build step createCdnAssets that will create an archive kibana-<version>-cdn-assets.tar.gz with static assets organized using the request structure of the kibana client.

By default CDN assets are created
Adding the flag node scripts/build --skip-cdn-assets will skip creation
ci:build-cdn-assets can be used to create and upload the archive for testing

Testing: see #169408. Builds are available in the artifacts tab on the Build Distribution step.

Extract builds

 python3 -m http.server -b localhost -d kibana-8.12.0-SNAPSHOT-cdn-assets 8000

 echo 'server.cdn.url: http://localhost:8000' >> kibana-8.12.0-SNAPSHOT/config/kibana.yml

```
kibana-8.12.0-SNAPSHOT/bin/kibana
```

elasticmachine · 2023-10-24T19:09:28Z

Pinging @elastic/kibana-core (Team:Core)

elasticmachine · 2023-10-24T19:09:30Z

Pinging @elastic/kibana-operations (Team:Operations)

src/dev/build/tasks/create_cdn_assets_task.ts

pgayvallet

Sounds promising, but I think I found a few things we may want to fix.

src/dev/build/tasks/create_cdn_assets_task.ts

pgayvallet · 2023-10-26T07:47:18Z

src/dev/build/tasks/create_cdn_assets_task.ts

+    // Shared dependencies
+    await copyAll(
+      resolve(buildSource, 'node_modules/@kbn/ui-shared-deps-npm/shared_built_assets'),
+      resolve(bundles, 'kbn-ui-shared-deps-npm')


I think we also need to copy the assets for kbn-monaco (unless the library won't load its assets from the CDN?)

kibana/packages/core/apps/core-apps-server-internal/src/bundle_routes/register_bundle_routes.ts

Lines 69 to 75 in 1b85815

registerRouteForBundle(router, {

publicPath: `${serverBasePath}/${buildNum}/bundles/kbn-monaco/`,

routePath: `/${buildNum}/bundles/kbn-monaco/`,

bundlesPath: KbnMonaco.bundleDir,

fileHashCache,

isDist,

});

Thanks for the source references, I added them to the build step too. I pushed ac91ae3

pgayvallet · 2023-10-26T07:53:27Z

src/dev/build/tasks/create_cdn_assets_task.ts

+        try {
+          const assetsSource = resolve(pluginRoot, 'assets');
+          const assetsDest = resolve(plugin, manifest.plugin.id, 'assets');
+          await access(assetsSource);
+          await copyAll(assetsSource, assetsDest);


If I read the path correctly, then I think this is wrong.

The plugins static assets aren't accessed/exposed as other bundles via /assets/{buildNum}/bundles/plugin/{id}/assets/{asset_path}, but via /plugins/{id}/assets/{asset_path}

E.g

kibana/src/plugins/home/server/services/sample_data/data_sets/flights/index.ts

Line 27 in 497e6c3

previewImagePath: '/plugins/home/assets/sample_data_resources/flights/dashboard.webp',

For this file: src/plugins/home/public/assets/sample_data_resources/flights/dashboard.webp

EDIT: I finally found where we're doing it:

kibana/packages/core/plugins/core-plugins-server-internal/src/plugins_service.ts

Lines 437 to 442 in e398e7b

for (const [pluginName, pluginInfo] of uiPluginInternalInfo) {

deps.http.registerStaticDir(

`/plugins/${pluginName}/assets/{path*}`,

pluginInfo.publicAssetsDir

);

}

I pushed 06a3ded

pgayvallet

Paths seems all correct to me, and I can't think of assets we may have forgotten to include there.

kibana-ci · 2023-11-06T14:28:05Z

💚 Build Succeeded

Buildkite Build
Commit: 3e78fac

Metrics [docs]

✅ unchanged

History

💛 Build #173119 was flaky e726b2d
💛 Build #170567 was flaky 01c6fb5

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

Ikuni17

lgtm

When testing docker contexts we skip all build steps related to Kibana artifacts. CDN assets rely on the Kibana base folder and should be skipped too. Fixes https://buildkite.com/elastic/kibana-artifacts-snapshot/builds/3445 Build https://buildkite.com/elastic/kibana-artifacts-snapshot/builds/3451 Bug introduced in #169707

jbudz added 2 commits October 24, 2023 13:27

[build] Create CDN assets

a923543

ci

3adc705

jbudz added Team:Operations Kibana-Operations Team release_note:skip Skip the PR/issue when compiling release notes ci:build-cdn-assets Team:Core Platform Core services: plugins, logging, config, saved objects, http, ES client, i18n, etc t// labels Oct 24, 2023

This comment was marked as duplicate.

Sign in to view

jbudz marked this pull request as ready for review October 24, 2023 19:09

jbudz requested a review from a team as a code owner October 24, 2023 19:09

jbudz requested a review from jloleysens October 24, 2023 19:09

jbudz commented Oct 24, 2023

View reviewed changes

src/dev/build/tasks/create_cdn_assets_task.ts Outdated Show resolved Hide resolved

fix assets directory

01c6fb5

pgayvallet reviewed Oct 26, 2023

View reviewed changes

jbudz added 4 commits November 3, 2023 07:43

Merge remote-tracking branch 'upstream/main' into build-cdn-assets

9a078b6

add kbn-monaco

330fb77

fix plugin assets

06a3ded

fix monaco path

ac91ae3

jbudz added the ci:build-all-platforms label Nov 3, 2023

jbudz added 2 commits November 3, 2023 08:32

Merge branch 'main' into build-cdn-assets

e726b2d

Merge branch 'main' into build-cdn-assets

3e78fac

pgayvallet approved these changes Nov 6, 2023

View reviewed changes

Ikuni17 approved these changes Nov 6, 2023

View reviewed changes

jbudz merged commit e3c05d6 into elastic:main Nov 6, 2023

kibanamachine added v8.12.0 backport:skip This PR does not require backporting labels Nov 6, 2023

jbudz mentioned this pull request Nov 7, 2023

[build] Skip CDN assets build when testing docker contexts #170735

Merged

v1v mentioned this pull request Apr 15, 2024

github-action(kibana-docker-image): skip CDN elastic/apm-pipeline-library#2582

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[build] Create CDN assets#169707

[build] Create CDN assets#169707
jbudz merged 9 commits intoelastic:mainfrom
jbudz:build-cdn-assets

jbudz commented Oct 24, 2023 •

edited

Loading

Uh oh!

This comment was marked as duplicate.

elasticmachine commented Oct 24, 2023

Uh oh!

elasticmachine commented Oct 24, 2023

Uh oh!

Uh oh!

pgayvallet left a comment

Uh oh!

Uh oh!

pgayvallet Oct 26, 2023 •

edited

Loading

Uh oh!

jbudz Nov 3, 2023

Uh oh!

pgayvallet Oct 26, 2023 •

edited

Loading

Uh oh!

jbudz Nov 3, 2023

Uh oh!

pgayvallet left a comment

Uh oh!

kibana-ci commented Nov 6, 2023

Uh oh!

Ikuni17 left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

	registerRouteForBundle(router, {
	publicPath: `${serverBasePath}/${buildNum}/bundles/kbn-monaco/`,
	routePath: `/${buildNum}/bundles/kbn-monaco/`,
	bundlesPath: KbnMonaco.bundleDir,
	fileHashCache,
	isDist,
	});

	for (const [pluginName, pluginInfo] of uiPluginInternalInfo) {
	deps.http.registerStaticDir(
	`/plugins/${pluginName}/assets/{path*}`,
	pluginInfo.publicAssetsDir
	);
	}

Conversation

jbudz commented Oct 24, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

This comment was marked as duplicate.

elasticmachine commented Oct 24, 2023

Uh oh!

elasticmachine commented Oct 24, 2023

Uh oh!

Uh oh!

pgayvallet left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

pgayvallet Oct 26, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jbudz Nov 3, 2023

Choose a reason for hiding this comment

Uh oh!

pgayvallet Oct 26, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jbudz Nov 3, 2023

Choose a reason for hiding this comment

Uh oh!

pgayvallet left a comment

Choose a reason for hiding this comment

Uh oh!

kibana-ci commented Nov 6, 2023

💚 Build Succeeded

Metrics [docs]

History

Uh oh!

Ikuni17 left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

jbudz commented Oct 24, 2023 •

edited

Loading

pgayvallet Oct 26, 2023 •

edited

Loading

pgayvallet Oct 26, 2023 •

edited

Loading