Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ import { createServerRoute } from '../../../create_server_route';

const KUBERNETES = 'kubernetes';
const DOCKER = 'docker';
const METRICBEAT = 'metricbeat';

const infraSideNavRoute = createServerRoute({
endpoint: 'GET /internal/observability_navigation',
Expand All @@ -33,6 +34,42 @@ const infraSideNavRoute = createServerRoute({
const [fleetStart, core] = await Promise.all([plugins.fleet?.start(), context.core]);
const packageClient = fleetStart?.packageService.asScoped(request);

const metricbeatData = await core.elasticsearch.client.asCurrentUser.search({
index: 'metrics-kubernetes*',
ignore_unavailable: true,
allow_no_indices: true,
track_total_hits: true,
terminate_after: 1,
size: 0,
query: {
bool: {
should: [
{ term: { ['event.module']: KUBERNETES } },
{ term: { ['agent.type']: METRICBEAT } },
],
minimum_should_match: 1,
},
},
});
Comment on lines +37 to +53

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As discussed with Roshan, should we focus on otel and leave this and the toggle out?

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will remove them in the next PR and focus on OTel only (so all metricbeat logic here will be removed)


const otelData = await core.elasticsearch.client.asCurrentUser.search({

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm a bit on the fence about checking the existence of data due to possible performance issues. I think it's interesting for the PoC, though, to understand how to provide a good UX

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I couldn't find a better indication if we have otel data or not - I would keep it for now for the PoC purpose but it would be nice to have the semconv entity definition present if the otel data is present or the integration is installed 🤔 probably that's not the right place and we should rethink this and make it scalable

index: 'metrics-*.otel-*',
ignore_unavailable: true,
allow_no_indices: true,
track_total_hits: true,
terminate_after: 1,
size: 0,
query: {
bool: {
filter: [{ term: { ['data_stream.dataset']: '*.otel' } }],

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
filter: [{ term: { ['data_stream.dataset']: '*.otel' } }],
filter: [{ term: { ['data_stream.dataset']: 'k8sclusterreceiver.otel' } }],

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I found other indices that can also indicate we have k8s data, but this is too broad, I agree. I am wondering if the k8sclusterreceiver.otel will always have data in case of Kubernetes data? It should be the case right?

},
},
});

// TODO Type fick for the response
const hasEcsData = metricbeatData?.hits?.total?.value !== 0;
const hasOtelData = otelData?.hits?.total?.value !== 0;

const [installedPackage, ...navigationOverrides] = await Promise.all([
packageClient?.getInstallation(KUBERNETES),
core.savedObjects.client.get<NavigationOverridesSavedObject>(
Expand All @@ -45,90 +82,167 @@ const infraSideNavRoute = createServerRoute({
),
]);

if (!installedPackage && !navigationOverrides) {
const packageInstalled = installedPackage ? [installedPackage] : [];

// Maybe separate the ecs / otel cases in the future
if ((hasEcsData ?? hasOtelData) && !installedPackage) {
// System package is always required
await packageClient?.ensureInstalledPackage({ pkgName: 'system' });
// Kubernetes package is required for both classic kubernetes and otel
await packageClient?.ensureInstalledPackage({ pkgName: 'kubernetes' });
const installedKubernetes = await packageClient?.getInstallation(KUBERNETES);
if (installedKubernetes) packageInstalled.push(installedKubernetes);
// Kubernetes otel package is required only for otel
if (hasOtelData && !hasEcsData) {
await packageClient?.ensureInstalledPackage({ pkgName: 'kubernetes_otel' });
const installedOtelKubernetes = await packageClient?.getInstallation('kubernetes_otel');
if (installedOtelKubernetes) packageInstalled.push(installedOtelKubernetes);
}
}
Comment on lines +85 to +101

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this the same from the onboarding flow?


if ((!installedPackage || packageInstalled.length === 0) && !navigationOverrides) {
return [];
}

// Mock data simulating the installed package's items returned by installedPackage.installed_kibana
const mockInstalledPackage = installedPackage
? [
{
id: 'kubernetes-0a672d50-bcb1-11ec-b64f-7dd6e8e82013',
entityType: 'k8s.cronjob',
sideNavTitle: 'Cron jobs',
sideNavOrder: 900,
type: 'dashboard',
},
{
id: 'kubernetes-21694370-bcb2-11ec-b64f-7dd6e8e82013',
entityType: 'k8s.statefulset',
sideNavTitle: 'Stateful sets',
sideNavOrder: 600,
type: 'dashboard',
},
{
id: 'kubernetes-3912d9a0-bcb2-11ec-b64f-7dd6e8e82013',
entityType: 'k8s.volume',
sideNavTitle: 'Volumes',
sideNavOrder: 500,
type: 'dashboard',
},
{
id: 'kubernetes-3d4d9290-bcb1-11ec-b64f-7dd6e8e82013',
entityType: 'k8s.pod',
sideNavTitle: 'Pods',
sideNavOrder: 300,
type: 'dashboard',
},
const k8sEntitiesSemConv = [
{
id: 'entity.k8s.cluster', // -> entityType
type: 'entity',
stability: 'development',
name: 'k8s.cluster',
brief: 'A Kubernetes Cluster.',
attributes: [{ ref: 'k8s.cluster.name' }, { ref: 'k8s.cluster.uid' }],
},
{
id: 'entity.k8s.node',
type: 'entity',
stability: 'development',
name: 'k8s.node',
brief: 'A Kubernetes Node object.',
attributes: [
{ ref: 'k8s.node.name' },
{ ref: 'k8s.node.uid' },
{
id: 'kubernetes-5be46210-bcb1-11ec-b64f-7dd6e8e82013',
entityType: 'k8s.deployment',
sideNavTitle: 'Deployments',
sideNavOrder: 400,
type: 'dashboard',
ref: 'k8s.node.label',
requirement_level: 'opt_in',
},
{
id: 'kubernetes-85879010-bcb1-11ec-b64f-7dd6e8e82013',
entityType: 'k8s.daemonset',
sideNavTitle: 'Daemon sets',
sideNavOrder: 700,
type: 'dashboard',
ref: 'k8s.node.annotation',
requirement_level: 'opt_in',
},
{
id: 'kubernetes-9bf990a0-bcb1-11ec-b64f-7dd6e8e82013',
entityType: 'k8s.job',
sideNavTitle: 'Jobs',
sideNavOrder: 800,
type: 'dashboard',
},
{
id: 'kubernetes-b945b7b0-bcb1-11ec-b64f-7dd6e8e82013',
entityType: 'k8s.node',
sideNavTitle: 'Nodes',
sideNavOrder: 200,
type: 'dashboard',
},
{
id: 'kubernetes-f4dc26db-1b53-4ea2-a78b-1bfab8ea267c',
sideNavTitle: 'Overview',
sideNavOrder: 100,
type: 'dashboard',
},
{
id: 'kubernetes-ff1b3850-bcb1-11ec-b64f-7dd6e8e82013',
entityType: 'k8s.service',
sideNavTitle: 'Services',
sideNavOrder: 800,
type: 'dashboard',
},
]
],
},
];
const otelMenuItems = !hasEcsData
? k8sEntitiesSemConv.map((entity, index) => ({
// id: `kubernetes_otel-${entity.id}`,
id: `kubernetes_otel-cluster-overview`,
entityType: entity.id,
sideNavTitle: entity.brief,
sideNavOrder: (index || 1) * 100,
type: 'dashboard',
}))
: [];

// Mock data simulating the installed package's items returned by installedPackage.installed_kibana
const mockInstalledPackage =
installedPackage && packageInstalled.length > 0
? [
...(hasOtelData && !hasEcsData
? [
{
id: 'kubernetes_otel-cluster-overview',
entityType: 'k8s.overview',
sideNavTitle: 'Overview (Otel)',
sideNavOrder: 100,
type: 'dashboard',
},
]
: [
{
id: 'kubernetes-0a672d50-bcb1-11ec-b64f-7dd6e8e82013',
entityType: 'k8s.cronjob',
sideNavTitle: 'Cron jobs',
sideNavOrder: 900,
type: 'dashboard',
},
{
id: 'kubernetes-21694370-bcb2-11ec-b64f-7dd6e8e82013',
entityType: 'k8s.statefulset',
sideNavTitle: 'Stateful sets',
sideNavOrder: 600,
type: 'dashboard',
},
{
id: 'kubernetes-3912d9a0-bcb2-11ec-b64f-7dd6e8e82013',
entityType: 'k8s.volume',
sideNavTitle: 'Volumes',
sideNavOrder: 500,
type: 'dashboard',
},
{
id: 'kubernetes-3d4d9290-bcb1-11ec-b64f-7dd6e8e82013',
entityType: 'k8s.pod',
sideNavTitle: 'Pods',
sideNavOrder: 300,
type: 'dashboard',
},
{
id: 'kubernetes-5be46210-bcb1-11ec-b64f-7dd6e8e82013',
entityType: 'k8s.deployment',
sideNavTitle: 'Deployments',
sideNavOrder: 400,
type: 'dashboard',
},
{
id: 'kubernetes-85879010-bcb1-11ec-b64f-7dd6e8e82013',
entityType: 'k8s.daemonset',
sideNavTitle: 'Daemon sets',
sideNavOrder: 700,
type: 'dashboard',
},
{
id: 'kubernetes-9bf990a0-bcb1-11ec-b64f-7dd6e8e82013',
entityType: 'k8s.job',
sideNavTitle: 'Jobs',
sideNavOrder: 800,
type: 'dashboard',
},
{
id: 'kubernetes-b945b7b0-bcb1-11ec-b64f-7dd6e8e82013',
entityType: 'k8s.node',
sideNavTitle: 'Nodes',
sideNavOrder: 200,
type: 'dashboard',
},
{
id: 'kubernetes-f4dc26db-1b53-4ea2-a78b-1bfab8ea267c',
sideNavTitle: 'Overview',
sideNavOrder: 100,
type: 'dashboard',
},
{
id: 'kubernetes-ff1b3850-bcb1-11ec-b64f-7dd6e8e82013',
entityType: 'k8s.service',
sideNavTitle: 'Services',
sideNavOrder: 800,
type: 'dashboard',
},
]),
]
: [];

const integrationSubItems =
mockInstalledPackage
.filter((p) => !!p.sideNavTitle)
.sort((a, b) => (a.sideNavOrder ?? 0) - (b.sideNavOrder ?? 0)) ?? [];

integrationSubItems.push(
...(otelMenuItems
.filter((p) => !!p.sideNavTitle)
.sort((a, b) => (a.sideNavOrder ?? 0) - (b.sideNavOrder ?? 0)) ?? [])
);

const integrationNavigation =
integrationSubItems.length > 0
? [
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -16,8 +16,10 @@ export interface ObservabilityNavigationServer {

// eslint-disable-next-line @typescript-eslint/no-empty-interface
export interface ObservabilityNavigationPluginSetup {}
// eslint-disable-next-line @typescript-eslint/no-empty-interface
export interface ObservabilityNavigationPluginStart {}

export interface ObservabilityNavigationPluginStart {
core: CoreStart;
}

// eslint-disable-next-line @typescript-eslint/no-empty-interface
export interface ObservabilityNavigationPluginSetupDependencies {}
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/

import React from 'react';
import { EuiLink } from '@elastic/eui';
import type { ObservabilityOnboardingLocatorParams } from '@kbn/deeplinks-observability';
import { OBSERVABILITY_ONBOARDING_LOCATOR } from '@kbn/deeplinks-observability';
import { useKibana } from '@kbn/kibana-react-plugin/public';
import type { SharePublicStart } from '@kbn/share-plugin/public/plugin';
import { i18n } from '@kbn/i18n';

const ADD_DATA_KUBERNETES_LABEL = i18n.translate('xpack.infra.metricsHeaderAddDataButtonLabel', {
defaultMessage: 'Add Kubernetes data',
});

export const AddKubernetesDataLink = () => {
const { share } = useKibana<{ share: SharePublicStart }>().services;
const onboardingLocator = share?.url.locators.get<ObservabilityOnboardingLocatorParams>(
OBSERVABILITY_ONBOARDING_LOCATOR
);

return (
<EuiLink
data-test-subj="infraAddDataLink"
href={onboardingLocator?.getRedirectUrl({
category: 'kubernetes',
})}
color="primary"
>
{ADD_DATA_KUBERNETES_LABEL}
</EuiLink>
);
};
Original file line number Diff line number Diff line change
Expand Up @@ -12,8 +12,12 @@ import type { DashboardApi, DashboardCreationOptions } from '@kbn/dashboard-plug
import { KUBERNETES_DASHBOARD_LOCATOR_ID } from '@kbn/observability-shared-plugin/common';
import type { SerializableRecord } from '@kbn/utility-types';
import type { DashboardState } from '@kbn/dashboard-plugin/common';
import { EuiLoadingSpinner } from '@elastic/eui';
import { FETCH_STATUS } from '@kbn/observability-shared-plugin/public';
import { useKibanaContextForPlugin } from '../../../../../hooks/use_kibana';
import { useDatePickerContext } from '../../hooks/use_date_picker';
import { AddKubernetesDataLink } from '../add_kubernetes_data/add_kubernetes_data';
import { useFetchDashboardById } from '../../hooks/use_fetch_dashboard_by_id';

export const RenderDashboard = ({ dashboardId }: { dashboardId: string }) => {
const {
Expand Down Expand Up @@ -61,6 +65,15 @@ export const RenderDashboard = ({ dashboardId }: { dashboardId: string }) => {
dashboard.setQuery({ query: '', language: 'kuery' });
}, [dashboard, dashboardId, from, to]);

const { data: dashboardData, status } = useFetchDashboardById(dashboardId);

if (!dashboardData && status === FETCH_STATUS.LOADING) {
return <EuiLoadingSpinner size="xl" />;
}
if (!dashboardData && status !== FETCH_STATUS.LOADING && dashboardId.startsWith('kubernetes')) {
return <AddKubernetesDataLink />;
}

return (
<DashboardRenderer
locator={locator}
Expand Down