Skip to content

[Fleet] omit system properties when syncing ingest pipelines #241096

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
juliaElastic merged 3 commits into from
Oct 30, 2025
Merged

[Fleet] omit system properties when syncing ingest pipelines #241096

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
12e5c6f
omit system properties when syncing ingest pipelines
juliaElastic Oct 29, 2025
95f64da
omit system properties instead
juliaElastic Oct 29, 2025
fbedcea
address review comment
juliaElastic Oct 29, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 11 additions & 0 deletions x-pack/platform/plugins/shared/fleet/common/types/models/epm.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,8 @@

import type { estypes } from '@elastic/elasticsearch';

import type { IngestPipeline } from '@elastic/elasticsearch/lib/api/types';

import type {
ASSETS_SAVED_OBJECT_TYPE,
agentAssetTypes,
Expand Down Expand Up @@ -799,6 +801,15 @@ export interface IndexTemplate {
modified_date_millis?: number;
}

export interface IngestPipelineWithDateFields extends IngestPipeline {
// These properties are returned on ES read operations and
// not allowed to be set on ES write operations
created_date?: number;
created_date_millis?: number;
modified_date?: number;
modified_date_millis?: number;
}

export interface ESAssetMetadata {
package?: {
name: string;
Expand Down
6 changes: 6 additions & 0 deletions x-pack/platform/plugins/shared/fleet/server/tasks/sync_integrations/custom_assets.test.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -702,6 +702,11 @@ describe('custom assets', () => {
},
],
version: 1,
description: 'description pipeline',
created_date: '2024-01-01T12:00:00.000Z',
created_date_millis: 1704110400000,
modified_date: '2025-01-01T12:00:00.000Z',
modified_date_millis: 1735732800000,
},
type: 'ingest_pipeline',
},
Expand All @@ -721,6 +726,7 @@ describe('custom assets', () => {
},
],
version: 1,
description: 'description pipeline',
},
expect.anything()
);
Expand Down
13 changes: 12 additions & 1 deletion x-pack/platform/plugins/shared/fleet/server/tasks/sync_integrations/custom_assets.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,8 @@ import { retryTransientEsErrors } from '../../services/epm/elasticsearch/retry';
import { packagePolicyService } from '../../services';
import { SO_SEARCH_LIMIT } from '../../constants';

import type { IngestPipelineWithDateFields } from '../../../common/types';

import type { CustomAssetsData, IntegrationsData, SyncIntegrationsData } from './model';

const DELETED_ASSET_TTL = 7 * 24 * 60 * 60 * 1000; // 7 days
Expand Down Expand Up @@ -297,12 +299,21 @@ async function updateIngestPipeline(

if (shouldUpdatePipeline) {
logger.debug(`Updating ingest pipeline: ${customAsset.name}`);

// Remove system-managed properties (dates) that cannot be set during create/update of ingest pipelines
const {
created_date: createdDate,
created_date_millis: createdDateMillis,
modified_date: modifiedDate,
modified_date_millis: modifiedDateMillis,
...updatedIngestPipeline
} = customAsset.pipeline as IngestPipelineWithDateFields;
return retryTransientEsErrors(
() =>
esClient.ingest.putPipeline(
{
Comment thread
juliaElastic marked this conversation as resolved.
id: customAsset.name,
...customAsset.pipeline,
...updatedIngestPipeline,
},
{
signal: abortController.signal,
Expand Down
Loading