Skip to content

Optimize tsdb data stream timestamp parsing if ingest pipeline is used #84046

@martijnvg

Description

@martijnvg

For tsdb data streams, the @timestamp field is used to determine to which backing index a data stream resolves to.

Currently in IndexAbstraction.DataStream#getWriteIndex(...) the source of the document is parsed in order to extract the @timestamp field. If an ingest pipeline is active then this parsing is redundant, because ingest parse the source of a document to into a map of maps in order for a pipeline to process it.

After the final pipeline has executed, ingest can extract the @timestamp field and attach it as String/Long to the IndexRequest class as a field. Then IndexAbstraction.DataStream#getWriteIndex(...) can check for whether that field has been set and if so use that parse it into a Instant instance.

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions