-
Notifications
You must be signed in to change notification settings - Fork 2.3k
Expand fetch phase profiling to support inner hits and top hits aggregation phases #18936
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Expand fetch phase profiling to support inner hits and top hits aggregation phases #18936
Conversation
Signed-off-by: Andre van de Ven <[email protected]>
Signed-off-by: Andre van de Ven <[email protected]>
Signed-off-by: Andre van de Ven <[email protected]>
Signed-off-by: Andre van de Ven <[email protected]>
Signed-off-by: Andre van de Ven <[email protected]>
Signed-off-by: Andre van de Ven <[email protected]>
|
❌ Gradle check result for d760f9f: FAILURE Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change? |
Signed-off-by: Andre van de Ven <[email protected]>
Signed-off-by: Andre van de Ven <[email protected]>
|
❌ Gradle check result for d14fe2d: FAILURE Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change? |
Signed-off-by: Andre van de Ven <[email protected]>
|
❌ Gradle check result for 96bc00f: FAILURE Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change? |
Signed-off-by: Andre van de Ven <[email protected]>
|
❌ Gradle check result for 46b9140: FAILURE Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change? |
Signed-off-by: Andre van de Ven <[email protected]>
|
❌ Gradle check result for 5db9865: FAILURE Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change? |
Signed-off-by: Andre van de Ven <[email protected]>
|
❌ Gradle check result for 7b76295: FAILURE Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change? |
…segment search Signed-off-by: Andre van de Ven <[email protected]>
Signed-off-by: Andre van de Ven <[email protected]>
Signed-off-by: Andre van de Ven <[email protected]>
|
❌ Gradle check result for c8e2ae1: FAILURE Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change? |
…ation due to concurrency Signed-off-by: Andre van de Ven <[email protected]>
|
❌ Gradle check result for 7267d03: FAILURE Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change? |
00c407f to
9738ccb
Compare
|
@msfroh I updated the rest tests to skip versions before 3.3, so should be good to go now! EDIT: Gradle check needs to run again because I fixed changelog merge issues. Will ping again when it finishes running! |
Signed-off-by: Andre van de Ven <[email protected]>
|
@msfroh good to go now! |
…gation phases (opensearch-project#18936) --------- Signed-off-by: Andre van de Ven <[email protected]> Signed-off-by: Andre van de Ven <[email protected]> Signed-off-by: Andre van de Ven <[email protected]> Co-authored-by: Andre van de Ven <[email protected]>
…gation phases (opensearch-project#18936) --------- Signed-off-by: Andre van de Ven <[email protected]> Signed-off-by: Andre van de Ven <[email protected]> Signed-off-by: Andre van de Ven <[email protected]> Co-authored-by: Andre van de Ven <[email protected]>
…gation phases (opensearch-project#18936) --------- Signed-off-by: Andre van de Ven <[email protected]> Signed-off-by: Andre van de Ven <[email protected]> Signed-off-by: Andre van de Ven <[email protected]> Co-authored-by: Andre van de Ven <[email protected]>
* Add overload for channelFactory Signed-off-by: Rajat Gupta <[email protected]> * Fix tests Signed-off-by: Rajat Gupta <[email protected]> * Add Changelog entry Signed-off-by: Rajat Gupta <[email protected]> * Fix conflicts Signed-off-by: Rajat Gupta <[email protected]> * When update operations fail during preparation (e.g., version conflicts), (#18917) TransportShardBulkAction still triggers refresh even though no actual writes occurred. This fix checks if locationToSync is null (indicating no writes) and prevents refresh in such cases. Fixes #15261 Signed-off-by: Atri Sharma <[email protected]> * Remove all entries from changelog to be released in 3.2 (#18989) Signed-off-by: Andrew Ross <[email protected]> * Add temporal routing processors for time-based document routing (#18966) Implements TemporalRoutingProcessor for ingest pipelines and TemporalRoutingSearchProcessor for search pipelines based on RFC #18920. Features: - Route documents to shards based on timestamp fields - Support hour, day, week, and month granularities - Optional hash bucketing for better distribution - Automatic search routing to relevant time ranges - ISO week format support The processors enable efficient time-based data organization for log and metrics workloads by co-locating documents from the same time period on the same shards. --------- Signed-off-by: Atri Sharma <[email protected]> * Add CompletionStage variants to methods in the Client Interface and default to ActionListener impl (#18998) * Add CompletableFuture variables to methods in the Client Interface and default to ActionListener impl Signed-off-by: Craig Perkins <[email protected]> * Add to CHANGELOG Signed-off-by: Craig Perkins <[email protected]> * Fix typo in CHANGELOG Signed-off-by: Craig Perkins <[email protected]> * Switch to CompletionStage Signed-off-by: Craig Perkins <[email protected]> * Update CHANGELOG entry Signed-off-by: Craig Perkins <[email protected]> --------- Signed-off-by: Craig Perkins <[email protected]> * Expand fetch phase profiling to support inner hits and top hits aggregation phases (#18936) --------- Signed-off-by: Andre van de Ven <[email protected]> Signed-off-by: Andre van de Ven <[email protected]> Signed-off-by: Andre van de Ven <[email protected]> Co-authored-by: Andre van de Ven <[email protected]> * IllegalArgumentException when scroll ID has a node no longer part of the Cluster (#19031) --------- Signed-off-by: Anurag Rai <[email protected]> Signed-off-by: Anurag Rai <[email protected]> * Add Changelog entry Signed-off-by: Rajat Gupta <[email protected]> * Add secondary constructor Signed-off-by: Rajat Gupta <[email protected]> * Modify changelog Signed-off-by: Rajat Gupta <[email protected]> * Update changelog Signed-off-by: Rajat Gupta <[email protected]> * Add another constructor to fix breaking change check Signed-off-by: Rajat Gupta <[email protected]> --------- Signed-off-by: Rajat Gupta <[email protected]> Signed-off-by: Atri Sharma <[email protected]> Signed-off-by: Andrew Ross <[email protected]> Signed-off-by: Craig Perkins <[email protected]> Signed-off-by: Andre van de Ven <[email protected]> Signed-off-by: Andre van de Ven <[email protected]> Signed-off-by: Andre van de Ven <[email protected]> Signed-off-by: Anurag Rai <[email protected]> Signed-off-by: Anurag Rai <[email protected]> Co-authored-by: Rajat Gupta <[email protected]> Co-authored-by: Atri Sharma <[email protected]> Co-authored-by: Andrew Ross <[email protected]> Co-authored-by: Craig Perkins <[email protected]> Co-authored-by: Andre van de Ven <[email protected]> Co-authored-by: Andre van de Ven <[email protected]> Co-authored-by: Anurag Rai <[email protected]>
…gation phases (opensearch-project#18936) --------- Signed-off-by: Andre van de Ven <[email protected]> Signed-off-by: Andre van de Ven <[email protected]> Signed-off-by: Andre van de Ven <[email protected]> Co-authored-by: Andre van de Ven <[email protected]>
* Add overload for channelFactory Signed-off-by: Rajat Gupta <[email protected]> * Fix tests Signed-off-by: Rajat Gupta <[email protected]> * Add Changelog entry Signed-off-by: Rajat Gupta <[email protected]> * Fix conflicts Signed-off-by: Rajat Gupta <[email protected]> * When update operations fail during preparation (e.g., version conflicts), (opensearch-project#18917) TransportShardBulkAction still triggers refresh even though no actual writes occurred. This fix checks if locationToSync is null (indicating no writes) and prevents refresh in such cases. Fixes opensearch-project#15261 Signed-off-by: Atri Sharma <[email protected]> * Remove all entries from changelog to be released in 3.2 (opensearch-project#18989) Signed-off-by: Andrew Ross <[email protected]> * Add temporal routing processors for time-based document routing (opensearch-project#18966) Implements TemporalRoutingProcessor for ingest pipelines and TemporalRoutingSearchProcessor for search pipelines based on RFC opensearch-project#18920. Features: - Route documents to shards based on timestamp fields - Support hour, day, week, and month granularities - Optional hash bucketing for better distribution - Automatic search routing to relevant time ranges - ISO week format support The processors enable efficient time-based data organization for log and metrics workloads by co-locating documents from the same time period on the same shards. --------- Signed-off-by: Atri Sharma <[email protected]> * Add CompletionStage variants to methods in the Client Interface and default to ActionListener impl (opensearch-project#18998) * Add CompletableFuture variables to methods in the Client Interface and default to ActionListener impl Signed-off-by: Craig Perkins <[email protected]> * Add to CHANGELOG Signed-off-by: Craig Perkins <[email protected]> * Fix typo in CHANGELOG Signed-off-by: Craig Perkins <[email protected]> * Switch to CompletionStage Signed-off-by: Craig Perkins <[email protected]> * Update CHANGELOG entry Signed-off-by: Craig Perkins <[email protected]> --------- Signed-off-by: Craig Perkins <[email protected]> * Expand fetch phase profiling to support inner hits and top hits aggregation phases (opensearch-project#18936) --------- Signed-off-by: Andre van de Ven <[email protected]> Signed-off-by: Andre van de Ven <[email protected]> Signed-off-by: Andre van de Ven <[email protected]> Co-authored-by: Andre van de Ven <[email protected]> * IllegalArgumentException when scroll ID has a node no longer part of the Cluster (opensearch-project#19031) --------- Signed-off-by: Anurag Rai <[email protected]> Signed-off-by: Anurag Rai <[email protected]> * Add Changelog entry Signed-off-by: Rajat Gupta <[email protected]> * Add secondary constructor Signed-off-by: Rajat Gupta <[email protected]> * Modify changelog Signed-off-by: Rajat Gupta <[email protected]> * Update changelog Signed-off-by: Rajat Gupta <[email protected]> * Add another constructor to fix breaking change check Signed-off-by: Rajat Gupta <[email protected]> --------- Signed-off-by: Rajat Gupta <[email protected]> Signed-off-by: Atri Sharma <[email protected]> Signed-off-by: Andrew Ross <[email protected]> Signed-off-by: Craig Perkins <[email protected]> Signed-off-by: Andre van de Ven <[email protected]> Signed-off-by: Andre van de Ven <[email protected]> Signed-off-by: Andre van de Ven <[email protected]> Signed-off-by: Anurag Rai <[email protected]> Signed-off-by: Anurag Rai <[email protected]> Co-authored-by: Rajat Gupta <[email protected]> Co-authored-by: Atri Sharma <[email protected]> Co-authored-by: Andrew Ross <[email protected]> Co-authored-by: Craig Perkins <[email protected]> Co-authored-by: Andre van de Ven <[email protected]> Co-authored-by: Andre van de Ven <[email protected]> Co-authored-by: Anurag Rai <[email protected]>
* Add overload for channelFactory Signed-off-by: Rajat Gupta <[email protected]> * Fix tests Signed-off-by: Rajat Gupta <[email protected]> * Add Changelog entry Signed-off-by: Rajat Gupta <[email protected]> * Fix conflicts Signed-off-by: Rajat Gupta <[email protected]> * When update operations fail during preparation (e.g., version conflicts), (opensearch-project#18917) TransportShardBulkAction still triggers refresh even though no actual writes occurred. This fix checks if locationToSync is null (indicating no writes) and prevents refresh in such cases. Fixes opensearch-project#15261 Signed-off-by: Atri Sharma <[email protected]> * Remove all entries from changelog to be released in 3.2 (opensearch-project#18989) Signed-off-by: Andrew Ross <[email protected]> * Add temporal routing processors for time-based document routing (opensearch-project#18966) Implements TemporalRoutingProcessor for ingest pipelines and TemporalRoutingSearchProcessor for search pipelines based on RFC opensearch-project#18920. Features: - Route documents to shards based on timestamp fields - Support hour, day, week, and month granularities - Optional hash bucketing for better distribution - Automatic search routing to relevant time ranges - ISO week format support The processors enable efficient time-based data organization for log and metrics workloads by co-locating documents from the same time period on the same shards. --------- Signed-off-by: Atri Sharma <[email protected]> * Add CompletionStage variants to methods in the Client Interface and default to ActionListener impl (opensearch-project#18998) * Add CompletableFuture variables to methods in the Client Interface and default to ActionListener impl Signed-off-by: Craig Perkins <[email protected]> * Add to CHANGELOG Signed-off-by: Craig Perkins <[email protected]> * Fix typo in CHANGELOG Signed-off-by: Craig Perkins <[email protected]> * Switch to CompletionStage Signed-off-by: Craig Perkins <[email protected]> * Update CHANGELOG entry Signed-off-by: Craig Perkins <[email protected]> --------- Signed-off-by: Craig Perkins <[email protected]> * Expand fetch phase profiling to support inner hits and top hits aggregation phases (opensearch-project#18936) --------- Signed-off-by: Andre van de Ven <[email protected]> Signed-off-by: Andre van de Ven <[email protected]> Signed-off-by: Andre van de Ven <[email protected]> Co-authored-by: Andre van de Ven <[email protected]> * IllegalArgumentException when scroll ID has a node no longer part of the Cluster (opensearch-project#19031) --------- Signed-off-by: Anurag Rai <[email protected]> Signed-off-by: Anurag Rai <[email protected]> * Add Changelog entry Signed-off-by: Rajat Gupta <[email protected]> * Add secondary constructor Signed-off-by: Rajat Gupta <[email protected]> * Modify changelog Signed-off-by: Rajat Gupta <[email protected]> * Update changelog Signed-off-by: Rajat Gupta <[email protected]> * Add another constructor to fix breaking change check Signed-off-by: Rajat Gupta <[email protected]> --------- Signed-off-by: Rajat Gupta <[email protected]> Signed-off-by: Atri Sharma <[email protected]> Signed-off-by: Andrew Ross <[email protected]> Signed-off-by: Craig Perkins <[email protected]> Signed-off-by: Andre van de Ven <[email protected]> Signed-off-by: Andre van de Ven <[email protected]> Signed-off-by: Andre van de Ven <[email protected]> Signed-off-by: Anurag Rai <[email protected]> Signed-off-by: Anurag Rai <[email protected]> Co-authored-by: Rajat Gupta <[email protected]> Co-authored-by: Atri Sharma <[email protected]> Co-authored-by: Andrew Ross <[email protected]> Co-authored-by: Craig Perkins <[email protected]> Co-authored-by: Andre van de Ven <[email protected]> Co-authored-by: Andre van de Ven <[email protected]> Co-authored-by: Anurag Rai <[email protected]>
…gation phases (opensearch-project#18936) --------- Signed-off-by: Andre van de Ven <[email protected]> Signed-off-by: Andre van de Ven <[email protected]> Signed-off-by: Andre van de Ven <[email protected]> Co-authored-by: Andre van de Ven <[email protected]>
* Add overload for channelFactory Signed-off-by: Rajat Gupta <[email protected]> * Fix tests Signed-off-by: Rajat Gupta <[email protected]> * Add Changelog entry Signed-off-by: Rajat Gupta <[email protected]> * Fix conflicts Signed-off-by: Rajat Gupta <[email protected]> * When update operations fail during preparation (e.g., version conflicts), (opensearch-project#18917) TransportShardBulkAction still triggers refresh even though no actual writes occurred. This fix checks if locationToSync is null (indicating no writes) and prevents refresh in such cases. Fixes opensearch-project#15261 Signed-off-by: Atri Sharma <[email protected]> * Remove all entries from changelog to be released in 3.2 (opensearch-project#18989) Signed-off-by: Andrew Ross <[email protected]> * Add temporal routing processors for time-based document routing (opensearch-project#18966) Implements TemporalRoutingProcessor for ingest pipelines and TemporalRoutingSearchProcessor for search pipelines based on RFC opensearch-project#18920. Features: - Route documents to shards based on timestamp fields - Support hour, day, week, and month granularities - Optional hash bucketing for better distribution - Automatic search routing to relevant time ranges - ISO week format support The processors enable efficient time-based data organization for log and metrics workloads by co-locating documents from the same time period on the same shards. --------- Signed-off-by: Atri Sharma <[email protected]> * Add CompletionStage variants to methods in the Client Interface and default to ActionListener impl (opensearch-project#18998) * Add CompletableFuture variables to methods in the Client Interface and default to ActionListener impl Signed-off-by: Craig Perkins <[email protected]> * Add to CHANGELOG Signed-off-by: Craig Perkins <[email protected]> * Fix typo in CHANGELOG Signed-off-by: Craig Perkins <[email protected]> * Switch to CompletionStage Signed-off-by: Craig Perkins <[email protected]> * Update CHANGELOG entry Signed-off-by: Craig Perkins <[email protected]> --------- Signed-off-by: Craig Perkins <[email protected]> * Expand fetch phase profiling to support inner hits and top hits aggregation phases (opensearch-project#18936) --------- Signed-off-by: Andre van de Ven <[email protected]> Signed-off-by: Andre van de Ven <[email protected]> Signed-off-by: Andre van de Ven <[email protected]> Co-authored-by: Andre van de Ven <[email protected]> * IllegalArgumentException when scroll ID has a node no longer part of the Cluster (opensearch-project#19031) --------- Signed-off-by: Anurag Rai <[email protected]> Signed-off-by: Anurag Rai <[email protected]> * Add Changelog entry Signed-off-by: Rajat Gupta <[email protected]> * Add secondary constructor Signed-off-by: Rajat Gupta <[email protected]> * Modify changelog Signed-off-by: Rajat Gupta <[email protected]> * Update changelog Signed-off-by: Rajat Gupta <[email protected]> * Add another constructor to fix breaking change check Signed-off-by: Rajat Gupta <[email protected]> --------- Signed-off-by: Rajat Gupta <[email protected]> Signed-off-by: Atri Sharma <[email protected]> Signed-off-by: Andrew Ross <[email protected]> Signed-off-by: Craig Perkins <[email protected]> Signed-off-by: Andre van de Ven <[email protected]> Signed-off-by: Andre van de Ven <[email protected]> Signed-off-by: Andre van de Ven <[email protected]> Signed-off-by: Anurag Rai <[email protected]> Signed-off-by: Anurag Rai <[email protected]> Co-authored-by: Rajat Gupta <[email protected]> Co-authored-by: Atri Sharma <[email protected]> Co-authored-by: Andrew Ross <[email protected]> Co-authored-by: Craig Perkins <[email protected]> Co-authored-by: Andre van de Ven <[email protected]> Co-authored-by: Andre van de Ven <[email protected]> Co-authored-by: Anurag Rai <[email protected]>
* Add overload for channelFactory Signed-off-by: Rajat Gupta <[email protected]> * Fix tests Signed-off-by: Rajat Gupta <[email protected]> * Add Changelog entry Signed-off-by: Rajat Gupta <[email protected]> * Fix conflicts Signed-off-by: Rajat Gupta <[email protected]> * When update operations fail during preparation (e.g., version conflicts), (opensearch-project#18917) TransportShardBulkAction still triggers refresh even though no actual writes occurred. This fix checks if locationToSync is null (indicating no writes) and prevents refresh in such cases. Fixes opensearch-project#15261 Signed-off-by: Atri Sharma <[email protected]> * Remove all entries from changelog to be released in 3.2 (opensearch-project#18989) Signed-off-by: Andrew Ross <[email protected]> * Add temporal routing processors for time-based document routing (opensearch-project#18966) Implements TemporalRoutingProcessor for ingest pipelines and TemporalRoutingSearchProcessor for search pipelines based on RFC opensearch-project#18920. Features: - Route documents to shards based on timestamp fields - Support hour, day, week, and month granularities - Optional hash bucketing for better distribution - Automatic search routing to relevant time ranges - ISO week format support The processors enable efficient time-based data organization for log and metrics workloads by co-locating documents from the same time period on the same shards. --------- Signed-off-by: Atri Sharma <[email protected]> * Add CompletionStage variants to methods in the Client Interface and default to ActionListener impl (opensearch-project#18998) * Add CompletableFuture variables to methods in the Client Interface and default to ActionListener impl Signed-off-by: Craig Perkins <[email protected]> * Add to CHANGELOG Signed-off-by: Craig Perkins <[email protected]> * Fix typo in CHANGELOG Signed-off-by: Craig Perkins <[email protected]> * Switch to CompletionStage Signed-off-by: Craig Perkins <[email protected]> * Update CHANGELOG entry Signed-off-by: Craig Perkins <[email protected]> --------- Signed-off-by: Craig Perkins <[email protected]> * Expand fetch phase profiling to support inner hits and top hits aggregation phases (opensearch-project#18936) --------- Signed-off-by: Andre van de Ven <[email protected]> Signed-off-by: Andre van de Ven <[email protected]> Signed-off-by: Andre van de Ven <[email protected]> Co-authored-by: Andre van de Ven <[email protected]> * IllegalArgumentException when scroll ID has a node no longer part of the Cluster (opensearch-project#19031) --------- Signed-off-by: Anurag Rai <[email protected]> Signed-off-by: Anurag Rai <[email protected]> * Add Changelog entry Signed-off-by: Rajat Gupta <[email protected]> * Add secondary constructor Signed-off-by: Rajat Gupta <[email protected]> * Modify changelog Signed-off-by: Rajat Gupta <[email protected]> * Update changelog Signed-off-by: Rajat Gupta <[email protected]> * Add another constructor to fix breaking change check Signed-off-by: Rajat Gupta <[email protected]> --------- Signed-off-by: Rajat Gupta <[email protected]> Signed-off-by: Atri Sharma <[email protected]> Signed-off-by: Andrew Ross <[email protected]> Signed-off-by: Craig Perkins <[email protected]> Signed-off-by: Andre van de Ven <[email protected]> Signed-off-by: Andre van de Ven <[email protected]> Signed-off-by: Andre van de Ven <[email protected]> Signed-off-by: Anurag Rai <[email protected]> Signed-off-by: Anurag Rai <[email protected]> Co-authored-by: Rajat Gupta <[email protected]> Co-authored-by: Atri Sharma <[email protected]> Co-authored-by: Andrew Ross <[email protected]> Co-authored-by: Craig Perkins <[email protected]> Co-authored-by: Andre van de Ven <[email protected]> Co-authored-by: Andre van de Ven <[email protected]> Co-authored-by: Anurag Rai <[email protected]>
* Add overload for channelFactory Signed-off-by: Rajat Gupta <[email protected]> * Fix tests Signed-off-by: Rajat Gupta <[email protected]> * Add Changelog entry Signed-off-by: Rajat Gupta <[email protected]> * Fix conflicts Signed-off-by: Rajat Gupta <[email protected]> * When update operations fail during preparation (e.g., version conflicts), (opensearch-project#18917) TransportShardBulkAction still triggers refresh even though no actual writes occurred. This fix checks if locationToSync is null (indicating no writes) and prevents refresh in such cases. Fixes opensearch-project#15261 Signed-off-by: Atri Sharma <[email protected]> * Remove all entries from changelog to be released in 3.2 (opensearch-project#18989) Signed-off-by: Andrew Ross <[email protected]> * Add temporal routing processors for time-based document routing (opensearch-project#18966) Implements TemporalRoutingProcessor for ingest pipelines and TemporalRoutingSearchProcessor for search pipelines based on RFC opensearch-project#18920. Features: - Route documents to shards based on timestamp fields - Support hour, day, week, and month granularities - Optional hash bucketing for better distribution - Automatic search routing to relevant time ranges - ISO week format support The processors enable efficient time-based data organization for log and metrics workloads by co-locating documents from the same time period on the same shards. --------- Signed-off-by: Atri Sharma <[email protected]> * Add CompletionStage variants to methods in the Client Interface and default to ActionListener impl (opensearch-project#18998) * Add CompletableFuture variables to methods in the Client Interface and default to ActionListener impl Signed-off-by: Craig Perkins <[email protected]> * Add to CHANGELOG Signed-off-by: Craig Perkins <[email protected]> * Fix typo in CHANGELOG Signed-off-by: Craig Perkins <[email protected]> * Switch to CompletionStage Signed-off-by: Craig Perkins <[email protected]> * Update CHANGELOG entry Signed-off-by: Craig Perkins <[email protected]> --------- Signed-off-by: Craig Perkins <[email protected]> * Expand fetch phase profiling to support inner hits and top hits aggregation phases (opensearch-project#18936) --------- Signed-off-by: Andre van de Ven <[email protected]> Signed-off-by: Andre van de Ven <[email protected]> Signed-off-by: Andre van de Ven <[email protected]> Co-authored-by: Andre van de Ven <[email protected]> * IllegalArgumentException when scroll ID has a node no longer part of the Cluster (opensearch-project#19031) --------- Signed-off-by: Anurag Rai <[email protected]> Signed-off-by: Anurag Rai <[email protected]> * Add Changelog entry Signed-off-by: Rajat Gupta <[email protected]> * Add secondary constructor Signed-off-by: Rajat Gupta <[email protected]> * Modify changelog Signed-off-by: Rajat Gupta <[email protected]> * Update changelog Signed-off-by: Rajat Gupta <[email protected]> * Add another constructor to fix breaking change check Signed-off-by: Rajat Gupta <[email protected]> --------- Signed-off-by: Rajat Gupta <[email protected]> Signed-off-by: Atri Sharma <[email protected]> Signed-off-by: Andrew Ross <[email protected]> Signed-off-by: Craig Perkins <[email protected]> Signed-off-by: Andre van de Ven <[email protected]> Signed-off-by: Andre van de Ven <[email protected]> Signed-off-by: Andre van de Ven <[email protected]> Signed-off-by: Anurag Rai <[email protected]> Signed-off-by: Anurag Rai <[email protected]> Co-authored-by: Rajat Gupta <[email protected]> Co-authored-by: Atri Sharma <[email protected]> Co-authored-by: Andrew Ross <[email protected]> Co-authored-by: Craig Perkins <[email protected]> Co-authored-by: Andre van de Ven <[email protected]> Co-authored-by: Andre van de Ven <[email protected]> Co-authored-by: Anurag Rai <[email protected]> Signed-off-by: Ankit Jain <[email protected]>
* Add overload for channelFactory Signed-off-by: Rajat Gupta <[email protected]> * Fix tests Signed-off-by: Rajat Gupta <[email protected]> * Add Changelog entry Signed-off-by: Rajat Gupta <[email protected]> * Fix conflicts Signed-off-by: Rajat Gupta <[email protected]> * When update operations fail during preparation (e.g., version conflicts), (opensearch-project#18917) TransportShardBulkAction still triggers refresh even though no actual writes occurred. This fix checks if locationToSync is null (indicating no writes) and prevents refresh in such cases. Fixes opensearch-project#15261 Signed-off-by: Atri Sharma <[email protected]> * Remove all entries from changelog to be released in 3.2 (opensearch-project#18989) Signed-off-by: Andrew Ross <[email protected]> * Add temporal routing processors for time-based document routing (opensearch-project#18966) Implements TemporalRoutingProcessor for ingest pipelines and TemporalRoutingSearchProcessor for search pipelines based on RFC opensearch-project#18920. Features: - Route documents to shards based on timestamp fields - Support hour, day, week, and month granularities - Optional hash bucketing for better distribution - Automatic search routing to relevant time ranges - ISO week format support The processors enable efficient time-based data organization for log and metrics workloads by co-locating documents from the same time period on the same shards. --------- Signed-off-by: Atri Sharma <[email protected]> * Add CompletionStage variants to methods in the Client Interface and default to ActionListener impl (opensearch-project#18998) * Add CompletableFuture variables to methods in the Client Interface and default to ActionListener impl Signed-off-by: Craig Perkins <[email protected]> * Add to CHANGELOG Signed-off-by: Craig Perkins <[email protected]> * Fix typo in CHANGELOG Signed-off-by: Craig Perkins <[email protected]> * Switch to CompletionStage Signed-off-by: Craig Perkins <[email protected]> * Update CHANGELOG entry Signed-off-by: Craig Perkins <[email protected]> --------- Signed-off-by: Craig Perkins <[email protected]> * Expand fetch phase profiling to support inner hits and top hits aggregation phases (opensearch-project#18936) --------- Signed-off-by: Andre van de Ven <[email protected]> Signed-off-by: Andre van de Ven <[email protected]> Signed-off-by: Andre van de Ven <[email protected]> Co-authored-by: Andre van de Ven <[email protected]> * IllegalArgumentException when scroll ID has a node no longer part of the Cluster (opensearch-project#19031) --------- Signed-off-by: Anurag Rai <[email protected]> Signed-off-by: Anurag Rai <[email protected]> * Add Changelog entry Signed-off-by: Rajat Gupta <[email protected]> * Add secondary constructor Signed-off-by: Rajat Gupta <[email protected]> * Modify changelog Signed-off-by: Rajat Gupta <[email protected]> * Update changelog Signed-off-by: Rajat Gupta <[email protected]> * Add another constructor to fix breaking change check Signed-off-by: Rajat Gupta <[email protected]> --------- Signed-off-by: Rajat Gupta <[email protected]> Signed-off-by: Atri Sharma <[email protected]> Signed-off-by: Andrew Ross <[email protected]> Signed-off-by: Craig Perkins <[email protected]> Signed-off-by: Andre van de Ven <[email protected]> Signed-off-by: Andre van de Ven <[email protected]> Signed-off-by: Andre van de Ven <[email protected]> Signed-off-by: Anurag Rai <[email protected]> Signed-off-by: Anurag Rai <[email protected]> Co-authored-by: Rajat Gupta <[email protected]> Co-authored-by: Atri Sharma <[email protected]> Co-authored-by: Andrew Ross <[email protected]> Co-authored-by: Craig Perkins <[email protected]> Co-authored-by: Andre van de Ven <[email protected]> Co-authored-by: Andre van de Ven <[email protected]> Co-authored-by: Anurag Rai <[email protected]> Signed-off-by: Ankit Jain <[email protected]>
* Add overload for channelFactory Signed-off-by: Rajat Gupta <[email protected]> * Fix tests Signed-off-by: Rajat Gupta <[email protected]> * Add Changelog entry Signed-off-by: Rajat Gupta <[email protected]> * Fix conflicts Signed-off-by: Rajat Gupta <[email protected]> * When update operations fail during preparation (e.g., version conflicts), (opensearch-project#18917) TransportShardBulkAction still triggers refresh even though no actual writes occurred. This fix checks if locationToSync is null (indicating no writes) and prevents refresh in such cases. Fixes opensearch-project#15261 Signed-off-by: Atri Sharma <[email protected]> * Remove all entries from changelog to be released in 3.2 (opensearch-project#18989) Signed-off-by: Andrew Ross <[email protected]> * Add temporal routing processors for time-based document routing (opensearch-project#18966) Implements TemporalRoutingProcessor for ingest pipelines and TemporalRoutingSearchProcessor for search pipelines based on RFC opensearch-project#18920. Features: - Route documents to shards based on timestamp fields - Support hour, day, week, and month granularities - Optional hash bucketing for better distribution - Automatic search routing to relevant time ranges - ISO week format support The processors enable efficient time-based data organization for log and metrics workloads by co-locating documents from the same time period on the same shards. --------- Signed-off-by: Atri Sharma <[email protected]> * Add CompletionStage variants to methods in the Client Interface and default to ActionListener impl (opensearch-project#18998) * Add CompletableFuture variables to methods in the Client Interface and default to ActionListener impl Signed-off-by: Craig Perkins <[email protected]> * Add to CHANGELOG Signed-off-by: Craig Perkins <[email protected]> * Fix typo in CHANGELOG Signed-off-by: Craig Perkins <[email protected]> * Switch to CompletionStage Signed-off-by: Craig Perkins <[email protected]> * Update CHANGELOG entry Signed-off-by: Craig Perkins <[email protected]> --------- Signed-off-by: Craig Perkins <[email protected]> * Expand fetch phase profiling to support inner hits and top hits aggregation phases (opensearch-project#18936) --------- Signed-off-by: Andre van de Ven <[email protected]> Signed-off-by: Andre van de Ven <[email protected]> Signed-off-by: Andre van de Ven <[email protected]> Co-authored-by: Andre van de Ven <[email protected]> * IllegalArgumentException when scroll ID has a node no longer part of the Cluster (opensearch-project#19031) --------- Signed-off-by: Anurag Rai <[email protected]> Signed-off-by: Anurag Rai <[email protected]> * Add Changelog entry Signed-off-by: Rajat Gupta <[email protected]> * Add secondary constructor Signed-off-by: Rajat Gupta <[email protected]> * Modify changelog Signed-off-by: Rajat Gupta <[email protected]> * Update changelog Signed-off-by: Rajat Gupta <[email protected]> * Add another constructor to fix breaking change check Signed-off-by: Rajat Gupta <[email protected]> --------- Signed-off-by: Rajat Gupta <[email protected]> Signed-off-by: Atri Sharma <[email protected]> Signed-off-by: Andrew Ross <[email protected]> Signed-off-by: Craig Perkins <[email protected]> Signed-off-by: Andre van de Ven <[email protected]> Signed-off-by: Andre van de Ven <[email protected]> Signed-off-by: Andre van de Ven <[email protected]> Signed-off-by: Anurag Rai <[email protected]> Signed-off-by: Anurag Rai <[email protected]> Co-authored-by: Rajat Gupta <[email protected]> Co-authored-by: Atri Sharma <[email protected]> Co-authored-by: Andrew Ross <[email protected]> Co-authored-by: Craig Perkins <[email protected]> Co-authored-by: Andre van de Ven <[email protected]> Co-authored-by: Andre van de Ven <[email protected]> Co-authored-by: Anurag Rai <[email protected]>
…gation phases (opensearch-project#18936) --------- Signed-off-by: Andre van de Ven <[email protected]> Signed-off-by: Andre van de Ven <[email protected]> Signed-off-by: Andre van de Ven <[email protected]> Co-authored-by: Andre van de Ven <[email protected]>
* Add overload for channelFactory Signed-off-by: Rajat Gupta <[email protected]> * Fix tests Signed-off-by: Rajat Gupta <[email protected]> * Add Changelog entry Signed-off-by: Rajat Gupta <[email protected]> * Fix conflicts Signed-off-by: Rajat Gupta <[email protected]> * When update operations fail during preparation (e.g., version conflicts), (opensearch-project#18917) TransportShardBulkAction still triggers refresh even though no actual writes occurred. This fix checks if locationToSync is null (indicating no writes) and prevents refresh in such cases. Fixes opensearch-project#15261 Signed-off-by: Atri Sharma <[email protected]> * Remove all entries from changelog to be released in 3.2 (opensearch-project#18989) Signed-off-by: Andrew Ross <[email protected]> * Add temporal routing processors for time-based document routing (opensearch-project#18966) Implements TemporalRoutingProcessor for ingest pipelines and TemporalRoutingSearchProcessor for search pipelines based on RFC opensearch-project#18920. Features: - Route documents to shards based on timestamp fields - Support hour, day, week, and month granularities - Optional hash bucketing for better distribution - Automatic search routing to relevant time ranges - ISO week format support The processors enable efficient time-based data organization for log and metrics workloads by co-locating documents from the same time period on the same shards. --------- Signed-off-by: Atri Sharma <[email protected]> * Add CompletionStage variants to methods in the Client Interface and default to ActionListener impl (opensearch-project#18998) * Add CompletableFuture variables to methods in the Client Interface and default to ActionListener impl Signed-off-by: Craig Perkins <[email protected]> * Add to CHANGELOG Signed-off-by: Craig Perkins <[email protected]> * Fix typo in CHANGELOG Signed-off-by: Craig Perkins <[email protected]> * Switch to CompletionStage Signed-off-by: Craig Perkins <[email protected]> * Update CHANGELOG entry Signed-off-by: Craig Perkins <[email protected]> --------- Signed-off-by: Craig Perkins <[email protected]> * Expand fetch phase profiling to support inner hits and top hits aggregation phases (opensearch-project#18936) --------- Signed-off-by: Andre van de Ven <[email protected]> Signed-off-by: Andre van de Ven <[email protected]> Signed-off-by: Andre van de Ven <[email protected]> Co-authored-by: Andre van de Ven <[email protected]> * IllegalArgumentException when scroll ID has a node no longer part of the Cluster (opensearch-project#19031) --------- Signed-off-by: Anurag Rai <[email protected]> Signed-off-by: Anurag Rai <[email protected]> * Add Changelog entry Signed-off-by: Rajat Gupta <[email protected]> * Add secondary constructor Signed-off-by: Rajat Gupta <[email protected]> * Modify changelog Signed-off-by: Rajat Gupta <[email protected]> * Update changelog Signed-off-by: Rajat Gupta <[email protected]> * Add another constructor to fix breaking change check Signed-off-by: Rajat Gupta <[email protected]> --------- Signed-off-by: Rajat Gupta <[email protected]> Signed-off-by: Atri Sharma <[email protected]> Signed-off-by: Andrew Ross <[email protected]> Signed-off-by: Craig Perkins <[email protected]> Signed-off-by: Andre van de Ven <[email protected]> Signed-off-by: Andre van de Ven <[email protected]> Signed-off-by: Andre van de Ven <[email protected]> Signed-off-by: Anurag Rai <[email protected]> Signed-off-by: Anurag Rai <[email protected]> Co-authored-by: Rajat Gupta <[email protected]> Co-authored-by: Atri Sharma <[email protected]> Co-authored-by: Andrew Ross <[email protected]> Co-authored-by: Craig Perkins <[email protected]> Co-authored-by: Andre van de Ven <[email protected]> Co-authored-by: Andre van de Ven <[email protected]> Co-authored-by: Anurag Rai <[email protected]>
Description
There are 3 types of fetch phases that are important to account for in profiling:
In the first stage of implementation, the fetch phase profile only supports profiling the standard fetch phase.
This change allows inner hits fetch and top hits aggregation fetch to be profiled and included as separate fetch phases alongside the standard fetch phase in the profile. If multiple separate inner hits searches are run or top hits aggregations are run, they will appear as separate breakdowns. Each individual fetch phase within a search or aggregation will be consolidated under the breakdown of its respective operation rather than creating a new breakdown for each individual fetch phase that is run. The logic for this is that top hits aggregation often runs fetch phase many times, depending on the number of documents aggregated, and lack of consolidation would lead to a verbose and crowded search profile.
Related Issues
Resolves #18862
#18864
Check List
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.