Increase segrep pressure checkpoint default limit to 30 - #16577
Merged
Conversation
gbbafna
force-pushed
the
segrep_bp_default
branch
from
November 6, 2024 05:00
4f8107d to
965595e
Compare
Contributor
|
❌ Gradle check result for 965595e: FAILURE Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change? |
gbbafna
force-pushed
the
segrep_bp_default
branch
from
November 6, 2024 07:19
965595e to
b14c7c8
Compare
gbbafna
marked this pull request as ready for review
November 6, 2024 07:19
gbbafna
requested review from
Bukhtawar,
CEHENKLE,
Rishikesh1159,
VachaShah,
anasalkouz,
andrross,
ashking94,
dblock,
dbwiddis,
jainankitk,
jed326,
kotwanikunal,
linuxpi,
mch2,
msfroh,
nknize,
owaiskazi19,
reta,
sachinpkale,
saratvemulapalli,
shwetathareja and
sohami
as code owners
November 6, 2024 07:19
Contributor
|
❌ Gradle check result for b14c7c8: FAILURE Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change? |
gbbafna
force-pushed
the
segrep_bp_default
branch
from
November 6, 2024 08:44
b14c7c8 to
3fef8c0
Compare
Contributor
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## main #16577 +/- ##
============================================
- Coverage 72.15% 72.07% -0.08%
+ Complexity 65128 65084 -44
============================================
Files 5315 5315
Lines 303573 303573
Branches 43925 43925
============================================
- Hits 219036 218807 -229
- Misses 66565 66773 +208
- Partials 17972 17993 +21 ☔ View full report in Codecov by Sentry. |
kotwanikunal
approved these changes
Nov 8, 2024
kotwanikunal
left a comment
kotwanikunal
left a comment
Member
There was a problem hiding this comment.
@gbbafna - Users who upgrade to this release will have the default value updated without them noticing. Do you think it makes sense to document this as a part of release highlights?
linuxpi
reviewed
Nov 11, 2024
linuxpi
reviewed
Nov 11, 2024
ashking94
approved these changes
Nov 11, 2024
gbbafna
force-pushed
the
segrep_bp_default
branch
from
November 11, 2024 06:42
3fef8c0 to
67bdeaf
Compare
Contributor
gbbafna
changed the title
ashking94
reviewed
Nov 11, 2024
Signed-off-by: Gaurav Bafna <gbbafna@amazon.com>
gbbafna
force-pushed
the
segrep_bp_default
branch
from
November 11, 2024 08:16
67bdeaf to
b2ccfeb
Compare
Contributor
opensearch-trigger-bot Bot
pushed a commit
that referenced
this pull request
Nov 11, 2024
Signed-off-by: Gaurav Bafna <gbbafna@amazon.com> (cherry picked from commit 10873f1) Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
gbbafna
pushed a commit
that referenced
this pull request
Nov 12, 2024
) (cherry picked from commit 10873f1) Signed-off-by: Gaurav Bafna <gbbafna@amazon.com> Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
akolarkunnu
pushed a commit
to akolarkunnu/OpenSearch
that referenced
this pull request
Jan 21, 2025
…roject#16577) Signed-off-by: Gaurav Bafna <gbbafna@amazon.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
The default
segrep.pressure.time.limitis 5 mins , while defaultsegrep.pressure.checkpoint.limitis just 4 which only allows for 40 sec lag given the refresh interval of 10 secs . Though the default refresh interval is 1sec, for remote backed clusters, we advice >10 sec due to prohibitive costs and high resource consumption of segment upload every sec. This PR increasessegrep.pressure.checkpoint.limitto 10 so that both checkpoint limit and time limit equates to same 300 sec and we don't give unnecessary rejections to the user .This relaxation is due to the fact that we don't have multipart download or block level fetches in download part, hence the observed lag on clusters are on a higher side (100s of seconds) during force merges .
Check List
API changes companion pull request created, if applicable.Public documentation issue/PR created, if applicable.By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.