feat(stf/branch): simplify merged iterator #22131
Conversation
|
Warning Rate limit exceeded@testinginprod has exceeded the limit for the number of commits or files that can be reviewed per hour. Please wait 16 minutes and 10 seconds before requesting another review. ⌛ How to resolve this issue?After the wait time has elapsed, a review can be triggered using the We recommend that you space out your commits to avoid hitting the rate limit. 🚦 How do rate limits work?CodeRabbit enforces hourly rate limits for each developer per organization. Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout. Please see our FAQ for further information. 📝 Walkthrough📝 Walkthrough📝 WalkthroughWalkthroughThe changes introduce a generic Changes
Possibly related PRs
Suggested reviewers
Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media? 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
Documentation and Community
|
| // The cache iterator may contain items that shadow or override items in the parent iterator. | ||
| // If the cache iterator has the same key as the parent, the cache's value takes precedence. | ||
| // Deleted items in the cache (indicated by nil values) are skipped. | ||
| type mergedIterator[Parent, Cache corestore.Iterator] struct { |
There was a problem hiding this comment.
Why do you use generics here? mergedIterator is a package private type and not likely to be extended. It should be fine to stick with the corestore.Iterator type.
There was a problem hiding this comment.
in theory, by fact i am not sure, they should provide a performance benefit since it does not do dynamic dispatch on every call
| return true | ||
| // Skip deleted items (value is nil) | ||
| if value == nil { | ||
| continue |
There was a problem hiding this comment.
personal preference: a recursion could be used instead of the for loop. But this may be easier to read
|
tested with #22141 |
|
@testinginprod your pull request is missing a changelog! |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
📜 Review details
Configuration used: .coderabbit.yml
Review profile: CHILL
📒 Files selected for processing (1)
- server/v2/stf/branch/mergeiter.go (1 hunks)
🧰 Additional context used
📓 Path-based instructions (1)
server/v2/stf/branch/mergeiter.go (1)
Pattern
**/*.go: Review the Golang code for conformity with the Uber Golang style guide, highlighting any deviations.
🔇 Additional comments (1)
server/v2/stf/branch/mergeiter.go (1)
68-71: Verify the expected behavior ofKey()andValue()when the iterator is invalidThe
Key()andValue()methods returnnilwhen the iterator is invalid (!i.valid). According to thecorestore.Iteratorinterface, it's important to confirm whether returningnilis appropriate in this scenario or if these methods should panic or handle errors differently. This ensures consistency and prevents unexpected behavior for users of the iterator.Please verify the interface's requirements:
Also applies to: 76-80
| // advance moves the iterator to the next valid (non-deleted) item. | ||
| // It handles merging logic between the parent and cache iterators. | ||
| func (i *mergedIterator[Parent, Cache]) advance() { | ||
| for { | ||
| // If parent is invalid, fast-forward cache. | ||
| if !iter.parent.Valid() { | ||
| iter.skipCacheDeletes(nil) | ||
| return iter.cache.Valid() | ||
| // Check if both iterators have reached the end | ||
| if !i.parent.Valid() && !i.cache.Valid() { | ||
| i.valid = false | ||
| return | ||
| } | ||
| // Parent is valid. | ||
|
|
||
| if !iter.cache.Valid() { | ||
| return true | ||
| var key, value []byte | ||
|
|
||
| // If parent iterator is exhausted, use the cache iterator | ||
| if !i.parent.Valid() { | ||
| key = i.cache.Key() | ||
| value = i.cache.Value() | ||
| i.cache.Next() | ||
| } else if !i.cache.Valid() { | ||
| // If cache iterator is exhausted, use the parent iterator | ||
| key = i.parent.Key() | ||
| value = i.parent.Value() | ||
| i.parent.Next() | ||
| } else { | ||
| // Both iterators are valid; compare keys | ||
| keyP, keyC := i.parent.Key(), i.cache.Key() | ||
| switch cmp := i.compare(keyP, keyC); { | ||
| case cmp < 0: | ||
| // Parent key is less than cache key | ||
| key = keyP | ||
| value = i.parent.Value() | ||
| i.parent.Next() | ||
| case cmp == 0: | ||
| // Keys are equal; cache overrides parent | ||
| key = keyC | ||
| value = i.cache.Value() | ||
| i.parent.Next() | ||
| i.cache.Next() | ||
| case cmp > 0: | ||
| // Cache key is less than parent key | ||
| key = keyC | ||
| value = i.cache.Value() | ||
| i.cache.Next() | ||
| } | ||
| } | ||
| // Parent is valid, cache is valid. | ||
|
|
||
| // Compare parent and cache. | ||
| keyP := iter.parent.Key() | ||
| keyC := iter.cache.Key() | ||
|
|
||
| switch iter.compare(keyP, keyC) { | ||
| case -1: // parent < cache. | ||
| return true | ||
| // Skip deleted items (value is nil) | ||
| if value == nil { | ||
| continue | ||
| } | ||
|
|
||
| case 0: // parent == cache. | ||
| // Skip over if cache item is a delete. | ||
| valueC := iter.cache.Value() | ||
| if valueC == nil { | ||
| iter.parent.Next() | ||
| iter.cache.Next() | ||
| // Update the current key and value, and mark iterator as valid | ||
| i.currKey = key | ||
| i.currValue = value | ||
| i.valid = true | ||
| return | ||
| } | ||
| } |
There was a problem hiding this comment.
Reset i.err in the advance method to avoid stale errors
In the advance() method, when advancing the iterator, i.err is not reset. This could result in the Error() method returning an outdated error even after the iterator has recovered from a previous invalid state. To ensure that i.err accurately reflects the current state of the iterator, consider resetting it at the beginning of the advance() method.
Apply this diff to reset i.err appropriately:
func (i *mergedIterator[Parent, Cache]) advance() {
+ i.err = nil
for {
if !i.parent.Valid() && !i.cache.Valid() {
i.valid = false
return
}
// Rest of the code...
}
}📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| // advance moves the iterator to the next valid (non-deleted) item. | |
| // It handles merging logic between the parent and cache iterators. | |
| func (i *mergedIterator[Parent, Cache]) advance() { | |
| for { | |
| // If parent is invalid, fast-forward cache. | |
| if !iter.parent.Valid() { | |
| iter.skipCacheDeletes(nil) | |
| return iter.cache.Valid() | |
| // Check if both iterators have reached the end | |
| if !i.parent.Valid() && !i.cache.Valid() { | |
| i.valid = false | |
| return | |
| } | |
| // Parent is valid. | |
| if !iter.cache.Valid() { | |
| return true | |
| var key, value []byte | |
| // If parent iterator is exhausted, use the cache iterator | |
| if !i.parent.Valid() { | |
| key = i.cache.Key() | |
| value = i.cache.Value() | |
| i.cache.Next() | |
| } else if !i.cache.Valid() { | |
| // If cache iterator is exhausted, use the parent iterator | |
| key = i.parent.Key() | |
| value = i.parent.Value() | |
| i.parent.Next() | |
| } else { | |
| // Both iterators are valid; compare keys | |
| keyP, keyC := i.parent.Key(), i.cache.Key() | |
| switch cmp := i.compare(keyP, keyC); { | |
| case cmp < 0: | |
| // Parent key is less than cache key | |
| key = keyP | |
| value = i.parent.Value() | |
| i.parent.Next() | |
| case cmp == 0: | |
| // Keys are equal; cache overrides parent | |
| key = keyC | |
| value = i.cache.Value() | |
| i.parent.Next() | |
| i.cache.Next() | |
| case cmp > 0: | |
| // Cache key is less than parent key | |
| key = keyC | |
| value = i.cache.Value() | |
| i.cache.Next() | |
| } | |
| } | |
| // Parent is valid, cache is valid. | |
| // Compare parent and cache. | |
| keyP := iter.parent.Key() | |
| keyC := iter.cache.Key() | |
| switch iter.compare(keyP, keyC) { | |
| case -1: // parent < cache. | |
| return true | |
| // Skip deleted items (value is nil) | |
| if value == nil { | |
| continue | |
| } | |
| case 0: // parent == cache. | |
| // Skip over if cache item is a delete. | |
| valueC := iter.cache.Value() | |
| if valueC == nil { | |
| iter.parent.Next() | |
| iter.cache.Next() | |
| // Update the current key and value, and mark iterator as valid | |
| i.currKey = key | |
| i.currValue = value | |
| i.valid = true | |
| return | |
| } | |
| } | |
| // advance moves the iterator to the next valid (non-deleted) item. | |
| // It handles merging logic between the parent and cache iterators. | |
| func (i *mergedIterator[Parent, Cache]) advance() { | |
| i.err = nil | |
| for { | |
| // Check if both iterators have reached the end | |
| if !i.parent.Valid() && !i.cache.Valid() { | |
| i.valid = false | |
| return | |
| } | |
| var key, value []byte | |
| // If parent iterator is exhausted, use the cache iterator | |
| if !i.parent.Valid() { | |
| key = i.cache.Key() | |
| value = i.cache.Value() | |
| i.cache.Next() | |
| } else if !i.cache.Valid() { | |
| // If cache iterator is exhausted, use the parent iterator | |
| key = i.parent.Key() | |
| value = i.parent.Value() | |
| i.parent.Next() | |
| } else { | |
| // Both iterators are valid; compare keys | |
| keyP, keyC := i.parent.Key(), i.cache.Key() | |
| switch cmp := i.compare(keyP, keyC); { | |
| case cmp < 0: | |
| // Parent key is less than cache key | |
| key = keyP | |
| value = i.parent.Value() | |
| i.parent.Next() | |
| case cmp == 0: | |
| // Keys are equal; cache overrides parent | |
| key = keyC | |
| value = i.cache.Value() | |
| i.parent.Next() | |
| i.cache.Next() | |
| case cmp > 0: | |
| // Cache key is less than parent key | |
| key = keyC | |
| value = i.cache.Value() | |
| i.cache.Next() | |
| } | |
| } | |
| // Skip deleted items (value is nil) | |
| if value == nil { | |
| continue | |
| } | |
| // Update the current key and value, and mark iterator as valid | |
| i.currKey = key | |
| i.currValue = value | |
| i.valid = true | |
| return | |
| } | |
| } |
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
| // If the iterator is invalid, it returns nil. | ||
| func (i *mergedIterator[Parent, Cache]) Key() []byte { | ||
| if !i.valid { | ||
| panic("called key on invalid iterator") |
Check warning
Code scanning / CodeQL
Panic in BeginBock or EndBlock consensus methods Warning
| // If the iterator is invalid, it returns nil. | ||
| func (i *mergedIterator[Parent, Cache]) Value() []byte { | ||
| if !i.valid { | ||
| panic("called value on invalid iterator") |
Check warning
Code scanning / CodeQL
Panic in BeginBock or EndBlock consensus methods Warning
Description
Closes: #XXXX
Author Checklist
All items are required. Please add a note to the item if the item is not applicable and
please add links to any relevant follow up issues.
I have...
!in the type prefix if API or client breaking changeCHANGELOG.mdReviewers Checklist
All items are required. Please add a note if the item is not applicable and please add
your handle next to the items reviewed if you only reviewed selected items.
Please see Pull Request Reviewer section in the contributing guide for more information on how to review a pull request.
I have...
Summary by CodeRabbit
New Features
Improvements