feat(shwap): Add ODS file #3482
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
walldiss
requested review from
renaynay,
Wondertan,
vgonkivs,
distractedm1nd and
ramin
as code owners
|
Lint will be get fixed after linter upgrade #3483 |
walldiss
force-pushed
the
ods-accessor
branch
2 times, most recently
from
8164195 to
dda1b17
Compare
walldiss
force-pushed
the
shwap
branch
3 times, most recently
from
c4550bf to
60e757e
Compare
cristaloleg
reviewed
Jun 10, 2024
cristaloleg
reviewed
Jun 10, 2024
cristaloleg
reviewed
Jun 10, 2024
cristaloleg
reviewed
Jun 10, 2024
cristaloleg
reviewed
Jun 10, 2024
walldiss
commented
Jun 11, 2024
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## shwap #3482 +/- ##
========================================
Coverage ? 44.10%
========================================
Files ? 291
Lines ? 16891
Branches ? 0
========================================
Hits ? 7450
Misses ? 8572
Partials ? 869 ☔ View full report in Codecov by Sentry. |
Wondertan
reviewed
Jun 12, 2024
Comment on lines
+45
to
+48
| shares := make([]share.Share, len(original)*2) | ||
| copy(shares, original) | ||
| copy(shares[len(original):], parity) | ||
| return shares, nil |
There was a problem hiding this comment.
Feels like there is an optimization opportunity to avoid copying that we can return to later
There was a problem hiding this comment.
Need to prealloc shares anyway, so copying is necessary. Can do implicitly by append tho
Comment on lines
+168
to
+172
| namespaceBytes := share.GetNamespace(sh) | ||
| leave := make([]byte, len(sh)+len(namespaceBytes)) | ||
| copy(leave, namespaceBytes) | ||
| copy(leave[len(namespaceBytes):], sh) | ||
| leaves = append(leaves, leave) |
There was a problem hiding this comment.
Is this a refactoring from appends to copies?
There was a problem hiding this comment.
I like that you noticed. There was a very nasty mutations bug that took me few hours to fix.
Wondertan
reviewed
Jun 17, 2024
Wondertan
reviewed
Jun 17, 2024
Wondertan
reviewed
Jun 17, 2024
Wondertan
reviewed
Jun 18, 2024
store/file/ods.go
Outdated
Comment on lines
200
to
212
| col, err := f.readCol(axisIdx, 0) | ||
| return eds.AxisHalf{ | ||
| Shares: col, | ||
| IsParity: false, | ||
| }, err | ||
| case rsmt2d.Row: | ||
| return f.readRow(axisIdx) | ||
| row, err := f.readRow(axisIdx) | ||
| return eds.AxisHalf{ | ||
| Shares: row, | ||
| IsParity: false, | ||
| }, err | ||
| } | ||
| return nil, fmt.Errorf("unknown axis") | ||
| return eds.AxisHalf{}, fmt.Errorf("unknown axis") |
Wondertan
approved these changes
Jun 18, 2024
cristaloleg
reviewed
Jun 19, 2024
cristaloleg
approved these changes
Jun 19, 2024
walldiss
added a commit
to walldiss/celestia-node
that referenced
this pull request
Jul 6, 2024
This PR adds ods file, that implements eds.Accessor interface. The file stores ods part of eds on disk and lazily reads data upon request. If requested data is from Q4, it reads full ods in single read and stores it in-memory for later re-use.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR adds ods file, that implements eds.Accessor interface. The file stores ods part of eds on disk and lazily reads data upon request. If requested data is from Q4, it reads full ods in single read and stores it in-memory for later re-use.
Based on #3425