Implement EC logic in PUT #3457
Conversation
cthulhu-rider
force-pushed
the
ec-place
branch
2 times, most recently
from
54df907 to
bb9781b
Compare
cthulhu-rider
force-pushed
the
ec-place
branch
8 times, most recently
from
4e3eb97 to
dba5703
Compare
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## master #3457 +/- ##
==========================================
+ Coverage 23.93% 24.20% +0.27%
==========================================
Files 669 677 +8
Lines 50251 50538 +287
==========================================
+ Hits 12028 12235 +207
- Misses 37257 37312 +55
- Partials 966 991 +25 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
cthulhu-rider
force-pushed
the
ec-place
branch
3 times, most recently
from
ea251ef to
dfebffd
Compare
cthulhu-rider
requested review from
End-rey,
carpawell and
roman-khimov
as code owners
cthulhu-rider
force-pushed
the
ec-place
branch
6 times, most recently
from
f88f10e to
cd56a48
Compare
cthulhu-rider
force-pushed
the
ec-place
branch
2 times, most recently
from
bc80c19 to
4fdde77
Compare
| return nil, fmt.Errorf("restore Reed-Solomon: %w", err) | ||
| } | ||
|
|
||
| // TODO: last part may be shorter, do not overallocate buffer. |
There was a problem hiding this comment.
I doubt it's a problem practically, just a tiny rounding error.
There was a problem hiding this comment.
thinkin about parts of several MB len with the tiny last one (1B in particular)
There was a problem hiding this comment.
But aren't EC part all ~same size? The difference I expect here is on the order of NUM_OF_PARTS bytes.
There was a problem hiding this comment.
they all exactly the same size. So the last part may require alignment. And it may be quite large
| // header. Returns [ErrAttributeNotFound] if the attribute is missing. | ||
| // | ||
| // GetIntAttribute ignores all attribute values except the first. | ||
| func GetIntAttribute(hdr object.Object, attr string) (int, error) { |
There was a problem hiding this comment.
This wants to be in SDK, isn't it?
There was a problem hiding this comment.
yes, eventually. We can consider all general-purpose internal/ parts as candidates for moving to SDK. But doing this right away is inconvenient due to modules
cthulhu-rider
force-pushed
the
ec-place
branch
6 times, most recently
from
1a5f7ce to
2917378
Compare
| } | ||
| } else { | ||
| err = t.placementIterator.iterateNodesForObject(id, t.sendObject) | ||
| broadcast := tombOrLink || (!t.localOnly && typ == objectSDK.TypeLock) || len(t.obj.Children()) > 0 |
There was a problem hiding this comment.
Makes me wonder about localOnly && tombOrLink, but at least that's the way it was done previously.
Use one instruction to calculate the flag. Also, do it per-object and do not store in the struct field. In addition, evaluate only for non-local requests. Signed-off-by: Leonard Lyubich <[email protected]>
Will be useful for EC policies in #3420. Signed-off-by: Leonard Lyubich <[email protected]>
So it can be reused. Signed-off-by: Leonard Lyubich <[email protected]>
This extends PUT server logic to comply with EC policies - independently or together with REP ones. This logic will remain inactive until it becomes possible to create containers with EC policies. This works for small objects only: encoding and distribution will be the same for big objects, but SNs will deny them without proper adaptation. Closes #3419. Refs #3420. Signed-off-by: Leonard Lyubich <[email protected]>
cthulhu-rider
force-pushed
the
ec-place
branch
from
2917378 to
cd3e5d3
Compare
No description provided.