-
Notifications
You must be signed in to change notification settings - Fork 50
Description
Is your feature request related to a problem? Please describe.
to support EC with RS scheme, parity blocks (or shards) should be produced and stored along with the data ones. The system is not capable of doing this now
Describe the solution you'd like
for each K data objects produced from N bytes of original payload according to split scheme, R extra objects are created, making [1, ..., K, K+1, ..., K+R] list so that:
- all objects have
L = N / Kbytes of payload exceptK-th one which may haveN mod Ksize - having access to any of K+R objects, it is possible to resolve full
[1, ..., K+R]list - parity objects remain outside queries which is currently used to assemble objects without redundancy (split v2)
to comply with these requirements
- each object has
IDX: iattribute - each object has
DATA: Kattribute - each object has
PARITY: Rattribute - each object except 1st has
SHARDS_FIRST: ID(1)attribute - each parity object has
SHARDS_LAST: ID(K)attribute - each parity object has no
splitfield
all attributes are __NEOFS__EC_RS_-prefixed
DATA and PARITY preserve list size, IDX - ordering. FIRST groups all shards, controls placement and distribution across container. LAST will help when last object is inaccessible (here)
Describe alternatives you've considered
unset split field may be wrong for GC. Parity objects should be removed along with data split members. To achieve this now, split.parent must be set to some chain member. Not root ID cuz we have 3rd requirement. Maybe the 1st?