-
Notifications
You must be signed in to change notification settings - Fork 17
fix: rearrange pieces in sector for no padding #306
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'd like to somehow avoid the added state in the DB, any added thing there will make interactions more complex as it's another thing that needs to be kept in sync with everything.
@@ -0,0 +1,171 @@ | |||
CREATE TABLE deal_index_tracker ( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why do we need this instead of looking e.g. at the _initial_pieces tables and the piece_index there?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I wasn't a big fan of this approach while I was writing it. I was trying to find some way to avoid state but could not think of a reliable way which did not involve some really long function. Here is the problem:
- Typically we can easily identify the piece with piece CID and Size.
- Problem starts when we have the same piece in more than 1 location in the same sector. How do I reliably connect a piece to a deal. We don't even need to index it later but we still need to know about the offset. This was a reason I went with a stateful solution.
Another approach I can think of would be to add an index_column to market_mk12_deal_pipeline table and populate it during the transfer functions from within the SQL. It seems cheaper than the current solution.
If you have any insight which avoids any of this, I can really use it.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
From Slack:
We are not going to allow same piece more than once in a sector. This removes the stateful tracking to find unique pieces and their offset in sector.
a4c1236
to
443f6a1
Compare
443f6a1
to
f7b0502
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Don't see anything wrong here, we'll need to make sure to test all this quite well though
No description provided.