Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Experiment with writing a compressed ledger and rsync uncompressed #707

Closed
sakridge opened this issue Jul 19, 2018 · 5 comments
Closed

Experiment with writing a compressed ledger and rsync uncompressed #707

sakridge opened this issue Jul 19, 2018 · 5 comments

Comments

@sakridge
Copy link
Member

rsync of the ledger with compression requires the leader to compress on-the-fly for each client. In theory writing the ledger compressed to disk may reduce disk IO and also improve rsync speed since it doesn't need to compress on-the-fly.
@aeyakovenko
Copy link
Member

If the ledger is binary, I am not sure how much compression will offer. Let’s fix the format

@sakridge
Copy link
Member Author

@aeyakovenko Yea, do we have a binary ledger ticket already? This could be more of experiment with ledger format or I can close and add compression as an idea to try to that one.

@aeyakovenko
Copy link
Member

i dont think we have one yet. i've been playing around with a 2 file format, one fixed entry size for PoH, the other one for data

@rob-solana is looking at it as well

@mvines mvines added this to the The Future! milestone Sep 5, 2018
@garious
Copy link
Contributor

garious commented Jan 16, 2019

@sakridge, is this still an issue?

@mvines
Copy link
Member

mvines commented May 4, 2019

Obsoleted by #4158

@mvines mvines closed this as completed May 4, 2019
@rob-solana rob-solana removed this from the The Future! milestone May 21, 2019
jeffwashington pushed a commit to jeffwashington/solana that referenced this issue Apr 16, 2024
@lijunwangs
Parameterize max streams per ms (solana-labs#707)
f2aa4f0 
Make PPS a parameter instead of the hard coded
buffalojoec pushed a commit to buffalojoec/solana that referenced this issue Apr 16, 2024
@lijunwangs @buffalojoec
Parameterize max streams per ms (solana-labs#707)
3f089d4 
Make PPS a parameter instead of the hard coded
lijunwangs added a commit to lijunwangs/solana that referenced this issue Apr 18, 2024
@lijunwangs
Parameterize max streams per ms (solana-labs#707)
7b66356 
Make PPS a parameter instead of the hard coded
lijunwangs pushed a commit to lijunwangs/solana that referenced this issue Apr 24, 2024
@mergify
v1.18: Parameterize max streams per ms (backport of solana-labs#707) (s…
ee7d35b 
…olana-labs#828)

Make PPS a parameter instead of the hard coded
yihau pushed a commit that referenced this issue Apr 26, 2024
@mergify @yihau
v1.18: Parameterize max streams per ms (backport of #707) (#828)
14f5fa7 
Make PPS a parameter instead of the hard coded
lijunwangs added a commit to lijunwangs/solana that referenced this issue Apr 30, 2024
@mergify @lijunwangs
v1.17: Parameterize max streams per ms (backport of solana-labs#707) (s…
a7cfe9b 
…olana-labs#1105)

* Parameterize max streams per ms (solana-labs#707)

Make PPS a parameter instead of the hard coded


Co-authored-by: Lijun Wang <[email protected]>
yihau pushed a commit that referenced this issue May 7, 2024
@mergify @lijunwangs @yihau
v1.17: Parameterize max streams per ms (backport of #707) (#1105)
f51d29f 
* Parameterize max streams per ms (#707)

Make PPS a parameter instead of the hard coded


Co-authored-by: Lijun Wang <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@garious @sakridge @aeyakovenko @mvines @rob-solana