feat: Begin replacing old progress bars with Rich bars #169
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This introduces an enhancement over the old progress bars, using rich to provide colorful, live bars. Base types for skinning them are included.
uses the new rich-based progress bars to display download speed information etc.
- make bars hideable, - polish, - default the progress bar to visible, require show=False explicitly. - add more bar styles, - use some of the bars,
- Remove ROWID from Station table to improve performance, - Introduce StationDemand and StationSupply tables for breaking up StationItem, - Reduce size of station/system index,
- advance progress bars properly, - actually display the text of the progress bar, duh
Line CountingBar except it includes a TimeRemaining column (long running tasks seem to warrant this)
- small tweaks to improve performance of eddblink import,
- commit batching to try and improve eddblink import speed,
- use rich progress bars to give better insight into commit rate
one interesting thing to try is to change the progress bar description around the COMMIT while importing prices:
```
prog.increment(description="COMMIT")
cursor.execute("COMMIT")
transaction_items = 0
cursor.execute("BEGIN TRANSACTION")
prog.increment(description="Processing")
```
and then adjust the batch size. when batchsize is too low, this makes the description flicker, but you can get a sense for how increasing the batch size reduces the total import time until the amount of memory/wal etc starts to make the commit time excessively long
ProgressBars.mp4 |
kfsone
force-pushed
the
feat/use-rich-pbars
branch
from
ba8900e to
4ecf0da
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Suggest you try running this a few times to decide if you like how the bars look, where they're used, etc.
Once this is landed, I'm going to finish off changes to eddblink that reduces the time it takes to collect all the "is this file out of date?" by doing them over a single HTTPS connection.