Skip to content

tools: Block Generator - allow exporting to files#5714

Merged
winder merged 11 commits intoalgorand:masterfrom
tzaffi:tools-blockgen-file-export
Sep 21, 2023
Merged

tools: Block Generator - allow exporting to files#5714
winder merged 11 commits intoalgorand:masterfrom
tzaffi:tools-blockgen-file-export

Conversation

@tzaffi
Copy link
Copy Markdown
Contributor

@tzaffi tzaffi commented Aug 30, 2023

Block Generator in File Exporting Mode + README + Makefile + Remove Python Runner

Functional changes

  • tools/block-generator/generator/server.go - adding some calling context in maybeWriteError()
  • allow exporting blocks into conduit's data directory. This is useful for generating Conduit fixtures. In particular:
    • tools/block-generator/runner/run.go - handle file exporting with a new conduit.yml template
    • tools/block-generator/runner/runner.go - don't enforce the need for a connection string in the case of a file exporter

Non functional changes:

  • New Makefile targets:
    • make pg-up - standalone docker run
    • make pg-query - query postgres (handy since the DB credentials are variables of the Makefile)
    • make run-file-exporter - runner command tailored for file system exporting
    • make benchmark-blocks-export - encapsulates the previous to create blocks with a top-level report directory named according to the scenario
  • Update the README with an explanation of the statistical distribution of block generator scenarios
  • Slight rewording of the runner and daemon's help outputs
  • Remove the python runner - all its previous functionality can be achieved with make targets

@tzaffi tzaffi self-assigned this Aug 30, 2023
@tzaffi tzaffi changed the title Tools blockgen file export tools: Block Generator - allow exporting to files Aug 30, 2023
@codecov
Copy link
Copy Markdown

codecov Bot commented Aug 30, 2023

Codecov Report

Merging #5714 (316ff04) into master (02b99bb) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master    #5714   +/-   ##
=======================================
  Coverage   55.54%   55.54%           
=======================================
  Files         476      476           
  Lines       66853    66853           
=======================================
  Hits        37131    37131           
- Misses      27193    27201    +8     
+ Partials     2529     2521    -8     

see 12 files with indirect coverage changes

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

@tzaffi tzaffi marked this pull request as ready for review August 30, 2023 22:00
@tzaffi tzaffi requested review from a team, Eric-Warehime, algochoi, shiqizng and winder and removed request for a team August 30, 2023 22:01
Comment thread tools/block-generator/README.md Outdated
Comment thread tools/block-generator/README.md Outdated
Comment thread tools/block-generator/README.md Outdated
Comment thread tools/block-generator/README.md Outdated
Comment thread tools/block-generator/README.md
Comment thread tools/block-generator/README.md Outdated
Copy link
Copy Markdown
Contributor

@winder winder left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks pretty good overall.

Comment thread tools/block-generator/runner/run.go
Comment thread tools/block-generator/runner/run.go Outdated
Comment thread tools/block-generator/runner/run.go
Comment thread tools/block-generator/runner/runner.go
Comment thread tools/block-generator/Makefile Outdated
Comment thread tools/block-generator/Makefile Outdated
Comment thread tools/block-generator/README.md Outdated
Comment thread tools/block-generator/README.md
Comment thread tools/block-generator/README.md
Comment thread tools/block-generator/runner/runner.go
shiqizng
shiqizng previously approved these changes Sep 5, 2023
@winder winder merged commit 2f09710 into algorand:master Sep 21, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants