Skip to content

chore: add traces and histograms to avm simulator#10233

Merged
PhilWindle merged 5 commits intomasterfrom
10146-choreavm-sim-emit-traces
Nov 27, 2024
Merged

chore: add traces and histograms to avm simulator#10233
PhilWindle merged 5 commits intomasterfrom
10146-choreavm-sim-emit-traces

Conversation

@just-mitch
Copy link
Collaborator

@just-mitch just-mitch commented Nov 27, 2024 •
edited
Loading

Add a histogram for tracking the mana/second of avm enqueued call simulations.

Assert that we are above 10 Mmana/second in the public transfer test.

This is just a dummy value so we don't get hit with flake on slow CI machines: it tests that the metric is getting reported at all.

Also some traces at the enqueued call level.

See graph attached of my local execution. Interestingly, transfer_in_public sits constantly around 5.5 Mmana/s, but total_supply was the worst at around 3 Mmana/s.

Screenshot 2024-11-26 at 21 17 50

fix #10145
fix #10146

@just-mitch just-mitch linked an issue Nov 27, 2024 that may be closed by this pull request
chore(avm-sim): emit traces #10146
Closed
PhilWindle
PhilWindle approved these changes Nov 27, 2024
View reviewed changes
Copy link
Collaborator

@PhilWindle PhilWindle left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@PhilWindle PhilWindle enabled auto-merge (squash) November 27, 2024 13:50
@just-mitch
Copy link
Collaborator Author

just-mitch commented Nov 27, 2024

CI failed to hit the 2.5Mmana/s I had set.
So I'm dropping the performance requirement on CI to be a dummy value of 10, just to ensure that the metrics are getting emitted and are measurable.

We'll need to run a separate bench on the live network where the machine has our official system requirements.

To that end, I filed #10246

@just-mitch
change nomenclature to mana/second
37f4ce3 
drop target to just a dummy 10 value
@github-actions
Copy link
Contributor

github-actions bot commented Nov 27, 2024

Changes to public function bytecode sizes

Generated at commit: 4c69a5734dec7a97526b4008049bbfdb964c7df2, compared to commit: 52047f05495ef95a778e8669fc4e115cacb590a0

🧾 Summary (100% most significant diffs)

Program Bytecode size in bytes (+/-) %
AvmTest::public_dispatch +9 ❌ +0.01%
AvmTest::bulk_testing -30 ✅ -0.13%
Full diff report 👇
Program Bytecode size in bytes (+/-) %
AvmTest::public_dispatch 60,771 (+9) +0.01%
AvmTest::bulk_testing 23,074 (-30) -0.13%

@PhilWindle PhilWindle merged commit e83726d into master Nov 27, 2024
@PhilWindle PhilWindle deleted the 10146-choreavm-sim-emit-traces branch November 27, 2024 14:37
@AztecBot AztecBot mentioned this pull request Nov 27, 2024
Merged
critesjosh pushed a commit that referenced this pull request Nov 27, 2024
@AztecBot
chore(master): Release 0.65.1 (#10219)
62fc917 
🤖 I have created a release *beep* *boop*
---


<details><summary>aztec-package: 0.65.1</summary>

##
[0.65.1](aztec-package-v0.65.0...aztec-package-v0.65.1)
(2024-11-27)


### Miscellaneous

* Delete old serialization methods
([#9951](#9951))
([10d3f6f](10d3f6f))
</details>

<details><summary>barretenberg.js: 0.65.1</summary>

##
[0.65.1](barretenberg.js-v0.65.0...barretenberg.js-v0.65.1)
(2024-11-27)


### Features

* Speed up transaction execution
([#10172](#10172))
([da265b6](da265b6))


### Bug Fixes

* Add pako as a dependency in bb.js
([#10186](#10186))
([b773c14](b773c14))
</details>

<details><summary>aztec-packages: 0.65.1</summary>

##
[0.65.1](aztec-packages-v0.65.0...aztec-packages-v0.65.1)
(2024-11-27)


### Features

* Add total mana used to header
([#9868](#9868))
([2478d19](2478d19))
* Assert metrics in network tests
([#10215](#10215))
([9380c0f](9380c0f))
* Avm inserts nullifiers from private
([#10129](#10129))
([3fc0c7c](3fc0c7c))
* Burn congestion fee
([#10231](#10231))
([20a33f2](20a33f2))
* Configure world state block history
([#10216](#10216))
([01eb392](01eb392))
* Integrate fee into rollup
([#10176](#10176))
([12744d6](12744d6))
* Speed up transaction execution
([#10172](#10172))
([da265b6](da265b6))
* Using current gas prices in cli-wallet
([#10105](#10105))
([15ffeea](15ffeea))


### Bug Fixes

* Add pako as a dependency in bb.js
([#10186](#10186))
([b773c14](b773c14))
* **avm:** Execution test ordering
([#10226](#10226))
([49b4a6c](49b4a6c))
* Deploy preview master
([#10227](#10227))
([321a175](321a175))
* Use current base fee for public fee payment
([#10230](#10230))
([f081d80](f081d80))


### Miscellaneous

* Add traces and histograms to avm simulator
([#10233](#10233))
([e83726d](e83726d)),
closes
[#10146](#10146)
* Avm-proving and avm-integration tests do not require simulator to
export function with jest mocks
([#10228](#10228))
([f28fcdb](f28fcdb))
* **avm:** Handle parsing error
([#10203](#10203))
([3c623fc](3c623fc)),
closes
[#9770](#9770)
* **avm:** Zero initialization in avm public inputs and execution test
fixes
([#10238](#10238))
([0c7c4c9](0c7c4c9))
* Bump timeout for after-hook for data store test again
([#10240](#10240))
([52047f0](52047f0))
* CIVC VK
([#10223](#10223))
([089c34c](089c34c))
* Declare global types
([#10206](#10206))
([7b2e343](7b2e343))
* Delete old serialization methods
([#9951](#9951))
([10d3f6f](10d3f6f))
* Fix migration notes
([#10252](#10252))
([05bdcd5](05bdcd5))
* Pull out some sync changes
([#10245](#10245))
([1bfc15e](1bfc15e))
* Remove docs from sync
([#10241](#10241))
([eeea0aa](eeea0aa))
* Replace relative paths to noir-protocol-circuits
([e7690ca](e7690ca))
</details>

<details><summary>barretenberg: 0.65.1</summary>

##
[0.65.1](barretenberg-v0.65.0...barretenberg-v0.65.1)
(2024-11-27)


### Features

* Add total mana used to header
([#9868](#9868))
([2478d19](2478d19))
* Configure world state block history
([#10216](#10216))
([01eb392](01eb392))
* Speed up transaction execution
([#10172](#10172))
([da265b6](da265b6))


### Bug Fixes

* **avm:** Execution test ordering
([#10226](#10226))
([49b4a6c](49b4a6c))


### Miscellaneous

* **avm:** Handle parsing error
([#10203](#10203))
([3c623fc](3c623fc)),
closes
[#9770](#9770)
* **avm:** Zero initialization in avm public inputs and execution test
fixes
([#10238](#10238))
([0c7c4c9](0c7c4c9))
* CIVC VK
([#10223](#10223))
([089c34c](089c34c))
* Pull out some sync changes
([#10245](#10245))
([1bfc15e](1bfc15e))
</details>

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).
AztecBot added a commit to AztecProtocol/barretenberg that referenced this pull request Nov 28, 2024
@AztecBot
chore(master): Release 0.65.1 (#10219)
14f7045 
🤖 I have created a release *beep* *boop*
---


<details><summary>aztec-package: 0.65.1</summary>

##
[0.65.1](AztecProtocol/aztec-packages@aztec-package-v0.65.0...aztec-package-v0.65.1)
(2024-11-27)


### Miscellaneous

* Delete old serialization methods
([#9951](AztecProtocol/aztec-packages#9951))
([10d3f6f](AztecProtocol/aztec-packages@10d3f6f))
</details>

<details><summary>barretenberg.js: 0.65.1</summary>

##
[0.65.1](AztecProtocol/aztec-packages@barretenberg.js-v0.65.0...barretenberg.js-v0.65.1)
(2024-11-27)


### Features

* Speed up transaction execution
([#10172](AztecProtocol/aztec-packages#10172))
([da265b6](AztecProtocol/aztec-packages@da265b6))


### Bug Fixes

* Add pako as a dependency in bb.js
([#10186](AztecProtocol/aztec-packages#10186))
([b773c14](AztecProtocol/aztec-packages@b773c14))
</details>

<details><summary>aztec-packages: 0.65.1</summary>

##
[0.65.1](AztecProtocol/aztec-packages@aztec-packages-v0.65.0...aztec-packages-v0.65.1)
(2024-11-27)


### Features

* Add total mana used to header
([#9868](AztecProtocol/aztec-packages#9868))
([2478d19](AztecProtocol/aztec-packages@2478d19))
* Assert metrics in network tests
([#10215](AztecProtocol/aztec-packages#10215))
([9380c0f](AztecProtocol/aztec-packages@9380c0f))
* Avm inserts nullifiers from private
([#10129](AztecProtocol/aztec-packages#10129))
([3fc0c7c](AztecProtocol/aztec-packages@3fc0c7c))
* Burn congestion fee
([#10231](AztecProtocol/aztec-packages#10231))
([20a33f2](AztecProtocol/aztec-packages@20a33f2))
* Configure world state block history
([#10216](AztecProtocol/aztec-packages#10216))
([01eb392](AztecProtocol/aztec-packages@01eb392))
* Integrate fee into rollup
([#10176](AztecProtocol/aztec-packages#10176))
([12744d6](AztecProtocol/aztec-packages@12744d6))
* Speed up transaction execution
([#10172](AztecProtocol/aztec-packages#10172))
([da265b6](AztecProtocol/aztec-packages@da265b6))
* Using current gas prices in cli-wallet
([#10105](AztecProtocol/aztec-packages#10105))
([15ffeea](AztecProtocol/aztec-packages@15ffeea))


### Bug Fixes

* Add pako as a dependency in bb.js
([#10186](AztecProtocol/aztec-packages#10186))
([b773c14](AztecProtocol/aztec-packages@b773c14))
* **avm:** Execution test ordering
([#10226](AztecProtocol/aztec-packages#10226))
([49b4a6c](AztecProtocol/aztec-packages@49b4a6c))
* Deploy preview master
([#10227](AztecProtocol/aztec-packages#10227))
([321a175](AztecProtocol/aztec-packages@321a175))
* Use current base fee for public fee payment
([#10230](AztecProtocol/aztec-packages#10230))
([f081d80](AztecProtocol/aztec-packages@f081d80))


### Miscellaneous

* Add traces and histograms to avm simulator
([#10233](AztecProtocol/aztec-packages#10233))
([e83726d](AztecProtocol/aztec-packages@e83726d)),
closes
[#10146](AztecProtocol/aztec-packages#10146)
* Avm-proving and avm-integration tests do not require simulator to
export function with jest mocks
([#10228](AztecProtocol/aztec-packages#10228))
([f28fcdb](AztecProtocol/aztec-packages@f28fcdb))
* **avm:** Handle parsing error
([#10203](AztecProtocol/aztec-packages#10203))
([3c623fc](AztecProtocol/aztec-packages@3c623fc)),
closes
[#9770](AztecProtocol/aztec-packages#9770)
* **avm:** Zero initialization in avm public inputs and execution test
fixes
([#10238](AztecProtocol/aztec-packages#10238))
([0c7c4c9](AztecProtocol/aztec-packages@0c7c4c9))
* Bump timeout for after-hook for data store test again
([#10240](AztecProtocol/aztec-packages#10240))
([52047f0](AztecProtocol/aztec-packages@52047f0))
* CIVC VK
([#10223](AztecProtocol/aztec-packages#10223))
([089c34c](AztecProtocol/aztec-packages@089c34c))
* Declare global types
([#10206](AztecProtocol/aztec-packages#10206))
([7b2e343](AztecProtocol/aztec-packages@7b2e343))
* Delete old serialization methods
([#9951](AztecProtocol/aztec-packages#9951))
([10d3f6f](AztecProtocol/aztec-packages@10d3f6f))
* Fix migration notes
([#10252](AztecProtocol/aztec-packages#10252))
([05bdcd5](AztecProtocol/aztec-packages@05bdcd5))
* Pull out some sync changes
([#10245](AztecProtocol/aztec-packages#10245))
([1bfc15e](AztecProtocol/aztec-packages@1bfc15e))
* Remove docs from sync
([#10241](AztecProtocol/aztec-packages#10241))
([eeea0aa](AztecProtocol/aztec-packages@eeea0aa))
* Replace relative paths to noir-protocol-circuits
([e7690ca](AztecProtocol/aztec-packages@e7690ca))
</details>

<details><summary>barretenberg: 0.65.1</summary>

##
[0.65.1](AztecProtocol/aztec-packages@barretenberg-v0.65.0...barretenberg-v0.65.1)
(2024-11-27)


### Features

* Add total mana used to header
([#9868](AztecProtocol/aztec-packages#9868))
([2478d19](AztecProtocol/aztec-packages@2478d19))
* Configure world state block history
([#10216](AztecProtocol/aztec-packages#10216))
([01eb392](AztecProtocol/aztec-packages@01eb392))
* Speed up transaction execution
([#10172](AztecProtocol/aztec-packages#10172))
([da265b6](AztecProtocol/aztec-packages@da265b6))


### Bug Fixes

* **avm:** Execution test ordering
([#10226](AztecProtocol/aztec-packages#10226))
([49b4a6c](AztecProtocol/aztec-packages@49b4a6c))


### Miscellaneous

* **avm:** Handle parsing error
([#10203](AztecProtocol/aztec-packages#10203))
([3c623fc](AztecProtocol/aztec-packages@3c623fc)),
closes
[#9770](AztecProtocol/aztec-packages#9770)
* **avm:** Zero initialization in avm public inputs and execution test
fixes
([#10238](AztecProtocol/aztec-packages#10238))
([0c7c4c9](AztecProtocol/aztec-packages@0c7c4c9))
* CIVC VK
([#10223](AztecProtocol/aztec-packages#10223))
([089c34c](AztecProtocol/aztec-packages@089c34c))
* Pull out some sync changes
([#10245](AztecProtocol/aztec-packages#10245))
([1bfc15e](AztecProtocol/aztec-packages@1bfc15e))
</details>

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@PhilWindle PhilWindle PhilWindle approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

chore(avm-sim): emit traces feat(sequencer): mana per second benchmarking suite

2 participants

@just-mitch @PhilWindle