Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bench: add FizzyCEngine #561

Merged
merged 2 commits into from
Oct 16, 2020
Merged

bench: add FizzyCEngine #561

merged 2 commits into from
Oct 16, 2020

Conversation

gumb0
Copy link
Collaborator

@gumb0 gumb0 commented Sep 30, 2020

No description provided.

@gumb0 gumb0 force-pushed the capi_execute branch 2 times, most recently from 6871e47 to c517410 Compare October 1, 2020 13:05
@codecov
Copy link

codecov bot commented Oct 1, 2020

Codecov Report

Merging #561 into master will increase coverage by 0.01%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##           master     #561      +/-   ##
==========================================
+ Coverage   98.24%   98.25%   +0.01%     
==========================================
  Files          63       64       +1     
  Lines        9370     9425      +55     
==========================================
+ Hits         9206     9261      +55     
  Misses        164      164              

@gumb0 gumb0 force-pushed the capi_execute branch 2 times, most recently from dfdbfbf to 6525b9d Compare October 5, 2020 11:17
@gumb0 gumb0 changed the base branch from capi_execute to capi-execute-dynamic-module October 5, 2020 14:47
@gumb0 gumb0 force-pushed the fizzy_c_engine branch 3 times, most recently from ded8136 to 17f123a Compare October 6, 2020 18:10
@gumb0 gumb0 changed the base branch from capi-execute-dynamic-module to capi-instance-memory October 6, 2020 18:11
@gumb0 gumb0 force-pushed the fizzy_c_engine branch 2 times, most recently from 79b8a72 to 6a190e6 Compare October 7, 2020 13:16
@gumb0 gumb0 force-pushed the capi-instance-memory branch from b6eea83 to 0a80dfc Compare October 7, 2020 13:28
@gumb0 gumb0 force-pushed the fizzy_c_engine branch 3 times, most recently from 085143c to f3ab3df Compare October 7, 2020 14:05
@gumb0 gumb0 force-pushed the capi-instance-memory branch from 0a80dfc to 486219a Compare October 7, 2020 16:44
@gumb0 gumb0 force-pushed the capi-instance-memory branch 2 times, most recently from bed7b06 to 7bf49d1 Compare October 12, 2020 10:59
@gumb0 gumb0 force-pushed the capi-instance-memory branch 2 times, most recently from 66c71d0 to 9791ba5 Compare October 14, 2020 11:20
@gumb0 gumb0 force-pushed the capi-instance-memory branch 5 times, most recently from b7af6f9 to 34bcd69 Compare October 15, 2020 13:05
@gumb0 gumb0 changed the base branch from capi-instance-memory to capi-resolve-instantiate October 15, 2020 14:17
Base automatically changed from capi-resolve-instantiate to master October 15, 2020 14:29
if (!fizzy_find_exported_function(
fizzy_get_instance_module(m_instance.get()), std::string{name}.c_str(), &func_idx))
{
// TODO check function type
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we have this solved by the API?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, it's the same as in C++ API.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fizzy_engine.cpp does check the signature.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But i think we can ignore that because we decided to do it only once in fizzy_engine and ignore the other implementaitons.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought to reuse the code from fizzy_engine,cpp, shoud I just remove TODO?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I think it is enough having it in one engine.

@gumb0 gumb0 force-pushed the fizzy_c_engine branch 2 times, most recently from d32ba10 to 0b71661 Compare October 15, 2020 14:48
@gumb0
Copy link
Collaborator Author

gumb0 commented Oct 15, 2020

Comparing fizzy vs fizzyc

[fizzy vs. fizzyc]/parse/blake2b_mean                                            -0.0062         -0.0037            46            45            46            45
[fizzy vs. fizzyc]/instantiate/blake2b_mean                                      +0.0183         +0.0196            52            53            52            53
[fizzy vs. fizzyc]/execute/blake2b/512_bytes_rounds_1_mean                       +0.0004         +0.0004           229           229           229           229
[fizzy vs. fizzyc]/execute/blake2b/512_bytes_rounds_16_mean                      -0.0022         -0.0023          3459          3452          3460          3452
[fizzy vs. fizzyc]/parse/ecpairing_mean                                          -0.0001         -0.0001          2610          2609          2610          2610
[fizzy vs. fizzyc]/instantiate/ecpairing_mean                                    -0.0012         -0.0001          2718          2714          2718          2718
[fizzy vs. fizzyc]/execute/ecpairing/onepoint_mean                               -0.0003         -0.0015       1084992       1084697       1087255       1085641
[fizzy vs. fizzyc]/parse/keccak256_mean                                          +0.0006         +0.0006            83            83            83            83
[fizzy vs. fizzyc]/instantiate/keccak256_mean                                    +0.0003         -0.0022            91            91            90            90
[fizzy vs. fizzyc]/execute/keccak256/512_bytes_rounds_1_mean                     -0.0026         -0.0016           271           270           270           269
[fizzy vs. fizzyc]/execute/keccak256/512_bytes_rounds_16_mean                    -0.0061         -0.0057          3965          3940          3960          3937
[fizzy vs. fizzyc]/parse/memset_mean                                             -0.0026         -0.0025            11            11            11            11
[fizzy vs. fizzyc]/instantiate/memset_mean                                       +0.0121         +0.0122            17            17            17            17
[fizzy vs. fizzyc]/execute/memset/256_bytes_mean                                 -0.0027         -0.0027            19            19            19            19
[fizzy vs. fizzyc]/execute/memset/60000_bytes_mean                               +0.0039         +0.0026          4219          4236          4218          4228
[fizzy vs. fizzyc]/parse/mul256_opt0_mean                                        +0.0005         +0.0005            14            14            14            14
[fizzy vs. fizzyc]/instantiate/mul256_opt0_mean                                  +0.0048         +0.0048            20            20            20            20
[fizzy vs. fizzyc]/execute/mul256_opt0/input1_mean                               +0.0021         +0.0021            77            77            77            77
[fizzy vs. fizzyc]/parse/ramanujan_pi_mean                                       +0.0039         +0.0039            49            50            49            50
[fizzy vs. fizzyc]/instantiate/ramanujan_pi_mean                                 +0.0025         +0.0025            56            56            56            56
[fizzy vs. fizzyc]/execute/ramanujan_pi/33_runs_mean                             +0.0018         +0.0024           316           317           316           317
[fizzy vs. fizzyc]/parse/sha1_mean                                               +0.0021         +0.0019            75            75            75            75
[fizzy vs. fizzyc]/instantiate/sha1_mean                                         +0.0002         +0.0001            81            81            81            81
[fizzy vs. fizzyc]/execute/sha1/512_bytes_rounds_1_mean                          -0.0002         -0.0002           247           247           247           247
[fizzy vs. fizzyc]/execute/sha1/512_bytes_rounds_16_mean                         +0.0016         +0.0016          3443          3449          3443          3449
[fizzy vs. fizzyc]/parse/sha256_mean                                             +0.0007         +0.0007           125           125           125           125
[fizzy vs. fizzyc]/instantiate/sha256_mean                                       +0.0005         +0.0004           132           132           132           132
[fizzy vs. fizzyc]/execute/sha256/512_bytes_rounds_1_mean                        -0.0010         -0.0010           225           225           225           225
[fizzy vs. fizzyc]/execute/sha256/512_bytes_rounds_16_mean                       +0.0001         +0.0001          3085          3085          3085          3085
[fizzy vs. fizzyc]/parse/taylor_pi_mean                                          -0.0016         -0.0016             5             5             5             5
[fizzy vs. fizzyc]/instantiate/taylor_pi_mean                                    +0.0204         +0.0204            11            11            11            11
[fizzy vs. fizzyc]/execute/taylor_pi/pi_1000000_runs_mean                        -0.0003         -0.0003        105867        105834        105855        105823
[fizzy vs. fizzyc]/parse/micro/eli_interpreter_mean                              +0.0009         +0.0009             8             8             8             8
[fizzy vs. fizzyc]/instantiate/micro/eli_interpreter_mean                        +0.0102         +0.0102            14            14            14            14
[fizzy vs. fizzyc]/execute/micro/eli_interpreter/exec105_mean                    +0.0013         +0.0013            12            12            12            12
[fizzy vs. fizzyc]/parse/micro/factorial_mean                                    -0.0020         -0.0020             2             2             2             2
[fizzy vs. fizzyc]/instantiate/micro/factorial_mean                              +0.0757         +0.0757             2             2             2             2
[fizzy vs. fizzyc]/execute/micro/factorial/20_mean                               +0.0015         +0.0015             1             1             1             1
[fizzy vs. fizzyc]/parse/micro/fibonacci_mean                                    +0.0009         +0.0009             2             2             2             2
[fizzy vs. fizzyc]/instantiate/micro/fibonacci_mean                              +0.0444         +0.0444             3             3             3             3
[fizzy vs. fizzyc]/execute/micro/fibonacci/24_mean                               -0.0005         -0.0005         12509         12502         12507         12501
[fizzy vs. fizzyc]/parse/micro/host_adler32_mean                                 +0.0001         +0.0001             3             3             3             3
[fizzy vs. fizzyc]/instantiate/micro/host_adler32_mean                           +0.0329         +0.0329             7             7             7             7
[fizzy vs. fizzyc]/execute/micro/host_adler32/1_mean                             +0.3695         +0.3696             0             0             0             0
[fizzy vs. fizzyc]/execute/micro/host_adler32/1000_mean                          +0.0629         +0.0629            77            82            77            82
[fizzy vs. fizzyc]/parse/micro/spinner_mean                                      -0.0010         -0.0010             2             2             2             2
[fizzy vs. fizzyc]/instantiate/micro/spinner_mean                                +0.0649         +0.0649             2             2             2             2
[fizzy vs. fizzyc]/execute/micro/spinner/1_mean                                  +0.0075         +0.0075             0             0             0             0
[fizzy vs. fizzyc]/execute/micro/spinner/1000_mean                               +0.0003         +0.0003            26            26            26            26
[fizzy vs. fizzyc]/parse/stress/guido-fuzzer-find-1_mean                         +0.0001         +0.0001           272           272           272           272
[fizzy vs. fizzyc]/instantiate/stress/guido-fuzzer-find-1_mean                   +0.0004         +0.0004           320           320           320           320

@gumb0 gumb0 marked this pull request as ready for review October 15, 2020 15:49
@axic
Copy link
Member

axic commented Oct 15, 2020

[fizzy vs. fizzyc]/execute/micro/host_adler32/1000_mean                          +0.0629         +0.0629            77            82            77            82

This is the only benchmark which should be affected and 6% does not seem to be terrible, but wonder if it can be improved or if this is just noise. Anyway we should handle it outside of this PR.

@gumb0 gumb0 marked this pull request as draft October 15, 2020 16:22
@gumb0 gumb0 force-pushed the fizzy_c_engine branch 2 times, most recently from 50db6d6 to 102f3e3 Compare October 15, 2020 17:32
@gumb0 gumb0 marked this pull request as ready for review October 15, 2020 17:39
@gumb0 gumb0 requested a review from chfast October 16, 2020 08:43
@@ -0,0 +1,121 @@
// Fizzy: A fast WebAssembly interpreter
// Copyright 2019-2020 The Fizzy Authors.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
// Copyright 2019-2020 The Fizzy Authors.
// Copyright 2020 The Fizzy Authors.

@gumb0 gumb0 merged commit c38cb38 into master Oct 16, 2020
@gumb0 gumb0 deleted the fizzy_c_engine branch October 16, 2020 10:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants