Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve report to show duration/i (e.g: 12.34 ns/i). #132

Merged

Conversation

huacnlee
Copy link
Contributor

@huacnlee huacnlee commented Dec 20, 2023

After update, the new report will look like:

ruby 3.2.0 (2022-12-25 revision a528908271) +YJIT [arm64-darwin23]
Warming up --------------------------------------
          string.new     1.485M i/100ms
       string.concat     1.086M i/100ms
        string.split   649.289k i/100ms
Calculating -------------------------------------
          string.new     15.947M (± 1.2%) i/s   (62.71 ns/i) -     32.665M in   2.048640s
       string.concat     10.846M (± 0.7%) i/s   (92.20 ns/i) -     21.718M in   2.002441s
        string.split      6.506M (± 0.3%) i/s  (153.70 ns/i) -     13.635M in   2.095740s

In some time we need to know how fast (in avg time), the before report only show the iterations/s, but we don't know how long of each iteration.

So I add the duration of each iteration in the report.

This may be useful to compare the performance with other languages, (e.g. Go, Rust etc.) they have the duration in per iteration.

For example Go:

Benchmark_string_alloc-8            	1000000000	         0.3361 ns/op

Use this code to test result:

require "bundler/inline"

gemfile(true) do
  source "https://rubygems.org"
  gem "benchmark-ips", github: "huacnlee/benchmark-ips", branch: "feat/report-duration-per-iter"
end

require "benchmark/ips"

text = "hello world"
part = "world"

Benchmark.ips do |x|
  x.config(time: 2, warmup: 1)

  x.report("string.new") do
    a = "hello world"
  end
  x.report("string.concat") do
    text + part
  end
  x.report("string.split") do
    text.split(" ")
  end
end

@huacnlee huacnlee force-pushed the feat/report-duration-per-iter branch from 81e8c7d to 58450fb Compare December 20, 2023 06:56
@huacnlee huacnlee changed the title Improve report to show duration/i (e.g: 12.24 ns/i). Improve report to show duration/i (e.g: 12.34 ns/i). Dec 20, 2023
@nateberkopec
Copy link
Collaborator

nateberkopec commented Jan 1, 2024

I like the feature, but I'm not sure about the presentation of it.

I think I would prefer 22.780M (± 4.9%) i/s (43.90 ns/i) - 46.044M in 2.027883s)

@huacnlee
Copy link
Contributor Author

huacnlee commented Jan 2, 2024

I have updated, and I left 15 width for (duration/i) for tidy print.

The new result:

Warming up --------------------------------------
          string.new     1.485M i/100ms
       string.concat     1.086M i/100ms
        string.split   649.289k i/100ms
Calculating -------------------------------------
          string.new     15.947M (± 1.2%) i/s   (62.71 ns/i) -     32.665M in   2.048640s
       string.concat     10.846M (± 0.7%) i/s   (92.20 ns/i) -     21.718M in   2.002441s
        string.split      6.506M (± 0.3%) i/s  (153.70 ns/i) -     13.635M in   2.095740s

```
Warming up --------------------------------------
          string.new     1.485M i/100ms
       string.concat     1.086M i/100ms
        string.split   649.289k i/100ms
Calculating -------------------------------------
          string.new     15.947M (± 1.2%) i/s   (62.71 ns/i) -     32.665M in   2.048640s
       string.concat     10.846M (± 0.7%) i/s   (92.20 ns/i) -     21.718M in   2.002441s
        string.split      6.506M (± 0.3%) i/s  (153.70 ns/i) -     13.635M in   2.095740s
```
@huacnlee huacnlee force-pushed the feat/report-duration-per-iter branch from 69dcf93 to 4e406c4 Compare January 2, 2024 02:56
addition: 4955278.9 i/s - 4.85x slower
addition2: 36558904.5 i/s
addition3: 36359284.0 i/s - same-ish: difference falls within error
addition-test-long-label: 36135428.8 i/s - same-ish: difference falls within error
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for updating this!

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fasza

@nateberkopec nateberkopec merged commit 00c1428 into evanphx:master Jan 2, 2024
11 checks passed
@huacnlee huacnlee deleted the feat/report-duration-per-iter branch July 10, 2024 17:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants