Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tuning ignores explicit evals parameter #297

Closed
Ethan-Russell opened this issue Dec 22, 2022 · 1 comment · Fixed by #318
Closed

Tuning ignores explicit evals parameter #297

Ethan-Russell opened this issue Dec 22, 2022 · 1 comment · Fixed by #318
Labels

Comments

@Ethan-Russell
Copy link

I have a mutating function that I am trying to benchmark which requires setup to be run before every function call, or else it will throw an error. I set the evals parameter to 1 thinking that would do the trick. However, I'm finding that tune! will ignore this and run multiple times per setup. Here is a minimum working example:

function myfunc(v)
   isempty(v) || error()
   push!(v,1)
end

julia> @btime myfunc(v) setup=(v=Int[]) evals=1
  0.001 ns (1 allocation: 80 bytes)
1-element Vector{Int64}:
 1

julia> b = @benchmarkable myfunc(v) setup=(v=Int[]) evals=1
Benchmark(evals=1, seconds=5.0, samples=10000)

julia> tune!(b)
ERROR:
Stacktrace:
  [1] error()
    @ Base .\error.jl:42
  [2] myfunc
    @ .\REPL[2]:5 [inlined]
  [3] var"##core#315"(v::Vector{Int64})
    @ Main C:\Users\erussell\.julia\packages\BenchmarkTools\7xSXH\src\execution.jl:489
  [4] var"##sample#316"(::Tuple{}, __params::BenchmarkTools.Parameters)
    @ Main C:\Users\erussell\.julia\packages\BenchmarkTools\7xSXH\src\execution.jl:497
  [5] _lineartrial(b::BenchmarkTools.Benchmark, p::BenchmarkTools.Parameters; maxevals::Int64, kwargs::Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
    @ BenchmarkTools C:\Users\erussell\.julia\packages\BenchmarkTools\7xSXH\src\execution.jl:161
  [6] _lineartrial(b::BenchmarkTools.Benchmark, p::BenchmarkTools.Parameters)
    @ BenchmarkTools C:\Users\erussell\.julia\packages\BenchmarkTools\7xSXH\src\execution.jl:153
  [7] #invokelatest#2
    @ .\essentials.jl:716 [inlined]
  [8] invokelatest
    @ .\essentials.jl:714 [inlined]
  [9] #lineartrial#46
    @ C:\Users\erussell\.julia\packages\BenchmarkTools\7xSXH\src\execution.jl:35 [inlined]
 [10] lineartrial
    @ C:\Users\erussell\.julia\packages\BenchmarkTools\7xSXH\src\execution.jl:35 [inlined]
 [11] tune!(b::BenchmarkTools.Benchmark, p::BenchmarkTools.Parameters; progressid::Nothing, nleaves::Float64, ndone::Float64, verbose::Bool, pad::String, kwargs::Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
    @ BenchmarkTools C:\Users\erussell\.julia\packages\BenchmarkTools\7xSXH\src\execution.jl:251
 [12] tune! (repeats 2 times)
    @ C:\Users\erussell\.julia\packages\BenchmarkTools\7xSXH\src\execution.jl:250 [inlined]
 [13] top-level scope
    @ REPL[7]:1
@gdalle
Copy link
Collaborator

gdalle commented Jun 13, 2023

Same issue as #24

@gdalle gdalle linked a pull request Jun 14, 2023 that will close this issue
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants