Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Runtime Restriction of Benchmark Methods #27

Open
wants to merge 7 commits into
base: master
Choose a base branch
from

Conversation

Matthimatiker
Copy link
Contributor

This change introduces a new (optional) annotation @maxRuntime, which allows one to restrict the runtime of a benchmark method to a specified number of seconds.

This is useful, if a benchmark is executed on different hardware or if a generic event is used to benchmark several implementations with really different performance characteristics.
In these cases, you might want to stop a benchmark early instead of waiting for hours until the results of a slow system are available.

In the following example, the benchmark method slowIndexingAlgo() is stopped once 10000 iterations are reached or if the runtime of 5 minutes (300 seconds) is exceeded:

    /**
     * @iterations 10000
     * @maxRuntime 300
     */
    public function slowIndexingAlgo()
    {
        $this->slow->index($this->data);
    }

@coveralls
Copy link

Coverage Status

Coverage increased (+0.77%) when pulling cbd99ca on Matthimatiker:restrict_runtime_pull_request into 51fe4b6 on polyfractal:master.

@Matthimatiker
Copy link
Contributor Author

@polyfractal Any chance to get this merged?

@polyfractal
Copy link
Owner

@gabrielsch ^^^^ ?

I've largely stepped aside as maintainer of this library and given it over to @gabrielsch . Unless he has some objections, it LGTM from a casual skim and I'll merge

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants