Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement adapters to xunit-performance #679

Closed
AndreyAkinshin opened this issue Mar 10, 2018 · 3 comments
Closed

Implement adapters to xunit-performance #679

AndreyAkinshin opened this issue Mar 10, 2018 · 3 comments
Assignees
Labels

Comments

@AndreyAkinshin
Copy link
Member

The original idea by @AndyAyersMS:

How hard would it be for BDN to consume and do reasonable things with tests that contain xunit-performance attributes? That way we could still author perf tests like we do now but run them under either perf harness.

It should not be hard. In fact, the implementation of BenchmarkRunner.Run is pretty simple:

public static Summary Run<T>(IConfig config = null) =>
    BenchmarkRunnerCore.Run(BenchmarkConverter.TypeToBenchmarks(typeof(T), config), ToolchainExtensions.GetToolchain);

public static Summary Run(Type type, IConfig config = null) =>
    BenchmarkRunnerCore.Run(BenchmarkConverter.TypeToBenchmarks(type, config), ToolchainExtensions.GetToolchain);

public static Summary Run(Type type, MethodInfo[] methods, IConfig config = null) =>
    BenchmarkRunnerCore.Run(BenchmarkConverter.MethodsToBenchmarks(type, methods, config), ToolchainExtensions.GetToolchain);

BenchmarkConverter just takes method attributes and transform it to BenchmarkRunInfo which contains information about how to run the benchmarks without any knowledge about original attributes (see source code:
https://github.com/dotnet/BenchmarkDotNet/blob/v0.10.12/src/BenchmarkDotNet.Core/Running/BenchmarkConverter.cs)
It's pretty easy to make it pluggable and write XunitPerformanceBenchmarkConverter which transforms xunit-performance attributes to the BenchmarkDotNet represtnation.
I like this idea because it will allow to run all existed xunit benchmarks on BenchmarkDotNet without massive modifications of the source code.

@AndreyAkinshin AndreyAkinshin added this to the v0.11.x milestone Mar 10, 2018
@AndreyAkinshin AndreyAkinshin self-assigned this Mar 10, 2018
@AndreyAkinshin
Copy link
Member Author

This issue depends on #678

@adamsitnik
Copy link
Member

I did some research and it seems that it's not doable.

Why: In xunit-performance you don't just add [Benchmark] attribute, you also need to run the iterations in explicit way.

An example of empty benchmark from the xunit-performance docs:

[Benchmark]
void EmptyBenchmark()
{
    foreach (var iteration in Benchmark.Iterations)
        using (iteration.StartMeasurement())
            ; //do nothing
}

[Benchmark]
void EmptyBenchmark()
{
    Benchmark.Iterate(() => { /*do nothing*/ });
}

while in BenchmarkDotNet we do it out of the box for the user

@AndreyAkinshin
Copy link
Member Author

@adamsitnik, =(

@AndreyAkinshin AndreyAkinshin removed this from the v0.11.x milestone Mar 15, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants