Removing Delay in Smoke Test Helper Method #6403

link04 · 2024-12-05T14:41:50Z

Summary of changes

Removing as it didn't fix flake but on the contrary seems like maybe a new one was introduced as a result of it.

datadog-ddstaging · 2024-12-05T15:24:14Z

Datadog Report

Branch report: maximo/adding-delay-in-CheckForSmoke
Commit report: 32f1311
Test service: dd-trace-dotnet

✅ 0 Failed, 451390 Passed, 2729 Skipped, 19h 7m 40.56s Total Time
❄️ 2 New Flaky

New Flaky Tests (2)

AdditionalArguments - Datadog.Trace.Tools.dd_dotnet.ArtifactTests.RunCommandTests - Last Failure

Expand for error

 
 
 With StandardOutput:
 Waiting - PID: 680 - Main thread: 8932 - Profiler attached: True Args: traces 5 --dd-env test Sending 5 spans 
 With ErrorOutput:
  Unhandled Exception: System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation. ---> System.DllNotFoundException: Unable to load DLL 'datadog_profiling_ffi': The specified module could not be found. (Exception from HRESULT: 0x8007007E)    at Datadog.Trace.LibDatadog.TraceExporter.Native.ddog_trace_exporter_new(IntPtr& outHandle, CharSlice url, CharSlice tracerVersion, CharSlice language, CharSlice languageVersion, CharSlice languageInterpreter, CharSlice hostname, CharSlice env, CharSlice version, CharSlice service, TraceExporterInputFormat inputFormat, TraceExporterOutputFormat outputFormat, Boolean computeStats, AgentResponseCallback agentResponseCallback)    at Datadog.Trace.LibDatadog.TraceExporter..ctor(ImmutableTracerSettings settings)    at Datadog.Trace.TracerManagerFactory.GetApi(ImmutableTracerSettings settings, IDogStatsd statsd, Action\`1 updateSampleRates, IApiRequestFactory apiRequestFactory)    at Datadog.Trace.TracerManagerFactory.GetAgentWriter(ImmutableTracerSettings settings, IDogStatsd statsd, Action\`1 updateSampleRates, IDiscoveryService discoveryService)    at Datadog.Trace.TracerManagerFactory.CreateTracerManager(ImmutableTracerSettings settings, IAgentWriter agentWriter, ITraceSampler sampler, IScopeManager scopeMan ITelemetryController telemetry, IDiscoveryService discoveryService, DataStreamsManager dataStreamsManager, IRemoteConfigurationManager remoteConfigurationManager, IDynamicConfigurationManager dynamicConfigurationManager, ITracerFlareManager tracerFlareManager)    at Datadog.Trace.TracerManagerFactory.CreateTracerManager(ImmutableTracerSettings settings, TracerManager previous)    at Datadog.Trace.TracerManager.CreateInitializedTracer(ImmutableTracerSettings settings, TracerManagerFactory factory)    at System.Threading.LazyInitializer.EnsureInitializedCore[T](T& target, Boolean& initialized, Object& syncLock, Func\`1 valueFactory)    at Datadog.Trace.TracerManager.get_Instance()    at Datadog.Trace.Tracer.get_Instance()    --- End of inner exception stack trace ---    at System.RuntimeMethodHandle.InvokeMethod(Object target, Object[] arguments, Signature sig, Boolean constructor)    at System.Reflection.RuntimeMethodInfo.UnsafeInvokeInternal(Object obj, Object[] parameters, Object[] arguments)    at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)    at Samples.SampleHelpers.CreateScope(String operationName)    at Samples.Console_.Program.<AsyncMain>d__2.MoveNext() --- End of stack trace from previous location where exception was thrown ---    at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()    at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)    at Samples.Console_.Program.Main(String[] args) ".

Run - Datadog.Trace.Tools.dd_dotnet.ArtifactTests.RunCommandTests - Last Failure
Expand for error
```
 Expected exitCode to be 0, but found -1073741819 (difference of -1073741819).
```

andrewlock · 2024-12-05T15:49:20Z

Execution-Time Benchmarks Report ⏱️

Execution-time results for samples comparing the following branches/commits:

Execution-time benchmarks measure the whole time it takes to execute a program. And are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are shown in red. The following thresholds were used for comparing the execution times:

Welch test with statistical test for significance of 5%
Only results indicating a difference greater than 5% and 5 ms are considered.

Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard.

Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph).

gantt
    title Execution time (ms) FakeDbCommand (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6403) - mean (69ms)  : 66, 71
     .   : milestone, 69,
    master - mean (69ms)  : 66, 73
     .   : milestone, 69,

    section CallTarget+Inlining+NGEN
    This PR (6403) - mean (980ms)  : 953, 1006
     .   : milestone, 980,
    master - mean (975ms)  : 946, 1004
     .   : milestone, 975,

gantt
    title Execution time (ms) FakeDbCommand (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6403) - mean (108ms)  : 105, 110
     .   : milestone, 108,
    master - mean (108ms)  : 105, 110
     .   : milestone, 108,

    section CallTarget+Inlining+NGEN
    This PR (6403) - mean (676ms)  : 660, 692
     .   : milestone, 676,
    master - mean (680ms)  : 667, 693
     .   : milestone, 680,

gantt
    title Execution time (ms) FakeDbCommand (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6403) - mean (91ms)  : 90, 93
     .   : milestone, 91,
    master - mean (91ms)  : 90, 93
     .   : milestone, 91,

    section CallTarget+Inlining+NGEN
    This PR (6403) - mean (632ms)  : 617, 648
     .   : milestone, 632,
    master - mean (632ms)  : 615, 649
     .   : milestone, 632,

gantt
    title Execution time (ms) HttpMessageHandler (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6403) - mean (190ms)  : 187, 194
     .   : milestone, 190,
    master - mean (190ms)  : 186, 194
     .   : milestone, 190,

    section CallTarget+Inlining+NGEN
    This PR (6403) - mean (1,103ms)  : 1075, 1131
     .   : milestone, 1103,
    master - mean (1,091ms)  : 1060, 1122
     .   : milestone, 1091,

gantt
    title Execution time (ms) HttpMessageHandler (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6403) - mean (278ms)  : 272, 283
     .   : milestone, 278,
    master - mean (277ms)  : 273, 281
     .   : milestone, 277,

    section CallTarget+Inlining+NGEN
    This PR (6403) - mean (876ms)  : 843, 909
     .   : milestone, 876,
    master - mean (867ms)  : 844, 890
     .   : milestone, 867,

gantt
    title Execution time (ms) HttpMessageHandler (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6403) - mean (266ms)  : 260, 272
     .   : milestone, 266,
    master - mean (267ms)  : 262, 271
     .   : milestone, 267,

    section CallTarget+Inlining+NGEN
    This PR (6403) - mean (853ms)  : 819, 888
     .   : milestone, 853,
    master - mean (849ms)  : 822, 876
     .   : milestone, 849,

andrewlock · 2024-12-05T16:19:24Z

Benchmarks Report for tracer 🐌

Benchmarks for #6403 compared to master:

3 benchmarks are slower, with geometric mean 1.138
All benchmarks have the same allocations

The following thresholds were used for comparing the benchmark speeds:

Mann–Whitney U test with statistical test for significance of 5%
Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.ActivityBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Gen 2	Allocated
master	`StartStopWithChild`	net6.0	7.99μs	44.2ns	290ns	0.0159	0.00797	0	5.61 KB
master	`StartStopWithChild`	netcoreapp3.1	10μs	54.9ns	325ns	0.0192	0.00959	0	5.8 KB
master	`StartStopWithChild`	net472	16.3μs	43.1ns	155ns	1.04	0.301	0.095	6.2 KB
#6403	`StartStopWithChild`	net6.0	7.87μs	42.7ns	242ns	0.0157	0.00784	0	5.61 KB
#6403	`StartStopWithChild`	netcoreapp3.1	9.96μs	55.6ns	352ns	0.0152	0.00506	0	5.8 KB
#6403	`StartStopWithChild`	net472	16.1μs	35.5ns	133ns	1.05	0.318	0.0978	6.2 KB

Benchmarks.Trace.AgentWriterBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`WriteAndFlushEnrichedTraces`	net6.0	483μs	244ns	945ns	0	2.7 KB
master	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	691μs	257ns	963ns	0	2.7 KB
master	`WriteAndFlushEnrichedTraces`	net472	860μs	472ns	1.83μs	0.431	3.3 KB
#6403	`WriteAndFlushEnrichedTraces`	net6.0	496μs	302ns	1.17μs	0	2.7 KB
#6403	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	669μs	286ns	1.07μs	0	2.7 KB
#6403	`WriteAndFlushEnrichedTraces`	net472	864μs	517ns	2μs	0.428	3.3 KB

Benchmarks.Trace.AspNetCoreBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`SendRequest`	net6.0	149μs	852ns	6.32μs	0.146	14.47 KB
master	`SendRequest`	netcoreapp3.1	166μs	959ns	8.13μs	0.167	17.27 KB
master	`SendRequest`	net472	0.00049ns	0.000366ns	0.00132ns	0	0 b
#6403	`SendRequest`	net6.0	151μs	861ns	6.03μs	0.158	14.47 KB
#6403	`SendRequest`	netcoreapp3.1	162μs	893ns	7.31μs	0.168	17.27 KB
#6403	`SendRequest`	net472	0.000383ns	0.000265ns	0.000992ns	0	0 b

Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Gen 2	Allocated
master	`WriteAndFlushEnrichedTraces`	net6.0	566μs	2.91μs	14.9μs	0.584	0	0	41.6 KB
master	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	670μs	3.21μs	18.7μs	0.338	0	0	41.73 KB
master	`WriteAndFlushEnrichedTraces`	net472	862μs	3.65μs	13.7μs	8.39	2.52	0.419	53.36 KB
#6403	`WriteAndFlushEnrichedTraces`	net6.0	564μs	2.93μs	13.8μs	0.546	0	0	41.64 KB
#6403	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	663μs	3.34μs	15μs	0.317	0	0	41.86 KB
#6403	`WriteAndFlushEnrichedTraces`	net472	843μs	3.34μs	12.5μs	8.75	2.5	0.417	53.3 KB

Benchmarks.Trace.DbCommandBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Allocated
master	`ExecuteNonQuery`	net6.0	1.22μs	1.07ns	4.13ns	0.014	0	1.02 KB
master	`ExecuteNonQuery`	netcoreapp3.1	1.8μs	0.87ns	3.37ns	0.0135	0	1.02 KB
master	`ExecuteNonQuery`	net472	2.13μs	2.96ns	11.1ns	0.157	0.00107	987 B
#6403	`ExecuteNonQuery`	net6.0	1.33μs	0.93ns	3.6ns	0.014	0	1.02 KB
#6403	`ExecuteNonQuery`	netcoreapp3.1	1.76μs	2.22ns	8.02ns	0.0132	0	1.02 KB
#6403	`ExecuteNonQuery`	net472	2.07μs	1.93ns	7.47ns	0.156	0.00103	987 B

Benchmarks.Trace.ElasticsearchBenchmark - Slower ⚠️ Same allocations ✔️

Slower ⚠️ in #6403

Benchmark	diff/base	Base Median (ns)	Diff Median (ns)	Modality
Benchmarks.Trace.ElasticsearchBenchmark.CallElasticsearchAsync‑net6.0	1.128	1,174.82	1,325.15

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`CallElasticsearch`	net6.0	1.2μs	0.364ns	1.41ns	0.0138	976 B
master	`CallElasticsearch`	netcoreapp3.1	1.57μs	0.718ns	2.78ns	0.0132	976 B
master	`CallElasticsearch`	net472	2.5μs	2.23ns	8.33ns	0.157	995 B
master	`CallElasticsearchAsync`	net6.0	1.18μs	0.475ns	1.84ns	0.0135	952 B
master	`CallElasticsearchAsync`	netcoreapp3.1	1.73μs	0.943ns	3.27ns	0.0138	1.02 KB
master	`CallElasticsearchAsync`	net472	2.66μs	2.42ns	9.38ns	0.167	1.05 KB
#6403	`CallElasticsearch`	net6.0	1.3μs	0.608ns	2.19ns	0.0138	976 B
#6403	`CallElasticsearch`	netcoreapp3.1	1.62μs	0.592ns	2.29ns	0.0129	976 B
#6403	`CallElasticsearch`	net472	2.63μs	2.5ns	9.67ns	0.158	995 B
#6403	`CallElasticsearchAsync`	net6.0	1.32μs	1.29ns	5.01ns	0.0132	952 B
#6403	`CallElasticsearchAsync`	netcoreapp3.1	1.66μs	2.95ns	11.4ns	0.014	1.02 KB
#6403	`CallElasticsearchAsync`	net472	2.76μs	0.898ns	3.48ns	0.167	1.05 KB

Benchmarks.Trace.GraphQLBenchmark - Slower ⚠️ Same allocations ✔️

Slower ⚠️ in #6403

Benchmark	diff/base	Base Median (ns)	Diff Median (ns)	Modality
Benchmarks.Trace.GraphQLBenchmark.ExecuteAsync‑net6.0	1.138	1,157.51	1,316.80

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`ExecuteAsync`	net6.0	1.16μs	0.62ns	2.4ns	0.0134	952 B
master	`ExecuteAsync`	netcoreapp3.1	1.73μs	0.717ns	2.78ns	0.013	952 B
master	`ExecuteAsync`	net472	1.87μs	0.35ns	1.35ns	0.145	915 B
#6403	`ExecuteAsync`	net6.0	1.32μs	0.391ns	1.46ns	0.0134	952 B
#6403	`ExecuteAsync`	netcoreapp3.1	1.7μs	1.19ns	4.62ns	0.0129	952 B
#6403	`ExecuteAsync`	net472	1.86μs	0.619ns	2.32ns	0.145	915 B

Benchmarks.Trace.HttpClientBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`SendAsync`	net6.0	4.55μs	1.77ns	6.87ns	0.0315	2.31 KB
master	`SendAsync`	netcoreapp3.1	5.21μs	1.37ns	5.32ns	0.0393	2.85 KB
master	`SendAsync`	net472	7.35μs	2.75ns	10.6ns	0.495	3.12 KB
#6403	`SendAsync`	net6.0	4.41μs	1.61ns	6.23ns	0.033	2.31 KB
#6403	`SendAsync`	netcoreapp3.1	5.41μs	1.87ns	7ns	0.0378	2.85 KB
#6403	`SendAsync`	net472	7.35μs	2.27ns	8.81ns	0.493	3.12 KB

Benchmarks.Trace.ILoggerBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`EnrichedLog`	net6.0	1.49μs	2.42ns	9.37ns	0.0227	1.64 KB
master	`EnrichedLog`	netcoreapp3.1	2.24μs	0.95ns	3.42ns	0.0223	1.64 KB
master	`EnrichedLog`	net472	2.5μs	1.65ns	6.39ns	0.249	1.57 KB
#6403	`EnrichedLog`	net6.0	1.47μs	0.544ns	2.04ns	0.0231	1.64 KB
#6403	`EnrichedLog`	netcoreapp3.1	2.14μs	0.863ns	3.11ns	0.0214	1.64 KB
#6403	`EnrichedLog`	net472	2.67μs	1.82ns	6.82ns	0.25	1.57 KB

Benchmarks.Trace.Log4netBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Allocated
master	`EnrichedLog`	net6.0	121μs	187ns	726ns	0	0	4.28 KB
master	`EnrichedLog`	netcoreapp3.1	123μs	135ns	503ns	0	0	4.28 KB
master	`EnrichedLog`	net472	151μs	191ns	738ns	0.676	0.225	4.46 KB
#6403	`EnrichedLog`	net6.0	120μs	118ns	458ns	0.0601	0	4.28 KB
#6403	`EnrichedLog`	netcoreapp3.1	125μs	130ns	485ns	0	0	4.28 KB
#6403	`EnrichedLog`	net472	153μs	134ns	520ns	0.693	0.231	4.46 KB

Benchmarks.Trace.NLogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`EnrichedLog`	net6.0	3.02μs	1.04ns	4.03ns	0.03	2.2 KB
master	`EnrichedLog`	netcoreapp3.1	4.34μs	1.15ns	4.13ns	0.0303	2.2 KB
master	`EnrichedLog`	net472	4.98μs	1.63ns	6.32ns	0.319	2.02 KB
#6403	`EnrichedLog`	net6.0	2.99μs	1.78ns	6.88ns	0.0298	2.2 KB
#6403	`EnrichedLog`	netcoreapp3.1	4.09μs	1.74ns	6.73ns	0.0287	2.2 KB
#6403	`EnrichedLog`	net472	5μs	1.72ns	6.65ns	0.319	2.02 KB

Benchmarks.Trace.RedisBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`SendReceive`	net6.0	1.35μs	1.12ns	4.35ns	0.0163	1.14 KB
master	`SendReceive`	netcoreapp3.1	1.8μs	2.06ns	7.98ns	0.0152	1.14 KB
master	`SendReceive`	net472	2.14μs	1.71ns	6.62ns	0.184	1.16 KB
#6403	`SendReceive`	net6.0	1.39μs	0.463ns	1.73ns	0.016	1.14 KB
#6403	`SendReceive`	netcoreapp3.1	1.74μs	2.02ns	7.84ns	0.015	1.14 KB
#6403	`SendReceive`	net472	2.08μs	0.867ns	3.24ns	0.183	1.16 KB

Benchmarks.Trace.SerilogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`EnrichedLog`	net6.0	2.85μs	1.96ns	7.6ns	0.0228	1.6 KB
master	`EnrichedLog`	netcoreapp3.1	3.98μs	1.25ns	4.83ns	0.022	1.65 KB
master	`EnrichedLog`	net472	4.35μs	1.7ns	6.14ns	0.323	2.04 KB
#6403	`EnrichedLog`	net6.0	2.76μs	0.718ns	2.69ns	0.0221	1.6 KB
#6403	`EnrichedLog`	netcoreapp3.1	3.9μs	1.96ns	7.34ns	0.0213	1.65 KB
#6403	`EnrichedLog`	net472	4.44μs	3.12ns	12.1ns	0.323	2.04 KB

Benchmarks.Trace.SpanBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`StartFinishSpan`	net6.0	393ns	0.736ns	2.85ns	0.00805	576 B
master	`StartFinishSpan`	netcoreapp3.1	589ns	1.02ns	3.94ns	0.00795	576 B
master	`StartFinishSpan`	net472	598ns	1.58ns	6.12ns	0.0918	578 B
master	`StartFinishScope`	net6.0	475ns	0.628ns	2.43ns	0.00977	696 B
master	`StartFinishScope`	netcoreapp3.1	735ns	0.963ns	3.73ns	0.00951	696 B
master	`StartFinishScope`	net472	868ns	2.36ns	9.14ns	0.105	658 B
#6403	`StartFinishSpan`	net6.0	396ns	0.496ns	1.92ns	0.00811	576 B
#6403	`StartFinishSpan`	netcoreapp3.1	588ns	0.893ns	3.46ns	0.00765	576 B
#6403	`StartFinishSpan`	net472	576ns	1.17ns	4.52ns	0.0916	578 B
#6403	`StartFinishScope`	net6.0	477ns	0.704ns	2.73ns	0.00981	696 B
#6403	`StartFinishScope`	netcoreapp3.1	685ns	0.371ns	1.44ns	0.00955	696 B
#6403	`StartFinishScope`	net472	923ns	2.52ns	9.78ns	0.104	658 B

Benchmarks.Trace.TraceAnnotationsBenchmark - Slower ⚠️ Same allocations ✔️

Slower ⚠️ in #6403

Benchmark	diff/base	Base Median (ns)	Diff Median (ns)	Modality
Benchmarks.Trace.TraceAnnotationsBenchmark.RunOnMethodBegin‑netcoreapp3.1	1.149	869.18	998.40

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`RunOnMethodBegin`	net6.0	612ns	1.01ns	3.9ns	0.00959	696 B
master	`RunOnMethodBegin`	netcoreapp3.1	868ns	1.87ns	7.22ns	0.0092	696 B
master	`RunOnMethodBegin`	net472	1.14μs	2.3ns	8.89ns	0.104	658 B
#6403	`RunOnMethodBegin`	net6.0	584ns	0.954ns	3.69ns	0.0098	696 B
#6403	`RunOnMethodBegin`	netcoreapp3.1	997ns	1.83ns	6.85ns	0.00956	696 B
#6403	`RunOnMethodBegin`	net472	1.13μs	2.12ns	8.21ns	0.104	658 B

andrewlock · 2024-12-05T19:38:53Z

Throughput/Crank Report ⚡

Throughput results for AspNetCoreSimpleController comparing the following branches/commits:

Cases where throughput results for the PR are worse than latest master (5% drop or greater), results are shown in red.

Note that these results are based on a single point-in-time result for each branch. For full results, see one of the many, many dashboards!

gantt
    title Throughput Linux x64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (6403) (11.204M)   : 0, 11204485
    master (11.130M)   : 0, 11130050
    benchmarks/2.9.0 (11.033M)   : 0, 11032866

    section Automatic
    This PR (6403) (7.339M)   : 0, 7339047
    master (7.379M)   : 0, 7379130
    benchmarks/2.9.0 (7.786M)   : 0, 7785853

    section Trace stats
    master (7.581M)   : 0, 7580626

    section Manual
    master (11.172M)   : 0, 11171761

    section Manual + Automatic
    This PR (6403) (6.709M)   : 0, 6708618
    master (6.881M)   : 0, 6880813

    section DD_TRACE_ENABLED=0
    master (10.206M)   : 0, 10205701

gantt
    title Throughput Linux arm64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (6403) (9.743M)   : 0, 9742559
    master (9.471M)   : 0, 9470791
    benchmarks/2.9.0 (9.495M)   : 0, 9494821

    section Automatic
    This PR (6403) (6.375M)   : 0, 6375312
    master (6.383M)   : 0, 6382578

    section Trace stats
    master (6.636M)   : 0, 6636070

    section Manual
    master (9.504M)   : 0, 9504212

    section Manual + Automatic
    This PR (6403) (6.003M)   : 0, 6003238
    master (5.979M)   : 0, 5979420

    section DD_TRACE_ENABLED=0
    master (8.801M)   : 0, 8801277

gantt
    title Throughput Windows x64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (6403) (9.917M)   : 0, 9916823
    master (10.088M)   : 0, 10087506
    benchmarks/2.9.0 (10.020M)   : 0, 10019592

    section Automatic
    This PR (6403) (6.253M)   : 0, 6253017
    master (6.580M)   : 0, 6579757
    benchmarks/2.9.0 (7.255M)   : 0, 7255257

    section Trace stats
    master (7.196M)   : 0, 7196377

    section Manual
    master (9.826M)   : 0, 9825558

    section Manual + Automatic
    This PR (6403) (5.638M)   : 0, 5637658
    master (5.890M)   : 0, 5890057

    section DD_TRACE_ENABLED=0
    master (9.442M)   : 0, 9441593

## Summary of changes Removing as it didn't fix flake but on the contrary seems like maybe a new one was introduced as a result of it.

link04 and others added 4 commits December 3, 2024 17:23

adding delay at end of helper method

91f8acf

Merge branch 'master' into maximo/adding-delay-in-CheckForSmoke

b1ab0ae

Removing delay since it didn't solve flake

5842521

Merge branch 'master' into maximo/adding-delay-in-CheckForSmoke

953a8e5

Actually removing delay...

32f1311

andrewlock approved these changes Dec 5, 2024

View reviewed changes

link04 merged commit fd813c8 into master Dec 5, 2024
64 of 65 checks passed

link04 deleted the maximo/adding-delay-in-CheckForSmoke branch December 5, 2024 20:49

github-actions bot added this to the vNext-v3 milestone Dec 5, 2024

veerbia pushed a commit that referenced this pull request Dec 16, 2024

Removing Delay in Smoke Test Helper Method (#6403)

2ec56f8

## Summary of changes Removing as it didn't fix flake but on the contrary seems like maybe a new one was introduced as a result of it.

andrewlock removed area:tracer The core tracer library (Datadog.Trace, does not include OpenTracing, native code, or integrations) area:builds project files, build scripts, pipelines, versioning, releases, packages area:docs area:opentracing area:installers labels Dec 18, 2024

andrewlock removed area:vendors Code from other vendors area:shared-components area:profiler Issues related to the continous-profiler area:debugger area:integrations labels Dec 18, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Removing Delay in Smoke Test Helper Method #6403

Removing Delay in Smoke Test Helper Method #6403

link04 commented Dec 5, 2024

datadog-ddstaging bot commented Dec 5, 2024 •

edited

Loading

andrewlock commented Dec 5, 2024

andrewlock commented Dec 5, 2024

Raw results

Raw results

Raw results

Raw results

Raw results

Slower ⚠️ in #6403

Raw results

Slower ⚠️ in #6403

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Slower ⚠️ in #6403

Raw results

andrewlock commented Dec 5, 2024

Removing Delay in Smoke Test Helper Method #6403

Removing Delay in Smoke Test Helper Method #6403

Conversation

link04 commented Dec 5, 2024

Summary of changes

datadog-ddstaging bot commented Dec 5, 2024 • edited Loading

Datadog Report

New Flaky Tests (2)

andrewlock commented Dec 5, 2024

Execution-Time Benchmarks Report ⏱️

andrewlock commented Dec 5, 2024

Benchmarks Report for tracer 🐌

Benchmark details

Raw results

Raw results

Raw results

Raw results

Raw results

Slower ⚠️ in #6403

Raw results

Slower ⚠️ in #6403

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Slower ⚠️ in #6403

Raw results

andrewlock commented Dec 5, 2024

Throughput/Crank Report ⚡

datadog-ddstaging bot commented Dec 5, 2024 •

edited

Loading