Skip to content

Conversation

@blt
Copy link
Contributor

@blt blt commented Nov 9, 2022

In #15142 we introduced the split between the trusted and untrusted regresison detector workflow. Unfortunately the trusted side can only be tested once it's merged into master branch and we had a bug. The github-scripts that download artifacts referenced github.actions.rest when it should have read github.rest.actions.

Please note that because of the nature of workflow_run triggered workflows this change will not be reflected in the PR's workflow runs.

Signed-off-by: Brian L. Troutwine [email protected]

In #15142 we introduced the split between the trusted and untrusted regresison
detector workflow. Unfortunately the trusted side can only be tested once it's
merged into master branch and we had a bug. The github-scripts that download
artifacts referenced `github.actions.rest` when it should have read
`github.rest.actions`.

Please note that because of the nature of workflow_run triggered workflows this
change will not be reflected in the PR's workflow runs.

Signed-off-by: Brian L. Troutwine <[email protected]>
@netlify
Copy link

netlify bot commented Nov 9, 2022

Deploy Preview for vrl-playground canceled.

Name Link
🔨 Latest commit 8ead858
🔍 Latest deploy log https://app.netlify.com/sites/vrl-playground/deploys/636b1af58f6f1b0008762770

@netlify
Copy link

netlify bot commented Nov 9, 2022

Deploy Preview for vector-project ready!

Name Link
🔨 Latest commit 8ead858
🔍 Latest deploy log https://app.netlify.com/sites/vector-project/deploys/636b1af58d96cc0008c7e487
😎 Deploy Preview https://deploy-preview-15153--vector-project.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site settings.

@github-actions
Copy link

github-actions bot commented Nov 9, 2022

Soak Test Results

Baseline: 067795e
Comparison: 8ead858
Total Vector CPUs: 4

Explanation

A soak test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine, quickly, if vector performance is changed and to what degree by a pull request. Where appropriate units are scaled per-core.

The table below, if present, lists those experiments that have experienced a statistically significant change in their throughput performance between baseline and comparision SHAs, with 90.0% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±8.87% change in mean throughput are discarded. An experiment is erratic if its coefficient of variation is greater than 0.3. The abbreviated table will be omitted if no interesting changes are observed.

No interesting changes in throughput with confidence ≥ 90.00% and absolute Δ mean >= ±8.87%:

Fine details of change detection per experiment.
experiment Δ mean Δ mean % confidence baseline mean baseline stdev baseline stderr baseline outlier % baseline CoV comparison mean comparison stdev comparison stderr comparison outlier % comparison CoV erratic declared erratic
syslog_loki 279.0KiB 1.9 100.00% 14.34MiB 1.19MiB 24.92KiB 0 0.082952 14.61MiB 1.23MiB 25.65KiB 0 0.0843357 False False
datadog_agent_remap_blackhole 381.03KiB 0.65 99.98% 57.06MiB 4.0MiB 83.3KiB 0 0.0700987 57.43MiB 2.96MiB 61.72KiB 0 0.0515086 False False
syslog_humio_logs 87.99KiB 0.53 100.00% 16.29MiB 165.46KiB 3.38KiB 0 0.00991955 16.37MiB 112.63KiB 2.31KiB 0 0.00671661 False False
splunk_hec_route_s3 80.57KiB 0.37 81.41% 21.52MiB 2.15MiB 44.78KiB 0 0.099812 21.6MiB 1.97MiB 41.28KiB 0 0.0914045 False False
datadog_agent_remap_blackhole_acks 81.19KiB 0.14 61.35% 58.55MiB 3.85MiB 80.09KiB 0 0.0656582 58.63MiB 2.33MiB 48.72KiB 0 0.039706 False False
syslog_splunk_hec_logs 16.83KiB 0.1 61.85% 15.96MiB 715.85KiB 14.57KiB 0 0.0438001 15.97MiB 615.75KiB 12.56KiB 0 0.0376368 False False
http_text_to_http_json 38.37KiB 0.1 78.35% 38.81MiB 1.03MiB 21.59KiB 0 0.0266033 38.85MiB 1.07MiB 22.31KiB 0 0.0274725 False False
splunk_hec_to_splunk_hec_logs_noack 9.05KiB 0.04 58.28% 23.83MiB 433.44KiB 8.85KiB 0 0.0177601 23.84MiB 332.5KiB 6.79KiB 0 0.0136192 False False
splunk_hec_to_splunk_hec_logs_acks 7.35KiB 0.03 22.58% 23.74MiB 905.11KiB 18.4KiB 0 0.0372242 23.75MiB 876.01KiB 17.81KiB 0 0.0360164 False False
splunk_hec_indexer_ack_blackhole 7.1KiB 0.03 20.33% 23.73MiB 972.31KiB 19.77KiB 0 0.0399972 23.74MiB 943.96KiB 19.2KiB 0 0.0388198 False False
enterprise_http_to_http -72.21B -0 0.78% 23.85MiB 249.71KiB 5.1KiB 0 0.0102239 23.85MiB 248.46KiB 5.08KiB 0 0.0101727 False False
file_to_blackhole -62.91KiB -0.06 46.33% 95.34MiB 3.05MiB 63.16KiB 0 0.0319518 95.28MiB 3.84MiB 79.84KiB 0 0.0402733 False False
http_to_http_json -26.23KiB -0.11 96.04% 23.85MiB 373.95KiB 7.63KiB 0 0.0153104 23.82MiB 498.92KiB 10.2KiB 0 0.0204488 False False
datadog_agent_remap_datadog_logs_acks -66.55KiB -0.12 50.59% 53.2MiB 2.69MiB 56.16KiB 0 0.0504744 53.13MiB 3.82MiB 79.48KiB 0 0.0718462 False False
socket_to_socket_blackhole -45.99KiB -0.19 100.00% 23.52MiB 148.37KiB 3.03KiB 0 0.00615783 23.48MiB 271.01KiB 5.53KiB 0 0.0112695 False False
fluent_elasticsearch -217.25KiB -0.27 100.00% 79.47MiB 55.03KiB 1.11KiB 0 0.000676095 79.26MiB 1.86MiB 38.25KiB 0 0.023452 False False
syslog_log2metric_splunk_hec_metrics -45.06KiB -0.27 99.15% 16.16MiB 624.85KiB 12.72KiB 0 0.0377541 16.12MiB 562.44KiB 11.46KiB 0 0.0340758 False False
syslog_log2metric_humio_metrics -32.63KiB -0.33 99.99% 9.51MiB 157.82KiB 3.22KiB 0 0.0161951 9.48MiB 383.78KiB 7.82KiB 0 0.0395155 False False
http_to_http_noack -113.56KiB -0.47 100.00% 23.85MiB 250.3KiB 5.12KiB 0 0.0102486 23.74MiB 1.15MiB 24.0KiB 0 0.0485052 False False
http_pipelines_blackhole_acks -17.45KiB -0.52 96.11% 3.27MiB 342.58KiB 6.96KiB 0 0.102171 3.26MiB 234.64KiB 4.78KiB 0 0.0703462 False False
http_pipelines_blackhole -31.07KiB -0.67 100.00% 4.51MiB 121.53KiB 2.49KiB 0 0.0263149 4.48MiB 291.33KiB 5.94KiB 0 0.0635108 False False
datadog_agent_remap_datadog_logs -467.85KiB -0.82 100.00% 55.49MiB 240.37KiB 4.92KiB 0 0.00422951 55.03MiB 3.89MiB 80.91KiB 0 0.0706135 False False
http_to_http_acks -172.18KiB -0.97 52.01% 17.36MiB 8.32MiB 173.95KiB 0 0.479327 17.19MiB 8.17MiB 170.67KiB 0 0.475473 True True
http_pipelines_no_grok_blackhole -142.24KiB -1.33 100.00% 10.48MiB 282.21KiB 5.76KiB 0 0.0262866 10.34MiB 1.06MiB 22.11KiB 0 0.102669 False False
syslog_regex_logs2metric_ddmetrics -126.32KiB -1.62 100.00% 7.62MiB 905.71KiB 18.46KiB 0 0.11603 7.5MiB 962.9KiB 19.62KiB 0 0.125385 False False

Copy link
Contributor

@spencergilbert spencergilbert left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I love how wonky it is developing github actions 😄

@goxberry
Copy link
Contributor

goxberry commented Nov 9, 2022

I love how wonky it is developing github actions 😄

💯

@blt blt merged commit c17294b into master Nov 9, 2022
@blt blt deleted the undefined_rest branch November 9, 2022 13:48
@blt
Copy link
Contributor Author

blt commented Nov 9, 2022

I love how wonky it is developing github actions smile

100

It's not my favorite.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants