[single-machine-performance] file_to_blackhole: change optimization goal to egress throughput #15631
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This pull request intends to be a follow-up to PR #14528. Currently, regression detector jobs for the `file_to_blackhole` experiment use ingress throughput as an optimization goal. Recent work on single-machine-performance now enables users to set a single optimization goal per experiment instead of a single optimization goal for all experiments. This pull request changes the optimization goal of the `file_to_blackhole` experiment to egress throughput, which is of more interest than ingress throughput for that experiment. In addition, this commit updates the Regression Detection README to include information on how to set an optimization goal for each experiment. If an `experiment.yaml` file is absent for an experiment, the optimization goal for that experiment defaults to ingress throughput. This value can be changed for an experiment by placing an `experiment.yaml` file at the root of the directory tree for a given experiment, e.g., for the `file_to_blackhole` experiment, at `test/regression/cases/file_to_blackhole/experiment.yaml`. The contents of an example `experiment.yaml` file are ```yaml optimization_goal: egress_throughput ``` As the default, `ingress_throughput` is currently the only other supported value of `optimization_goal`. Signed-off-by: Geoffrey M. Oxberry <[email protected]>
goxberry
added
changelog/no-changelog
[deprecated] team/agent-platform
[deprecated] qa/skip-qa - use other qa/ labels
|
|
To enable the `egress_throughput` change to `file_to_blackhole`, the versions of `smp` and `lading` must be updated. Actually, another `smp` release is needed and the `smp` version used by `datadog-agent` must be updated to *that* version, but the version updates in this commit are the most recent versions available at the moment. Signed-off-by: Geoffrey M. Oxberry <[email protected]>
The `--target-inherit-environment` flag in `smp` has been removed because targets now inherit environment variables by default, as of `smp` 0.7.0. This commit removes that obsolete flag from `smp` calls in the Agent CI configuration. Signed-off-by: Geoffrey M. Oxberry <[email protected]>
blt
approved these changes
Feb 16, 2023
To enable per-experiment optimization goal support, the version of `smp` used by Agent must be version 0.7.1 or later. This commit updates the `smp` version used by Agent's CI pipeline to version 0.7.1 from 0.7.0 for that support. Signed-off-by: Geoffrey M. Oxberry <[email protected]>
blt
approved these changes
Feb 17, 2023
bkabrda
approved these changes
Feb 17, 2023
goxberry
deleted the
goxberry/set-file_to_blackhole-regression-optimization_goal-to-egress_throughput
branch
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What does this PR do?
This pull request changes the optimization goal of the
file_to_blackholeexperiment to egress throughput by adding anexperiment.yamlfile to that experiment. In addition, this pull request updates the Regression Detection README to include information on how to set an optimization goal for each experiment.Motivation
Measuring egress throughput for the
file_to_blackholeexperiment is of more interest than measuring ingress throughput; the latter is the current optimization goal for that experiment.Additional Notes
These changes follow upon recent work on single-machine-performance to enables users to set a single optimization goal per experiment instead of a single optimization goal for all experiments. These changes also follow up on PR #14528.
For each experiment, an
experiment.yamlfile is optional. If that file is absent for an experiment, the optimization goal for that experiment defaults to ingress throughput. As illustrated in this pull request, the optimization goal for an experiment can be changed for an experiment by placing anexperiment.yamlfile at the root of the directory tree for that experiment, e.g., for thefile_to_blackholeexperiment, attest/regression/cases/file_to_blackhole/experiment.yaml.The contents of an example
experiment.yamlfile areThe only other value of
optimization_goalcurrently supported isingress_throughput.Backend changes required to support
experiment.yamlfiles have already been deployed, so in theory, once this PR is merged,file_to_blackholeresults for egress throughput should replace the ingress throughput results currently emitted.Possible Drawbacks / Trade-offs
This pull request trades off additional complexity -- an optional additional file per experiment -- for flexibility in defining an optimization goal for each experiment, which supports more use cases.
Describe how to test/QA your changes
Changes have been dogfed using our dogfood project, so hopefully, this change will work without any disruptions. If it doesn't, reach out and we'd be glad to help.
Reviewer's Checklist
Triagemilestone is set.major_changelabel if your change either has a major impact on the code base, is impacting multiple teams or is changing important well-established internals of the Agent. This label will be use during QA to make sure each team pay extra attention to the changed behavior. For any customer facing change use a releasenote.changelog/no-changeloglabel has been applied.qa/skip-qalabel is not applied.team/..label has been applied, indicating the team(s) that should QA this change.need-change/operatorandneed-change/helmlabels have been applied.k8s/<min-version>label, indicating the lowest Kubernetes version compatible with this feature.