Skip to content

test/extended: check apiserver request latency #26739

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
dgrisonnet wants to merge 4 commits into from
Closed

test/extended: check apiserver request latency #26739

dgrisonnet wants to merge 4 commits into from

Conversation

@dgrisonnet
Copy link
Member

@dgrisonnet dgrisonnet commented Jan 12, 2022

Add test verifying that the apiserver request latency isn't heavily impacted by changes.

@openshift-ci openshift-ci bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jan 12, 2022
deads2k
deads2k reviewed Jan 12, 2022
View reviewed changes
deads2k
deads2k reviewed Jan 12, 2022
View reviewed changes
@dgrisonnet dgrisonnet force-pushed the apiserver-request-latency branch 2 times, most recently from f6ad46b to 3457c84 Compare January 12, 2022 17:19
dgrisonnet
dgrisonnet commented Jan 12, 2022
View reviewed changes
@dgrisonnet dgrisonnet force-pushed the apiserver-request-latency branch from 3457c84 to 258ff3d Compare January 12, 2022 17:23
deads2k
deads2k reviewed Jan 12, 2022
View reviewed changes
deads2k
deads2k reviewed Jan 12, 2022
View reviewed changes
deads2k
deads2k reviewed Jan 12, 2022
View reviewed changes
@dgrisonnet dgrisonnet force-pushed the apiserver-request-latency branch from 258ff3d to 9d47262 Compare January 12, 2022 18:13
@dgrisonnet dgrisonnet changed the title WIP: test/extended: expose apiserver request latency WIP: test/extended: check apiserver request latency Jan 12, 2022
@dgrisonnet dgrisonnet force-pushed the apiserver-request-latency branch from 9d47262 to 19a858f Compare January 12, 2022 20:23
deads2k
deads2k reviewed Jan 12, 2022
View reviewed changes
deads2k
deads2k reviewed Jan 12, 2022
View reviewed changes
deads2k
deads2k reviewed Jan 12, 2022
View reviewed changes
deads2k
deads2k reviewed Jan 12, 2022
View reviewed changes
@deads2k
Copy link
Contributor

deads2k commented Jan 12, 2022

I think this is close to being ready.

@dgrisonnet dgrisonnet force-pushed the apiserver-request-latency branch from 19a858f to 0c7df4f Compare January 12, 2022 21:00
@dgrisonnet
Copy link
Member Author

dgrisonnet commented Jan 12, 2022

Since the values that I observed in CI were varying a lot depending on the infrastructure and the kind of the job, I would take it slowly and first try to test these thresholds across all platforms before breaking anyone. To hasten the process, It might be worth adding this test as flaky for now until we have figured out thresholds that won't break anything and still be efficient, wdyt?

@deads2k
Copy link
Contributor

deads2k commented Jan 12, 2022

you need to hack/update-generate

@dgrisonnet dgrisonnet force-pushed the apiserver-request-latency branch from 0c7df4f to 681c047 Compare January 12, 2022 21:46
@dgrisonnet dgrisonnet changed the title WIP: test/extended: check apiserver request latency test/extended: check apiserver request latency Jan 12, 2022
@openshift-ci openshift-ci bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jan 12, 2022
@deads2k
Copy link
Contributor

deads2k commented Jan 12, 2022

/lgtm

thresholds can be reassessed based on actual data, but if they get looser that data should be based on runs from before the kube update.

@openshift-ci openshift-ci bot added lgtm Indicates that a PR is ready to be merged. approved Indicates a PR has been approved by an approver from all required OWNERS files. labels Jan 12, 2022
@dgrisonnet dgrisonnet force-pushed the apiserver-request-latency branch from de9d8d8 to b04e9e0 Compare January 14, 2022 16:14
@dgrisonnet
Copy link
Member Author

dgrisonnet commented Jan 14, 2022

There was a problem with the query, it should be good now

@deads2k
Copy link
Contributor

deads2k commented Jan 14, 2022

/lgtm

in slack we agreed to next steps: https://coreos.slack.com/archives/C02NZBANL3G/p1642169820339000?thread_ts=1642110108.326700&cid=C02NZBANL3G

David Eads 3 hours ago
so, breaking it into steps

  1. I'll merge pretty much whatever you want for 26739
  2. we need a perma-flaking test that always outputs the number we're looking for in some way that we can automatically parse
  3. we backport the PR from Add alpha API documentation #2 to 4.9
  4. in 4.10, we produce a cleaner, "write this data out" mechanism using the method I showed you for alerts.
  5. we write the uploader in ci-tools and write the data into bigquery
  6. we query it back out
    does that sound good?

David Eads 3 hours ago
your choice on 26739. At this point, something is better than nothing there

Damien Grisonnet 3 hours ago
yeah that's exactly what I was thinking

@openshift-ci openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label Jan 14, 2022
@openshift-ci
Copy link
Contributor

openshift-ci bot commented Jan 14, 2022

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: deads2k, dgrisonnet

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@dgrisonnet dgrisonnet mentioned this pull request Jan 17, 2022
Merged
@dgrisonnet dgrisonnet force-pushed the apiserver-request-latency branch from b04e9e0 to 135d207 Compare January 27, 2022 18:21
@openshift-ci openshift-ci bot removed the lgtm Indicates that a PR is ready to be merged. label Jan 27, 2022
@openshift-ci
Copy link
Contributor

openshift-ci bot commented Jan 27, 2022

New changes are detected. LGTM label has been removed.

@dgrisonnet dgrisonnet force-pushed the apiserver-request-latency branch from 135d207 to 5d88978 Compare January 27, 2022 18:22
@dgrisonnet
Copy link
Member Author

dgrisonnet commented Jan 28, 2022

/retest

@dgrisonnet dgrisonnet force-pushed the apiserver-request-latency branch from 5d88978 to 4599530 Compare January 28, 2022 14:12
@dgrisonnet dgrisonnet force-pushed the apiserver-request-latency branch from 4599530 to d67f738 Compare January 31, 2022 15:26
@dgrisonnet
Copy link
Member Author

dgrisonnet commented Feb 1, 2022

/retest

@dgrisonnet
Copy link
Member Author

dgrisonnet commented Mar 2, 2022

PTAL @deads2k

@openshift-bot
Copy link
Contributor

openshift-bot commented Aug 9, 2022

Issues go stale after 90d of inactivity.

Mark the issue as fresh by commenting /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
Exclude this issue from closing by commenting /lifecycle frozen.

If this issue is safe to close now please do so with /close.

/lifecycle stale

@openshift-ci openshift-ci bot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Aug 9, 2022
@openshift-bot
Copy link
Contributor

openshift-bot commented Oct 1, 2022

Stale issues rot after 30d of inactivity.

Mark the issue as fresh by commenting /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.
Exclude this issue from closing by commenting /lifecycle frozen.

If this issue is safe to close now please do so with /close.

/lifecycle rotten
/remove-lifecycle stale

@openshift-ci openshift-ci bot added lifecycle/rotten Denotes an issue or PR that has aged beyond stale and will be auto-closed. and removed lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. labels Oct 1, 2022
@openshift-ci
Copy link
Contributor

openshift-ci bot commented Nov 5, 2022

@dgrisonnet: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci/prow/e2e-aws-csi d67f738 link false /test e2e-aws-csi
ci/prow/e2e-gcp-upgrade d67f738 link true /test e2e-gcp-upgrade
ci/prow/e2e-gcp-ovn-upgrade d67f738 link true /test e2e-gcp-ovn-upgrade
ci/prow/e2e-aws-ovn-image-registry d67f738 link true /test e2e-aws-ovn-image-registry
ci/prow/e2e-gcp-ovn-image-ecosystem d67f738 link true /test e2e-gcp-ovn-image-ecosystem
ci/prow/e2e-gcp-ovn-builds d67f738 link true /test e2e-gcp-ovn-builds

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@openshift-bot
Copy link
Contributor

openshift-bot commented Dec 6, 2022

Rotten issues close after 30d of inactivity.

Reopen the issue by commenting /reopen.
Mark the issue as fresh by commenting /remove-lifecycle rotten.
Exclude this issue from closing again by commenting /lifecycle frozen.

/close

@openshift-merge-robot
Copy link
Contributor

openshift-merge-robot commented Dec 6, 2022

@dgrisonnet: PR needs rebase.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@openshift-merge-robot openshift-merge-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Dec 6, 2022
@openshift-ci openshift-ci bot closed this Dec 6, 2022
@openshift-ci
Copy link
Contributor

openshift-ci bot commented Dec 6, 2022

@openshift-bot: Closed this PR.

Details

In response to this:

Rotten issues close after 30d of inactivity.

Reopen the issue by commenting /reopen.
Mark the issue as fresh by commenting /remove-lifecycle rotten.
Exclude this issue from closing again by commenting /lifecycle frozen.

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@deads2k deads2k deads2k left review comments

@adambkaplan adambkaplan Awaiting requested review from adambkaplan

@gabemontero gabemontero Awaiting requested review from gabemontero

Assignees

@deads2k deads2k

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. lifecycle/rotten Denotes an issue or PR that has aged beyond stale and will be auto-closed. needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@dgrisonnet @deads2k @openshift-bot @openshift-merge-robot