metrics: gather cluster_installer series #189

crawford · 2019-06-26T23:03:04Z

No description provided.

crawford · 2019-06-26T23:10:28Z

squat · 2019-06-26T23:10:34Z

@crawford do you have a running cluster you tested this on? if so, could you also run:

make test/timeseries.txt
make generate

to get the new metric in the test/timeseries.txt file and also generate an updated docs/sample-metrics.md?

If you don't have one on hand, don't worry, we can do a follow up :)

crawford · 2019-06-26T23:14:07Z

@squat my generated timeseries data is significantly different than the last. Is that a concern?

squat · 2019-06-26T23:22:24Z

@crawford not an issue at all. It’s expected that pretty much every line should be different as timestamps and values will all change.

crawford · 2019-06-26T23:29:14Z

@squat updated. make generate didn't produce any new assets. I'm not sure how to get that doc updated.

squat · 2019-06-26T23:38:32Z

Sorry to complicate things. The makefile has recently changed. Plz try make generate -B or make generate-in-docker -B to force and build in a container.

crawford · 2019-06-26T23:49:01Z

@squat Okay, make generate -B did the trick. Is there an existing process for approving what gets whitelisted?

sichvoge · 2019-06-27T12:39:05Z

There is an internal approval chain for that. Currently, the monitoring team working on Telemeter and I approve new metrics that people like to push back. The engineering team makes sure that the metric is ok to send (specifically taking cardinality into account) and I just want to understand what the use cases are and if that aligns with Telemeters goal.

TD;LR: you need two LGTM: 1) telemeter engineer and 2) me :)

squat · 2019-06-27T12:41:02Z

/hold

From telemeter engineering team, this looks good to me. The new time series do not present a burden and are well within our current capacity.

squat · 2019-06-27T12:41:21Z

/lgtm

sichvoge · 2019-06-27T12:54:43Z

I am actually not really sure what the following means:

cluster_installer reports what installed the cluster, along with its version number and invoker.

What information are we sending off-cluster? Do you have an example?

squat · 2019-06-27T13:52:12Z

Ah, that should have been captured in the file I asked @crawford to generate, test/time-series.txt. But I see now that it's not there. @crawford the metric does not appear to be in Prometheus. Is the service exposing this metric not yet being scraped?

crawford · 2019-06-27T14:48:03Z

Hmm, let me take a closer look. I'll update the commit message with an example as well.

crawford · 2019-07-03T15:51:59Z

@squat Okay, this should be ready for review now.

crawford · 2019-07-05T15:37:16Z

/retest

crawford · 2019-07-08T15:09:52Z

/retest

smarterclayton · 2019-07-09T17:32:09Z

/approve

This metric is approved at a product level (high value in both triage and identification of unique issues)

squat

/lgtm

Spoke with Alex about this and looks good :) I’m glad we got this in

crawford · 2019-07-15T15:08:14Z

/hold cancel

Rather than hardcoding the path to bash (which is not always in /bin), use `env` to find it.

This series will allow us to determine whether a cluster was created using IPI or UPI. Once Hive and CI are updated, we'll also be able to determine if a cluster was created by CI or if it was created by Hive. The series captures the install type, the version of the tool, and its invoker (in addition to the standard labels): cluster_installer{ type="openshift-install", version="unreleased-master-1209-gfd08f44181f2111", invoker="alex", } 0 1562168623759

This new sample includes the new cluster_installer series.

crawford · 2019-07-15T16:27:05Z

Rebased onto master. @squat can you take one more look?

squat · 2019-07-15T16:31:44Z

ahh we had another conflict

squat · 2019-07-15T16:31:53Z

/lgtm
cmooooooooon

openshift-ci-robot · 2019-07-15T16:32:43Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: crawford, smarterclayton, squat

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

~~OWNERS~~ [smarterclayton,squat]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

smarterclayton · 2019-07-15T17:51:20Z

Does this also need to be added to the whitelist?

…

On Mon, Jul 15, 2019 at 1:48 PM OpenShift CI Robot ***@***.***> wrote: @crawford <https://github.com/crawford>: The following tests *failed*, say /retest to rerun them all: Test name Commit Details Rerun command ci/prow/e2e-aws-upgrade cb6690e <cb6690e> link <https://prow.svc.ci.openshift.org/view/gcs/origin-ci-test/pr-logs/pull/openshift_telemeter/189/pull-ci-openshift-telemeter-master-e2e-aws-upgrade/89> /test e2e-aws-upgrade ci/prow/benchmark cb6690e <cb6690e> link <https://prow.svc.ci.openshift.org/view/gcs/origin-ci-test/pr-logs/pull/openshift_telemeter/189/pull-ci-openshift-telemeter-master-benchmark/31> /test benchmark Full PR test history <https://prow.svc.ci.openshift.org/pr-history?org=openshift&repo=telemeter&pr=189>. Your PR dashboard <https://prow.svc.ci.openshift.org/pr?query=is:pr+state:open+author:crawford>. Please help us cut down on flakes by linking to <https://github.com/kubernetes/community/blob/master/contributors/devel/flaky-tests.md#filing-issues-for-flaky-tests> an open issue <https://github.com/openshift/telemeter/issues?q=is:issue+is:open> when you hit one in your PR. Instructions for interacting with me using PR comments are available here <https://git.k8s.io/community/contributors/guide/pull-requests.md>. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra <https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:> repository. I understand the commands that are listed here <https://go.k8s.io/bot-commands>. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#189?email_source=notifications&email_token=AAI37JZ6TFEV4IIQHBAJDRLP7SZ6XA5CNFSM4H3WXDK2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODZ6OPVQ#issuecomment-511502294>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAI37J4R4CGBXAI4MLKBA43P7SZ6XANCNFSM4H3WXDKQ> .

crawford · 2019-07-15T18:37:36Z

@smarterclayton yes it does. Where did the server manifests go?

squat · 2019-07-15T19:33:32Z

It looks like last week the telemeter-server manifests changed home to their new v2 location in github.com/observatorium/configuration. The important thing is to update the metrics.json file. This is the source of truth and will get picked up in the new v2 configuration repo (including the telemeter-server whitelist) the next time we bump.

However, it seems that the implications of this change were not completely considered. We’ll need to either revert or adjust the CI job to reflect this.

crawford · 2019-07-15T19:34:29Z

@squat cool. Do you know what's up with those two failed tests? Did I do something wrong in the PR or should I just kick them again?

squat · 2019-07-15T19:36:23Z

@crawford let me dig and take a looksie

crawford · 2019-07-16T02:26:04Z

/retest

crawford · 2019-08-05T16:55:30Z

/cherrypick release-4.1

openshift-cherrypick-robot · 2019-08-05T16:55:32Z

@crawford: #189 failed to apply on top of branch "release-4.1":

error: Failed to merge in the changes.
Using index info to reconstruct a base tree...
M	Makefile
Falling back to patching base and 3-way merge...
Auto-merging Makefile
CONFLICT (content): Merge conflict in Makefile
Patch failed at 0001 Makefile: improve portability of shell

Details

In response to this:

/cherrypick release-4.1

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

openshift-ci-robot requested review from paulfantom and s-urbaniak June 26, 2019 23:03

openshift-ci-robot added the size/S Denotes a PR that changes 10-29 lines, ignoring generated files. label Jun 26, 2019

crawford changed the title ~~Cluster installer~~ metrics: gather cluster_installer series Jun 26, 2019

crawford mentioned this pull request Jun 26, 2019

metrics: add cluster_installer series openshift/cluster-version-operator#213

Merged

openshift-ci-robot requested a review from smarterclayton June 26, 2019 23:10

openshift-ci-robot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/S Denotes a PR that changes 10-29 lines, ignoring generated files. labels Jun 26, 2019

openshift-ci-robot added size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. and removed size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Jun 26, 2019

openshift-ci-robot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Jun 27, 2019

openshift-ci-robot assigned squat Jun 27, 2019

openshift-ci-robot added lgtm Indicates that a PR is ready to be merged. approved Indicates a PR has been approved by an approver from all required OWNERS files. labels Jun 27, 2019

openshift-ci-robot removed the lgtm Indicates that a PR is ready to be merged. label Jul 3, 2019

squat approved these changes Jul 15, 2019

View reviewed changes

openshift-ci-robot added the lgtm Indicates that a PR is ready to be merged. label Jul 15, 2019

openshift-ci-robot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Jul 15, 2019

crawford added 3 commits July 15, 2019 09:24

Makefile: improve portability of shell

f4aef72

Rather than hardcoding the path to bash (which is not always in /bin), use `env` to find it.

test: update time series

cb6690e

This new sample includes the new cluster_installer series.

openshift-ci-robot removed the lgtm Indicates that a PR is ready to be merged. label Jul 15, 2019

openshift-ci-robot added the lgtm Indicates that a PR is ready to be merged. label Jul 15, 2019

openshift-merge-robot merged commit 0bacffc into openshift:master Jul 16, 2019

crawford deleted the cluster_installer branch July 16, 2019 16:46

crawford mentioned this pull request Aug 5, 2019

metrics: gather cluster_installer series #212

Merged

metrics: gather cluster_installer series #189

metrics: gather cluster_installer series #189

Uh oh!

Conversation

crawford commented Jun 26, 2019

Uh oh!

crawford commented Jun 26, 2019

Uh oh!

squat commented Jun 26, 2019

Uh oh!

crawford commented Jun 26, 2019

Uh oh!

squat commented Jun 26, 2019

Uh oh!

crawford commented Jun 26, 2019

Uh oh!

squat commented Jun 26, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

crawford commented Jun 26, 2019

Uh oh!

sichvoge commented Jun 27, 2019 • edited by squat Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

squat commented Jun 27, 2019

Uh oh!

squat commented Jun 27, 2019

Uh oh!

sichvoge commented Jun 27, 2019

Uh oh!

squat commented Jun 27, 2019

Uh oh!

crawford commented Jun 27, 2019

Uh oh!

crawford commented Jul 3, 2019

Uh oh!

crawford commented Jul 5, 2019

Uh oh!

crawford commented Jul 8, 2019

Uh oh!

smarterclayton commented Jul 9, 2019

Uh oh!

squat left a comment

Choose a reason for hiding this comment

Uh oh!

crawford commented Jul 15, 2019

Uh oh!

crawford commented Jul 15, 2019

Uh oh!

squat commented Jul 15, 2019

Uh oh!

squat commented Jul 15, 2019

Uh oh!

openshift-ci-robot commented Jul 15, 2019

Uh oh!

smarterclayton commented Jul 15, 2019 via email

Uh oh!

crawford commented Jul 15, 2019

Uh oh!

squat commented Jul 15, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

crawford commented Jul 15, 2019

Uh oh!

squat commented Jul 15, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

crawford commented Jul 16, 2019

Uh oh!

crawford commented Aug 5, 2019

Uh oh!

openshift-cherrypick-robot commented Aug 5, 2019

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

squat commented Jun 26, 2019 •

edited

Loading

sichvoge commented Jun 27, 2019 •

edited by squat

Loading

squat commented Jul 15, 2019 •

edited

Loading

squat commented Jul 15, 2019 •

edited

Loading