Sanitize Telemetry data #4758

valaparthvi · 2021-05-31T12:12:21Z

What type of PR is this?
/kind bug

What does this PR do / why we need it:
This PR sanitizes some more data that might have some potential PII data.

Which issue(s) this PR fixes:

Fixes part of #4462

PR acceptance criteria:

How to test changes / Special notes to the reviewer:
Running unit tests should be fine or perhaps test with data that potentially contains personal data.

girishramnani · 2021-06-02T07:23:25Z

pkg/segment/segment.go

+
+// sanitizeURL sanitizes URLs from the error string
+func sanitizeURL(errString string) string {
+	urlPattern, _ := regexp.Compile(`(http)\S*`)


Should we consider a regex that matches a URL that might not have prefix "http" as well?
https://stackoverflow.com/questions/42618872/regex-for-website-or-url-validation/42619368

valaparthvi · 2021-06-07T14:03:58Z

test /psi-unit-test-windows

[ssh:Windows 10]                             
INFO[2021-06-07T10:06:23Z]                                              
INFO[2021-06-07T10:06:23Z] {"component":"entrypoint","file":"prow/entrypoint/run.go:165","func":"k8s.io/test-infra/prow/entrypoint.Options.ExecuteProcess","level":"error","msg":"Process did not finish before 2h0m0s timeout","severity":"error","time":"2021-06-07T10:06:03Z"} 
INFO[2021-06-07T10:06:23Z] {"component":"entrypoint","file":"prow/entrypoint/run.go:255","func":"k8s.io/test-infra/prow/entrypoint.gracefullyTerminate","level":"error","msg":"Process did not exit before 15s grace period","severity":"error","time":"2021-06-07T10:06:18Z"} 
INFO[2021-06-07T10:06:23Z] {"component":"entrypoint","error":"process timed out","file":"prow/entrypoint/run.go:80","func":"k8s.io/test-infra/prow/entrypoint.Options.Run","level":"error","msg":"Error executing test process","severity":"error","time":"2021-06-07T10:06:18Z"} 
INFO[2021-06-07T10:06:23Z] error: failed to execute wrapped command: exit status 127

valaparthvi · 2021-06-07T14:13:02Z

/test psi-kubernetes-integration-e2e
/test psi-unit-test-windows

 ERRO[2021-06-07T08:06:11Z] [ssh:Fedora-32-minikube] !!!run failed, skipping!!! 
ERRO[2021-06-07T08:06:11Z] 2021/06/07 08:06:06 : failed due to err Failed the test, see logs above ^ 
ERRO[2021-06-07T08:06:11Z] {"component":"entrypoint","error":"wrapped process failed: exit status 1","file":"prow/entrypoint/run.go:80","func":"k8s.io/test-infra/prow/entrypoint.Options.Run","level":"error","msg":"Error executing test process","severity":"error","time":"2021-06-07T08:06:06Z"} 
ERRO[2021-06-07T08:06:11Z] error: failed to execute wrapped command: exit status 1

valaparthvi · 2021-06-07T14:14:39Z

/test psi-kubernetes-integration-e2e

valaparthvi · 2021-06-07T14:14:55Z

/test psi-unit-test-windows

valaparthvi · 2021-06-08T07:28:30Z

/retest

psi-kubernetes-integration-e2e - the make job apparently causing the below error has been disabled. Retesting should make the failure go away.

++ make goget-ginkgo 
INFO[2021-06-07T14:18:09Z] [ssh:Fedora-32-minikube] verifying github.com/mikefarah/yaml/[email protected]/go.mod: checksum mismatch 
INFO[2021-06-07T14:18:09Z]                                              
INFO[2021-06-07T14:18:09Z] [ssh:Fedora-32-minikube] 	downloaded: h1:RDklbk79AGWmwhnvt/jBztapEOGDOx6ZbXqjP6csGnQ= 
INFO[2021-06-07T14:18:09Z] [ssh:Fedora-32-minikube] 	go.sum:     h1:ahVqZF4n1W4NqwvVnZzC4es67xsW9uR/RRf2RRxieJU= 
INFO[2021-06-07T14:18:09Z] [ssh:Fedora-32-minikube]                     
INFO[2021-06-07T14:18:09Z] [ssh:Fedora-32-minikube] SECURITY ERROR      
INFO[2021-06-07T14:18:09Z] [ssh:Fedora-32-minikube] This download does NOT match an earlier download recorded in go.sum. 
INFO[2021-06-07T14:18:09Z] [ssh:Fedora-32-minikube] The bits may have been replaced on the origin server, or an attacker may 
INFO[2021-06-07T14:18:09Z] [ssh:Fedora-32-minikube] have intercepted the download attempt. 
INFO[2021-06-07T14:18:09Z] [ssh:Fedora-32-minikube]                     
INFO[2021-06-07T14:18:09Z] [ssh:Fedora-32-minikube] For more information, see 'go help module-auth'. 
INFO[2021-06-07T14:18:09Z] [ssh:Fedora-32-minikube] verifying github.com/mikefarah/yaml/[email protected]/go.mod: checksum mismatch 
INFO[2021-06-07T14:18:09Z]                                              
INFO[2021-06-07T14:18:09Z] [ssh:Fedora-32-minikube] 	downloaded: h1:RDklbk79AGWmwhnvt/jBztapEOGDOx6ZbXqjP6csGnQ= 
INFO[2021-06-07T14:18:09Z] [ssh:Fedora-32-minikube] 	go.sum:     h1:ahVqZF4n1W4NqwvVnZzC4es67xsW9uR/RRf2RRxieJU=

psi-unit-test-windows

 INFO[2021-06-07T16:17:59Z] ++ tar -xzf ci-firewall-linux-amd64.tar.gz   
INFO[2021-06-07T16:17:59Z] ++ ./ci-firewall request --mainbranch main --sendqueue amqp.ci.queue.win.unit.send --sendtopic amqp.ci.topic.win.unit.send --sendexchange amqp.ci.exchange.win.unit.send --setupscript scripts/setup_script_unit.sh --jenkinsproject odo-windows-unit-pr-build --runscript scripts/run_script_unit.sh --timeout 4h00m 
INFO[2021-06-07T16:17:59Z] {"component":"entrypoint","file":"prow/entrypoint/run.go:165","func":"k8s.io/test-infra/prow/entrypoint.Options.ExecuteProcess","level":"error","msg":"Process did not finish before 2h0m0s timeout","severity":"error","time":"2021-06-07T16:17:40Z"} 
INFO[2021-06-07T16:17:59Z] {"component":"entrypoint","file":"prow/entrypoint/run.go:255","func":"k8s.io/test-infra/prow/entrypoint.gracefullyTerminate","level":"error","msg":"Process did not exit before 15s grace period","severity":"error","time":"2021-06-07T16:17:55Z"} 
INFO[2021-06-07T16:17:59Z] {"component":"entrypoint","error":"process timed out","file":"prow/entrypoint/run.go:80","func":"k8s.io/test-infra/prow/entrypoint.Options.Run","level":"error","msg":"Error executing test process","severity":"error","time":"2021-06-07T16:17:55Z"}

girishramnani · 2021-06-08T07:55:53Z

pkg/segment/segment.go

+func sanitizeUserInfo(errString string) string {
+	user1, err1 := user.Current()
+	if err1 != nil {
+		return errors.Wrapf(err1, err1.Error()).Error()


I would suggest if we cannot get the user, we just return the error as is?
this function is about santising the userInfo and if we cannot get the user in my opinion we shouldn't add that to the error because lets say there is an error getting the current user you would send an error like "X component not found: cannot resolve user".

Right of course. It doesn't really make sense to wrap the error like that either, it isn't really doing anything!

girishramnani · 2021-06-08T08:42:46Z

/approve
/lgtm

openshift-ci · 2021-06-08T08:42:55Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: girishramnani

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [girishramnani]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

valaparthvi · 2021-06-08T09:47:54Z

/override psi-kubernetes-integration-e2e
Reason - #4776

openshift-ci · 2021-06-08T09:47:55Z

@valaparthvi: /override requires a failed status context to operate on.
The following unknown contexts were given:

psi-kubernetes-integration-e2e

Only the following contexts were expected:

ci/prow/psi-kubernetes-integration-e2e
ci/prow/psi-unit-test-mac
ci/prow/psi-unit-test-windows
ci/prow/unit
ci/prow/v4.7-integration-e2e
tide

In response to this:

/override psi-kubernetes-integration-e2e
Reason - #4776

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

valaparthvi · 2021-06-08T09:49:26Z

/override ci/prow/psi-kubernetes-integration-e2e
Reason - #4776

openshift-ci · 2021-06-08T09:49:29Z

@valaparthvi: Overrode contexts on behalf of valaparthvi: ci/prow/psi-kubernetes-integration-e2e

In response to this:

/override ci/prow/psi-kubernetes-integration-e2e
Reason - #4776

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

openshift-bot · 2021-06-08T18:21:14Z

/retest

Please review the full test history for this PR and help us cut down flakes.

openshift-bot · 2021-06-08T19:33:14Z

/retest

Please review the full test history for this PR and help us cut down flakes.

valaparthvi requested a review from kadel May 31, 2021 12:12

openshift-ci bot added the kind/bug Categorizes issue or PR as related to a bug. label May 31, 2021

openshift-ci bot requested review from dharmit and mohammedzee1000 May 31, 2021 12:12

openshift-ci bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. Required by Prow. label May 31, 2021

valaparthvi force-pushed the sanitization branch 2 times, most recently from e85f5e8 to 4a4963c Compare June 1, 2021 06:28

girishramnani reviewed Jun 2, 2021

View reviewed changes

valaparthvi mentioned this pull request Jun 3, 2021

Track telemetry implementation for odo #4462

Closed

17 tasks

Sanitize Telemetry data

e3b2269

valaparthvi force-pushed the sanitization branch from 4a4963c to f2ac21f Compare June 7, 2021 07:56

openshift-ci bot removed the approved Indicates a PR has been approved by an approver from all required OWNERS files. Required by Prow. label Jun 7, 2021

valaparthvi force-pushed the sanitization branch from f2ac21f to fef00f4 Compare June 7, 2021 08:03

Add more sanitization for IPv4 addresses and hostnames, add tests

04a0467

valaparthvi force-pushed the sanitization branch from fef00f4 to 04a0467 Compare June 8, 2021 07:34

girishramnani reviewed Jun 8, 2021

View reviewed changes

Add requested changes

4fd10a2

openshift-ci bot assigned girishramnani Jun 8, 2021

openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. Required by Prow. label Jun 8, 2021

openshift-ci bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. Required by Prow. label Jun 8, 2021

openshift-merge-robot merged commit ca5ebd6 into redhat-developer:main Jun 8, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Sanitize Telemetry data #4758

Sanitize Telemetry data #4758

valaparthvi commented May 31, 2021

girishramnani Jun 2, 2021

valaparthvi commented Jun 7, 2021

valaparthvi commented Jun 7, 2021 •

edited

Loading

valaparthvi commented Jun 7, 2021

valaparthvi commented Jun 7, 2021

valaparthvi commented Jun 8, 2021

girishramnani Jun 8, 2021

valaparthvi Jun 8, 2021

valaparthvi Jun 8, 2021

girishramnani Jun 8, 2021

girishramnani commented Jun 8, 2021

openshift-ci bot commented Jun 8, 2021

valaparthvi commented Jun 8, 2021

openshift-ci bot commented Jun 8, 2021

valaparthvi commented Jun 8, 2021

openshift-ci bot commented Jun 8, 2021

openshift-bot commented Jun 8, 2021

openshift-bot commented Jun 8, 2021

Sanitize Telemetry data #4758

Sanitize Telemetry data #4758

Conversation

valaparthvi commented May 31, 2021

girishramnani Jun 2, 2021

Choose a reason for hiding this comment

valaparthvi commented Jun 7, 2021

valaparthvi commented Jun 7, 2021 • edited Loading

valaparthvi commented Jun 7, 2021

valaparthvi commented Jun 7, 2021

valaparthvi commented Jun 8, 2021

girishramnani Jun 8, 2021

Choose a reason for hiding this comment

valaparthvi Jun 8, 2021

Choose a reason for hiding this comment

valaparthvi Jun 8, 2021

Choose a reason for hiding this comment

girishramnani Jun 8, 2021

Choose a reason for hiding this comment

girishramnani commented Jun 8, 2021

openshift-ci bot commented Jun 8, 2021

valaparthvi commented Jun 8, 2021

openshift-ci bot commented Jun 8, 2021

valaparthvi commented Jun 8, 2021

openshift-ci bot commented Jun 8, 2021

openshift-bot commented Jun 8, 2021

openshift-bot commented Jun 8, 2021

valaparthvi commented Jun 7, 2021 •

edited

Loading