Cherry-pick #18802 to 7.x: Add memory metrics into compute googlecloud #18831

kaiyan-sheng · 2020-05-28T20:57:31Z

Cherry-pick of PR #18802 to 7.x branch. Original message:

This PR is to add 4 memory metrics(only for e2 family VMs) into googlecloud compute metricset and also added the missing network.sent_packets_count metric.

* Add memory metrics into compute googlecloud * update unit test and data.json for memory metrics (cherry picked from commit 1eb4e79)

elasticmachine · 2020-05-28T21:05:14Z

Pinging @elastic/integrations-platforms (Team:Platforms)

elasticmachine · 2020-05-28T21:13:27Z

💔 Build Failed

Expand to view the summary

Build stats

Build Cause: [Pull request #18831 updated]
Start Time: 2020-05-28T21:00:25.188+0000
Duration: 69 min 43 sec

Test stats 🧪

Test	Results
Failed	0
Passed	2115
Skipped	465
Total	2580

Steps errors

Expand to view the steps failures

Name: Mage goIntegTest
- Description: mage goIntegTest
- Duration: 1 min 57 sec
- Start Time: 2020-05-28T21:34:26.538+0000
- log

Log output

Expand to view the last 100 lines of log output

[2020-05-28T22:08:43.778Z] 		at hudson.FilePath.validateAntFileMask(FilePath.java:2684)
[2020-05-28T22:08:43.778Z] 		at hudson.tasks.ArtifactArchiver.perform(ArtifactArchiver.java:265)
[2020-05-28T22:08:43.778Z] 		at org.jenkinsci.plugins.workflow.steps.CoreStep$Execution.run(CoreStep.java:80)
[2020-05-28T22:08:43.778Z] 		at org.jenkinsci.plugins.workflow.steps.CoreStep$Execution.run(CoreStep.java:67)
[2020-05-28T22:08:43.778Z] 		at org.jenkinsci.plugins.workflow.steps.SynchronousNonBlockingStepExecution.lambda$start$0(SynchronousNonBlockingStepExecution.java:47)
[2020-05-28T22:08:43.778Z] 		at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[2020-05-28T22:08:43.778Z] 		at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[2020-05-28T22:08:43.778Z] 		at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[2020-05-28T22:08:43.778Z] 		at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[2020-05-28T22:08:43.778Z] Caused: hudson.FilePath$TunneledInterruptedException
[2020-05-28T22:08:43.778Z] 	at hudson.FilePath$FileCallableWrapper.call(FilePath.java:3075)
[2020-05-28T22:08:43.778Z] 	at hudson.remoting.UserRequest.perform(UserRequest.java:212)
[2020-05-28T22:08:43.778Z] 	at hudson.remoting.UserRequest.perform(UserRequest.java:54)
[2020-05-28T22:08:43.778Z] 	at hudson.remoting.Request$2.run(Request.java:369)
[2020-05-28T22:08:43.778Z] 	at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
[2020-05-28T22:08:43.778Z] 	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[2020-05-28T22:08:43.778Z] 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[2020-05-28T22:08:43.778Z] 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[2020-05-28T22:08:43.778Z] 	at hudson.remoting.Engine$1.lambda$newThread$0(Engine.java:93)
[2020-05-28T22:08:43.778Z] Caused: java.lang.InterruptedException: java.lang.InterruptedException: no matches found within 10000
[2020-05-28T22:08:43.778Z] 	at hudson.FilePath.act(FilePath.java:1071)
[2020-05-28T22:08:43.778Z] 	at hudson.FilePath.act(FilePath.java:1058)
[2020-05-28T22:08:43.778Z] 	at hudson.FilePath.validateAntFileMask(FilePath.java:2684)
[2020-05-28T22:08:43.778Z] 	at hudson.tasks.ArtifactArchiver.perform(ArtifactArchiver.java:265)
[2020-05-28T22:08:43.778Z] 	at org.jenkinsci.plugins.workflow.steps.CoreStep$Execution.run(CoreStep.java:80)
[2020-05-28T22:08:43.778Z] 	at org.jenkinsci.plugins.workflow.steps.CoreStep$Execution.run(CoreStep.java:67)
[2020-05-28T22:08:43.778Z] 	at org.jenkinsci.plugins.workflow.steps.SynchronousNonBlockingStepExecution.lambda$start$0(SynchronousNonBlockingStepExecution.java:47)
[2020-05-28T22:08:43.778Z] 	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[2020-05-28T22:08:43.778Z] 	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[2020-05-28T22:08:43.778Z] 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[2020-05-28T22:08:43.778Z] 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[2020-05-28T22:08:43.778Z] 	at java.lang.Thread.run(Thread.java:748)
[2020-05-28T22:08:43.778Z] No artifacts found that match the file pattern "**/build/TEST*.out". Configuration error?
[2020-05-28T22:08:44.136Z] + curl -sSLo codecov https://codecov.io/bash
[2020-05-28T22:08:44.398Z] + FILE=auditbeat/build/coverage/full.cov
[2020-05-28T22:08:44.398Z] + [ -f auditbeat/build/coverage/full.cov ]
[2020-05-28T22:08:44.398Z] + FILE=filebeat/build/coverage/full.cov
[2020-05-28T22:08:44.398Z] + [ -f filebeat/build/coverage/full.cov ]
[2020-05-28T22:08:44.398Z] + FILE=heartbeat/build/coverage/full.cov
[2020-05-28T22:08:44.398Z] + [ -f heartbeat/build/coverage/full.cov ]
[2020-05-28T22:08:44.398Z] + FILE=libbeat/build/coverage/full.cov
[2020-05-28T22:08:44.398Z] + [ -f libbeat/build/coverage/full.cov ]
[2020-05-28T22:08:44.398Z] + FILE=metricbeat/build/coverage/full.cov
[2020-05-28T22:08:44.398Z] + [ -f metricbeat/build/coverage/full.cov ]
[2020-05-28T22:08:44.398Z] + FILE=packetbeat/build/coverage/full.cov
[2020-05-28T22:08:44.398Z] + [ -f packetbeat/build/coverage/full.cov ]
[2020-05-28T22:08:44.398Z] + FILE=winlogbeat/build/coverage/full.cov
[2020-05-28T22:08:44.398Z] + [ -f winlogbeat/build/coverage/full.cov ]
[2020-05-28T22:08:44.398Z] + FILE=journalbeat/build/coverage/full.cov
[2020-05-28T22:08:44.398Z] + [ -f journalbeat/build/coverage/full.cov ]
[2020-05-28T22:08:45.891Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18831/src/github.com/elastic/beats
[2020-05-28T22:08:46.213Z] + find . -type f -name TEST*.xml -path */build/* -delete
[2020-05-28T22:08:46.228Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18831/src/github.com/elastic/beats/Lint
[2020-05-28T22:08:46.309Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18831/src/github.com/elastic/beats/Metricbeat-OSS-Integration-tests
[2020-05-28T22:08:46.387Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18831/src/github.com/elastic/beats/Metricbeat-crosscompile
[2020-05-28T22:08:46.488Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18831/src/github.com/elastic/beats/Metricbeat-OSS-Unit-tests
[2020-05-28T22:08:46.581Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18831/src/github.com/elastic/beats/Metricbeat-Windows
[2020-05-28T22:08:46.666Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18831/src/github.com/elastic/beats/Metricbeat-x-pack-Windows
[2020-05-28T22:08:46.746Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18831/src/github.com/elastic/beats/Metricbeat-x-pack
[2020-05-28T22:08:46.828Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18831/src/github.com/elastic/beats/Metricbeat-Python-integration-tests
[2020-05-28T22:08:47.201Z] + cat
[2020-05-28T22:08:47.201Z] + /usr/local/bin/runbld ./runbld-script
[2020-05-28T22:08:47.201Z] Picked up JAVA_TOOL_OPTIONS: -Dfile.encoding=UTF8
[2020-05-28T22:08:53.841Z] runbld>>> runbld started
[2020-05-28T22:08:53.841Z] runbld>>> 1.6.11/a66728ff8f4356963772e6e6d2069392fa06acbe
[2020-05-28T22:08:55.767Z] runbld>>> The following profiles matched the job 'Beats/beats-beats-mbp/PR-18831' in order of occurrence in the config (last value wins).
[2020-05-28T22:08:57.152Z] runbld>>> Debug logging enabled.
[2020-05-28T22:08:57.152Z] runbld>>> Storing result
[2020-05-28T22:08:57.152Z] runbld>>> Store result: created {:total 2, :successful 2, :failed 0} 1
[2020-05-28T22:08:57.152Z] runbld>>> BUILD: https://c150076387b5421f9154dfbf536e5c60.us-west1.gcp.cloud.es.io:9243/build-1587637540455/t/20200528220856-8074880C
[2020-05-28T22:08:57.152Z] runbld>>> Adding system facts.
[2020-05-28T22:08:58.098Z] runbld>>> Adding vcs info for the latest commit:  7c1a5f67f7238dbf66692c006806bd74a7d1c953
[2020-05-28T22:08:58.098Z] runbld>>> >>>>>>>>>>>> SCRIPT EXECUTION BEGIN >>>>>>>>>>>>
[2020-05-28T22:08:58.098Z] runbld>>> Adding /usr/lib/jvm/java-8-openjdk-amd64/bin to the path.
[2020-05-28T22:08:58.360Z] + echo 'Processing JUnit reports with runbld...'
[2020-05-28T22:08:58.360Z] Processing JUnit reports with runbld...
[2020-05-28T22:08:58.621Z] runbld>>> <<<<<<<<<<<< SCRIPT EXECUTION END <<<<<<<<<<<<
[2020-05-28T22:08:58.621Z] runbld>>> DURATION: 19ms
[2020-05-28T22:08:58.621Z] runbld>>> STDOUT: 40 bytes
[2020-05-28T22:08:58.621Z] runbld>>> STDERR: 49 bytes
[2020-05-28T22:08:58.621Z] runbld>>> WRAPPED PROCESS: SUCCESS (0)
[2020-05-28T22:08:58.621Z] runbld>>> Searching for build metadata in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18831/src/github.com/elastic/beats
[2020-05-28T22:09:00.006Z] runbld>>> Storing build metadata: 
[2020-05-28T22:09:00.006Z] runbld>>> Adding test report.
[2020-05-28T22:09:00.006Z] runbld>>> Searching for junit test output files with the pattern: TEST-.*\.xml$ in: /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18831/src/github.com/elastic/beats
[2020-05-28T22:09:00.953Z] runbld>>> Found 11 test output files
[2020-05-28T22:09:01.904Z] runbld>>> Test output logs contained: Errors: 0 Failures: 0 Tests: 2580 Skipped: 413
[2020-05-28T22:09:01.904Z] runbld>>> Storing result
[2020-05-28T22:09:01.904Z] runbld>>> FAILURES: 0
[2020-05-28T22:09:02.166Z] runbld>>> Store result: updated {:total 2, :successful 2, :failed 0} 2
[2020-05-28T22:09:02.167Z] runbld>>> BUILD: https://c150076387b5421f9154dfbf536e5c60.us-west1.gcp.cloud.es.io:9243/build-1587637540455/t/20200528220856-8074880C
[2020-05-28T22:09:02.167Z] runbld>>> Email notification disabled by environment variable.
[2020-05-28T22:09:02.167Z] runbld>>> Slack notification disabled by environment variable.
[2020-05-28T22:09:07.683Z] Running on Jenkins in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18831
[2020-05-28T22:09:07.783Z] [INFO] getVaultSecret: Getting secrets
[2020-05-28T22:09:07.824Z] Masking supported pattern matches of $VAULT_ADDR or $VAULT_ROLE_ID or $VAULT_SECRET_ID
[2020-05-28T22:09:08.643Z] + chmod 755 generate-build-data.sh
[2020-05-28T22:09:08.643Z] + ./generate-build-data.sh https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats-beats-mbp/PR-18831/ https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats-beats-mbp/PR-18831/runs/2 FAILURE 4123195
[2020-05-28T22:09:08.643Z] INFO: curl https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats-beats-mbp/PR-18831/runs/2/steps/?limit=10000 -o steps-info.json
[2020-05-28T22:09:09.194Z] INFO: curl https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats-beats-mbp/PR-18831/runs/2/tests/?status=FAILED -o tests-errors.json

Add memory metrics into compute googlecloud (#18802)

86e11a9

* Add memory metrics into compute googlecloud * update unit test and data.json for memory metrics (cherry picked from commit 1eb4e79)

kaiyan-sheng added backport review labels May 28, 2020

botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label May 28, 2020

update changelog

7c1a5f6

kaiyan-sheng self-assigned this May 28, 2020

kaiyan-sheng added the Team:Platforms Label for the Integrations - Platforms team label May 28, 2020

botelastic bot removed the needs_team Indicates that the issue/PR needs a Team:* label label May 28, 2020

ChrsMark approved these changes May 29, 2020

View reviewed changes

kaiyan-sheng merged commit c9a6f9a into elastic:7.x May 29, 2020

kaiyan-sheng deleted the backport_18802_7.x branch May 29, 2020 13:18

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Cherry-pick #18802 to 7.x: Add memory metrics into compute googlecloud #18831

Cherry-pick #18802 to 7.x: Add memory metrics into compute googlecloud #18831

Uh oh!

kaiyan-sheng commented May 28, 2020

Uh oh!

elasticmachine commented May 28, 2020

Uh oh!

elasticmachine commented May 28, 2020 •

edited

Loading

Build stats

Test stats 🧪

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Cherry-pick #18802 to 7.x: Add memory metrics into compute googlecloud #18831

Cherry-pick #18802 to 7.x: Add memory metrics into compute googlecloud #18831

Uh oh!

Conversation

kaiyan-sheng commented May 28, 2020

Uh oh!

elasticmachine commented May 28, 2020

Uh oh!

elasticmachine commented May 28, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

💔 Build Failed

Build stats

Test stats 🧪

Steps errors

Log output

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

elasticmachine commented May 28, 2020 •

edited

Loading