Add data.json files for vsphere module. WIP #4209

amandahla · 2017-05-04T13:14:14Z

Related to #4028

Todo:

Data.json for metric Host
Data.json for metric Datastore
Data.json for metric Virtualmachine
Dashboard

elasticmachine · 2017-05-04T13:17:02Z

Jenkins standing by to test this. If you aren't a maintainer, you can ignore this comment. Someone with commit access, please review this and clear it for Jenkins to run.

elasticmachine · 2017-05-04T13:17:07Z

Jenkins standing by to test this. If you aren't a maintainer, you can ignore this comment. Someone with commit access, please review this and clear it for Jenkins to run.

ruflin · 2017-05-04T14:29:49Z

It would be nice if they would be auto generated with generate-json. If you have a local setup and the data tests, you could run go test -tags=integration github.com/elastic/beats/metricbeat/module/vsphere/... -data

amandahla · 2017-05-04T18:28:21Z

@ruflin I'm not sure if it's right but I included TestData on the tests so now the data.json file is generated based on simulator data.

ruflin · 2017-05-04T18:39:31Z

@amandahla That actually sounds great.

ruflin · 2017-05-04T19:37:00Z

jenkins, test it

amandahla · 2017-05-04T19:57:10Z

@ruflin I made a Kibana Dashboard for this module with a few visualizations. Can I commit here or open another PR? Thanks!

ruflin · 2017-05-04T20:48:52Z

Nice. Lets open a new PR so we can merge this one sooner.

andrewkroh · 2017-05-04T22:08:33Z

Hi @amandahla, in master, I'm seeing some test failures. Maybe they are already fixed by this PR, if not could you please take a look. Let me know if you need any more info.

From jenkins on linux:

command [go test -cover -coverprofile /tmp/gotestcover-647607641 github.com/elastic/beats/metricbeat/module/vsphere/datastore]: exit status 1
--- FAIL: TestFetchEventContents (0.01s)
	datastore_test.go:45: vsphere/datastore event: {
		  "capacity": {
		    "free": {
		      "bytes": 480915800064
		    },
		    "total": {
		      "bytes": 506726043648
		    },
		    "used": {
		      "bytes": 25810243584,
		      "pct": 5
		    }
		  },
		  "datacenter": "ha-datacenter",
		  "fstype": "local",
		  "name": "LocalDS_0"
		}
	assertions.go:225: 
                        
	Error Trace:	datastore_test.go:62
		
	Error:		Should be true
		
FAIL

From a windows vm:

PS C:\Gopath\src\github.com\elastic\beats\metricbeat\module\vsphere\datastore> go test -v                                                                                        
=== RUN   TestFetchEventContents                                                                                                                                                 
--- FAIL: TestFetchEventContents (0.01s)                                                                                                                                         
panic: runtime error: integer divide by zero [recovered]                                                                                                                         
        panic: runtime error: integer divide by zero                                                                                                                             
                                                                                                                                                                                 
goroutine 22 [running]:                                                                                                                                                          
panic(0xd03e00, 0xc04200a050)                                                                                                                                                    
        C:/go1.7.3/src/runtime/panic.go:500 +0x1af                                                                                                                               
testing.tRunner.func1(0xc042074180)                                                                                                                                              
        C:/go1.7.3/src/testing/testing.go:579 +0x264                                                                                                                             
panic(0xd03e00, 0xc04200a050)                                                                                                                                                    
        C:/go1.7.3/src/runtime/panic.go:458 +0x251                                                                                                                               
github.com/elastic/beats/metricbeat/module/vsphere/datastore.(*MetricSet).Fetch(0xc0422359e0, 0x0, 0x0, 0x0, 0x0, 0x0)                                                           
        C:/Gopath/src/github.com/elastic/beats/metricbeat/module/vsphere/datastore/datastore.go:107 +0x1235                                                                      
github.com/elastic/beats/metricbeat/module/vsphere/datastore.TestFetchEventContents(0xc042074180)                                                                                
        C:/Gopath/src/github.com/elastic/beats/metricbeat/module/vsphere/datastore/datastore_test.go:38 +0x71a                                                                   
testing.tRunner(0xc042074180, 0xf6bec0)                                                                                                                                          
        C:/go1.7.3/src/testing/testing.go:610 +0x88                                                                                                                              
created by testing.(*T).Run                                                                                                                                                      
        C:/go1.7.3/src/testing/testing.go:646 +0x2f3                                                                                                                             
exit status 2                                                                                                                                                                    
FAIL    github.com/elastic/beats/metricbeat/module/vsphere/datastore    0.106s

andrewkroh · 2017-05-05T05:39:40Z

I opened #4220 for the above issue.

ruflin · 2017-05-05T08:43:15Z

@amandahla Could you squash your commits into 1 commit and rebase on master? Also run make fmt and make update to make sure the build passes.

ruflin · 2017-05-05T19:52:17Z

jenkins, test it

elasticmachine · 2017-05-05T20:24:58Z

metricbeat/module/vsphere/datastore/datastore.go

 	f := find.NewFinder(m.Client, true)
+	if f == nil {
+		return nil, errors.New("Finder undefined for vsphere.")


[golint] _{reported by reviewdog 🐶}
error strings should not be capitalized or end with punctuation or a newline

ruflin · 2017-05-09T07:57:28Z

@amandahla PR looks good to me. Could you rebase again on master and fix the golint issue above?

ruflin · 2017-05-10T06:24:36Z

jenkins, test it

andrewkroh

There seems to be a merge problem. The fixes from https://github.com/elastic/beats/pull/4220/files are lost.

andrewkroh · 2017-05-10T13:29:19Z

metricbeat/module/vsphere/host/host.go

@@ -62,7 +62,7 @@ func New(base mb.BaseMetricSet) (mb.MetricSet, error) {
 func (m *MetricSet) Fetch() ([]common.MapStr, error) {
 	f := find.NewFinder(m.Client, true)
 	if f == nil {


Unfortunately find.NewFinder is lacking godocs to document the function's contract. But based on the implementation, the function cannot return nil so this check is unnecessary.

@andrewkroh Sorry, maybe my rebase went wrong. I'll correct this and remove this check. In another PR, I'll see if it's possible to use the new package view instead of finder (like here)

ruflin

There still seem to be some rebase issues. Did you rebase on top of the most recent version of master? Things seem to be in the diff which shouldn't. Not sure what happened.

ruflin · 2017-05-11T08:38:12Z

metricbeat/module/vsphere/host/host.go

@@ -29,7 +29,7 @@ type MetricSet struct {
 }

 func New(base mb.BaseMetricSet) (mb.MetricSet, error) {
-	logp.Experimental("The vsphere host metricset is experimental")
+	logp.Warn("EXPERIMENTAL: The vsphere host metricset is experimental")


This change should not be reverted.

Sorry. I tried to make the same as here so I change this warning.
https://github.com/elastic/beats/pull/4220/files

ruflin · 2017-05-11T08:38:38Z

metricbeat/module/vsphere/virtualmachine/virtualmachine.go

@@ -30,7 +30,7 @@ type MetricSet struct {
 }

 func New(base mb.BaseMetricSet) (mb.MetricSet, error) {
-	logp.Experimental("The vsphere virtualmachine metricset is experimental")
+	logp.Warn("EXPERIMENTAL: The vsphere virtualmachine metricset is experimental")


This change should not be reverted.

andrewkroh · 2017-05-11T13:16:39Z

jenkins, test it

monicasarbu · 2017-05-15T09:45:11Z

@amandahla Thank you for your PR. It would be nice if you could a Kibana dashboard as well. Thanks!

amandahla changed the title ~~Add data.json files for vsphere module~~ Add data.json files for vsphere module #WIP May 4, 2017

amandahla changed the title ~~Add data.json files for vsphere module #WIP~~ Add data.json files for vsphere module WIP May 4, 2017

amandahla changed the title ~~Add data.json files for vsphere module WIP~~ Add data.json files for vsphere module. WIP May 4, 2017

elasticmachine reviewed May 5, 2017

View reviewed changes

andrewkroh added in progress Pull request is currently in progress. Metricbeat Metricbeat labels May 9, 2017

andrewkroh requested changes May 10, 2017

View reviewed changes

andrewkroh reviewed May 10, 2017

View reviewed changes

andrewkroh added review and removed in progress Pull request is currently in progress. labels May 10, 2017

ruflin reviewed May 11, 2017

View reviewed changes

Add Vsphere module with datastore, hosts and virtualmachine metrics.

5430530

andrewkroh approved these changes May 11, 2017

View reviewed changes

andrewkroh merged commit 6852499 into elastic:master May 11, 2017

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add data.json files for vsphere module. WIP #4209

Add data.json files for vsphere module. WIP #4209

amandahla commented May 4, 2017 •

edited

Loading

elasticmachine commented May 4, 2017

elasticmachine commented May 4, 2017

ruflin commented May 4, 2017

amandahla commented May 4, 2017

ruflin commented May 4, 2017

ruflin commented May 4, 2017

amandahla commented May 4, 2017

ruflin commented May 4, 2017

andrewkroh commented May 4, 2017

andrewkroh commented May 5, 2017

ruflin commented May 5, 2017

ruflin commented May 5, 2017

elasticmachine May 5, 2017

ruflin commented May 9, 2017

ruflin commented May 10, 2017

andrewkroh left a comment

andrewkroh May 10, 2017

amandahla May 10, 2017

ruflin left a comment

ruflin May 11, 2017

amandahla May 11, 2017

ruflin May 11, 2017

andrewkroh commented May 11, 2017

monicasarbu commented May 15, 2017

Add data.json files for vsphere module. WIP #4209

Add data.json files for vsphere module. WIP #4209

Conversation

amandahla commented May 4, 2017 • edited Loading

elasticmachine commented May 4, 2017

elasticmachine commented May 4, 2017

ruflin commented May 4, 2017

amandahla commented May 4, 2017

ruflin commented May 4, 2017

ruflin commented May 4, 2017

amandahla commented May 4, 2017

ruflin commented May 4, 2017

andrewkroh commented May 4, 2017

andrewkroh commented May 5, 2017

ruflin commented May 5, 2017

ruflin commented May 5, 2017

elasticmachine May 5, 2017

Choose a reason for hiding this comment

ruflin commented May 9, 2017

ruflin commented May 10, 2017

andrewkroh left a comment

Choose a reason for hiding this comment

andrewkroh May 10, 2017

Choose a reason for hiding this comment

amandahla May 10, 2017

Choose a reason for hiding this comment

ruflin left a comment

Choose a reason for hiding this comment

ruflin May 11, 2017

Choose a reason for hiding this comment

amandahla May 11, 2017

Choose a reason for hiding this comment

ruflin May 11, 2017

Choose a reason for hiding this comment

andrewkroh commented May 11, 2017

monicasarbu commented May 15, 2017

amandahla commented May 4, 2017 •

edited

Loading