Enable metrics publishing for mt-gateway

[fitzoh]

As requested here.

Note that there are a couple places where prometheus instrumentation is used that have not been addressed:

metrictank/cmd/mt-gateway/ingest/metrics.go

Line 29 in d37ddf3

discardedSamples = promauto.NewCounterVec(

metrictank/publish/publish.go

Line 13 in d37ddf3

ingestedMetrics = promauto.NewCounterVec(prometheus.CounterOpts{

[replay]

the default value here does not match the one in the config file. I think it should also be mt-gateway.stats.default.$hostname

[fitzoh]

oops, e5ee174

[replay]

LGTM, just one very minor issue to fix

[replay]

[Dieterbe]

before we merge. can you confirm that you can take a tsdb-gw dashboard, s/tsdb-gw/mt-gateway/ and it works ?

[fitzoh]

First couple look good, http stuff isn't instrumented

[Dieterbe]

http stuff isn't instrumented

can we ?

[fitzoh]

can we ?

Copied over in c803c82... Not sure if we want to leave them as is or switch to tag based metrics.

[Dieterbe]

how is this copied over? the code looks quite different, making it hard to compare.. can you just make a literal copy, and maybe make any needed changes in a separate commit?

also, that screenie looks good. can you confirm these stats similarly?

[fitzoh]

how is this copied over? the code looks quite different, making it hard to compare.. can you just make a literal copy, and maybe make any needed changes in a separate commit?

Initial import in 1b16b0d, changes in subsequent commits

[fitzoh]

also, that screenie looks good. can you confirm these stats similarly?

After some tweaks to get things up and running...

The response codes were worrying me a little bit, but I was able to confirm that there's data there through the explore view. (The query the dashboard was running was groupByNode(perSecond(mt-gateway.stats.$env.$host.api.request.$path.status.*.counter32), 8, 'sum'))