feat: add spec files for entity modeling

[gsanchezgavier]

This PR adds the spec files model that will be use to compose entities with the metrics based on the file definition.

The functions added here will be used by the integration when running with the Agent and using sdk emitter.

All spec files will be located in a specific folder and pomi will load them every execution.

[CLAassistant]

All committers have signed the CLA.

[paologallinaharbur]

I had the some doubt when writing the prometheus parser, I do not know if we can assume we have only one service.
As we discussed today we might have a single exporter returning two (or more "services"). In that case either we will have multiple file or we need to change the structure to something like:

type SpecDef struct {
    Provider string      `yaml:"provider"`
    Service  []Service `yaml:"service"`
}

type Service struct {
	ServiceName  string           `yaml:"service"`
	Entities            []EntityDef  `yaml:"entities"`
}

In this case we would not follow cloud integration structure

[gsanchezgavier]

Thanks for bring this up. If for some reason the same exporter exposes metrics from different services we can define a spec file for each of the service and keep the same structure.
The service will be picked from the metric prefix and treated as defined in the corresponding spec file.
I'm thinking about that this solution could also work for the exporter related metrics like go_ or the ones related to the host where the exporter runs like process_

[ardias]

@gsanchezgavier To be clear, you're saying that in the case of 2 or more services for the same exporter we will create a file for each one? If that is your idea, I just worry about how it will impact external contributors, especially customers. It may be difficult to explain that, even with documentation, and the more files we have the harder it gets to contribute. We should really try to reduce the number of files required to make an exporter work.
I'm not sure we need to follow the exact same structure of the cloud integrations, especially if it doesn't work well for us.
Just my 2cts

[paologallinaharbur]

I agree that we should try as much as possible to have one file per integration.

Either we follow the cloud integration approach adding more services with the ---
ES:

provider: prometheus
service: ravendb1
display_name: Raven Db
entities:
[...]
---
provider: prometheus
service: ravendb2
display_name: Raven Db
entities:
[...]

Or we can change the structure allowing arrays of services in one yaml. We will need to change a bit the automatic tool but then we will be free to diverge when needed

ES:

provider: prometheus
display_name: Raven Db
services:
  - ravendb1
     entities:
     [...]
  - ravendb2
     entities:
     [...]

[gsanchezgavier]

@morecar needs your opinion here. I think cloud int. also splits some specs files for some integrations.
I also want to mention that i think having 2 different services in one exporter don't looks like a good prometheus best practice.

[ardias]

I think we may need to look into some more exporters to get an idea of the number of "services" that they expose.
On the other hand we can just assume that 1 exporter = 1 service. For example Redis has metrics that start with very different "prefixes": redis_ , process_ , go_ . If we ignore the go prefix (which the exporter can be configured to not return), the metrics all belong to the Redis service, just with different entities. I'm not sure about other exporters but I'm guessing it may be the same.

In summary, I would not make services an array. If we find we need more than one service for one exporter we can use the first option, having multiple services on the same file

[paologallinaharbur]

sounds good to me

[paologallinaharbur]

LGTM.

[paologallinaharbur]

Are we going to remove Prometheus prefix from entity name in next iteration?

[gsanchezgavier]

yes i'll do it

[gsanchezgavier]

rebased from main and remove provider prefix

[paologallinaharbur]

Thanks!