Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tracking issue: Prometheus compatibility MVP #111

Open
7 of 11 tasks
flyingmutant opened this issue Dec 31, 2022 · 0 comments
Open
7 of 11 tasks

Tracking issue: Prometheus compatibility MVP #111

flyingmutant opened this issue Dec 31, 2022 · 0 comments
Labels
A-prom Area: Prometheus / OpenMetrics / PromQL compatibility A-sh Area: statshouse (agent / aggregator) A-ui Area: statshouse-ui C-feature-request Category: request for new functionality C-tracking-issue Category: tracking issue

Comments

@flyingmutant
Copy link
Contributor

flyingmutant commented Dec 31, 2022

Initial Prometheus compatibility support: scraping, remote write, PromQL.

  • speed up ClickHouse access by using https://github.com/ClickHouse/ch-go
  • check the details of how VictoriaMetrics downsampling works (do MetricsQL query results looks the same no matter the downsampling?)
  • decide if we'll continue to use Prometheus PromQL implementation or migrate to MetricsQL
  • remove all fancy SQL mapping features and focus on in-memory computation on raw data

Setup:

  • setup StatsHouse agents on all nodes of our biggest k8s cluster
  • start scraping several big k8s (or node_exporter) metrics using k8s service discovery, probably prefixing them for namespacing (is this related to Tracking issue: group budgets #110?)
  • setup Prometheus/VM scraping of the same metrics
  • compare standard Grafana dashboards with StatsHouse vs Prometheus/VM data source

Optimizations:

  • translate regexp label selectors into SQL IN matches
  • consider making all internal operations operate on int tag values, translating to strings only before returning the result

Misc. things:

Out of MVP scope:

  • making sure PromQL is a superset of /api/query (e.g. topk semantics)
  • protobuf scraping

Histograms:

See also:

@flyingmutant flyingmutant added A-ui Area: statshouse-ui A-api Area: statshouse-api A-sh Area: statshouse (agent / aggregator) A-prom Area: Prometheus / OpenMetrics / PromQL compatibility C-feature-request Category: request for new functionality C-tracking-issue Category: tracking issue labels Dec 31, 2022
@flyingmutant flyingmutant mentioned this issue Dec 31, 2022
@flyingmutant flyingmutant removed the A-api Area: statshouse-api label Dec 31, 2022
@flyingmutant flyingmutant pinned this issue Jan 13, 2023
@alpinskiy alpinskiy mentioned this issue Feb 16, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-prom Area: Prometheus / OpenMetrics / PromQL compatibility A-sh Area: statshouse (agent / aggregator) A-ui Area: statshouse-ui C-feature-request Category: request for new functionality C-tracking-issue Category: tracking issue
Projects
None yet
Development

No branches or pull requests

1 participant