Skip to content

Conversation

@vivekgsharma
Copy link
Collaborator

@vivekgsharma vivekgsharma commented Mar 11, 2024

Issue being fixed or feature implemented

This change adds initial support for Elasticsearch, Metricbeat, and Kibana through new Ansible roles. It's part of setting up monitoring and visualization for the Dash network deployment tool.

What was done?

  • Added Ansible roles for Elasticsearch, Metricbeat, and Kibana.
  • Updated the deploy.yml file to invoke these roles.
  • Placeholder entries have been added for the target hosts for these services, to be defined later.
  • Note: Terraform resources for these services are not included in this PR and will be defined in subsequent updates.

How Has This Been Tested?

Breaking Changes

  • None: This is an additive change and does not alter existing infrastructure or workflows.

Checklist:

  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have added or updated relevant unit/integration/functional/e2e tests
  • I have made corresponding changes to the documentation

For repository code-owners and collaborators only

  • I have assigned this pull request to a milestone

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We already have the elastic_stack role

@@ -0,0 +1,31 @@
---
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How this is better than using already existing role?

- name: Copy Metricbeat module configurations
ansible.builtin.copy:
src: "{{ item }}"
dest: "/etc/metricbeat/modules.d/{{ item | basename }}"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are there any modules or module configuration is provided?

@lklimek
Copy link

lklimek commented Mar 14, 2024

Let me redefine the task.

Current state

  1. We already start an elastic cluster that is available at addresses like http://logs.testnet.networks.dash.org:5601/login?next=%2F
  2. We have a prometheus server deployed for devnets which contains services: prometheus, grafana
  3. Some initial work was done by Ivan to enable metricbeat in feat: system and docker metrics in kibana #617
  4. Right now, prometheus config is generated with Ansible (mainly list of addresses to collect metrics from)

Long-term goal (epic)

  1. Migrate Grafana to elastic cluster nodes.
  2. Automate configuration of Grafana.
  3. Replace promethus with metricbeat (read data from prometheus exporters into metricbeat)

This task

Goal of this task is to replace prometheus with metricbeat.

To achieve this, we need to:

  • get familiar with existing Ansible code
  • extend existing code by enabling and configuring metricbeat prometheus collector
  • configure metricbeat prometheus collector to reach each Tenderdash Prometheus exporter in the devnet

Acceptance criteria

  • Each Tenderdash node reports metrics to Elastic stack, for example, metric ending with _consensus_height exists in elastic for every hpmn
  • Grafana can visualize these metrics
  • Existing ansible code is modified whenever possible

@vivekgsharma vivekgsharma deleted the elasticsearch_metricbeat_kibana branch March 15, 2024 13:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants