Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 17 additions & 10 deletions MAINTAINERS.md
Original file line number Diff line number Diff line change
@@ -1,11 +1,18 @@
# Observability Maintainers
## Overview

## Maintainers
| Maintainer | GitHub ID | Affiliation |
|---------------|-------------------------------------------------|-------------|
| David Cui | [davidcui1225](https://github.com/davidcui1225) | Amazon |
| Eric Wei | [mengweieric](https://github.com/mengweieric) | Amazon |
| Joshua Li | [joshuali925](https://github.com/joshuali925) | Amazon |
| Shenoy Pratik | [ps48](https://github.com/ps48) | Amazon |
| Kavitha Mohan | [kavithacm] (https://github.com/kavithacm) | Amazon |
| Eugene Lee | [eugenesk24] (https://github.com/eugenesk24) | Amazon |
This document contains a list of maintainers in this repo. See [opensearch-project/.github/RESPONSIBILITIES.md](https://github.com/opensearch-project/.github/blob/main/RESPONSIBILITIES.md#maintainer-responsibilities) that explains what the role of maintainer means, what maintainers do in this and other repos, and how they should be doing it. If you're interested in contributing, and becoming a maintainer, see [CONTRIBUTING](CONTRIBUTING.md).

## Current Maintainers

| Maintainer | GitHub ID | Affiliation |
| ----------------- | ------------------------------------------------- | ----------- |
| David Cui | [davidcui1225](https://github.com/davidcui1225) | Amazon |
Comment on lines +7 to +9
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the workflow you are working? Am wondering if we can hop on a call... I think it is best to avoid including previous commits into your own PR from other folks

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I branched from 2.x & cherry picked relevant files from main branch fork..

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Doesn’t it represents the latest correct maintainers info ?

| Eric Wei | [mengweieric](https://github.com/mengweieric) | Amazon |
| Joshua Li | [joshuali925](https://github.com/joshuali925) | Amazon |
| Shenoy Pratik | [ps48](https://github.com/ps48) | Amazon |
| Kavitha Mohan | [kavithacm](https://github.com/kavithacm) | Amazon |
| Eugene Lee | [eugenesk24](https://github.com/eugenesk24) | Amazon |
| Rupal Mahajan | [rupal-bq](https://github.com/rupal-bq) | Amazon |
| Derek Ho | [derek-ho](https://github.com/derek-ho) | Amazon |
| Lior Perry | [YANG-DB](https://github.com/YANG-DB) | Amazon |
| Peter Fitzgibbons | [pjfitzgibbons](https://github.com/pjfitzgibbons) | Amazon |
120 changes: 120 additions & 0 deletions integrations/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,120 @@
# Definitions

## Bundle

An OpenSearch Integration Bundle may contain the following:
- dashboards
- visualisations
- configurations
These bundle assets are designed to assist monitor of logs and metrics for a particular resource (device, network element, service ) or group of related resources, such as “Nginx”, or “System”.

---

The Bundle consists of:

* Version
* Metadata configuration file
* Dashboards and visualisations and Notebooks
* Data stream index templates used for the signal's ingestion
* Documentation & information


## Integration

An integration is a type of _bundle_ defining data-streams for ingetion of a resource observed signals using logs, metrics, and traces.

### Structure
As mentioned above, integration is a collection of elements that formulate how to observe a specific data emitting resource - in our case a telemetry data producer.

A typical Observability Integration consists of the following parts:

***Metadata***

* Observability data producer resource
* Supplement Indices (mapping & naming)
* Collection Agent Version
* Transformation schema
* Optional test harnesses repository
* Verified version and documentation
* Category & classification (logs/traces/alerts/metrics)

***Display components***

* Dashboards
* Maps
* Applications
* Notebooks
* Operations Panels
* Saved PPL/SQL/DQL Queries
* Alerts

Since the structured data has an enormous contribution to the understanding of the system behaviour - each resource will define a well-structured mapping it conforms with.

Once input content has form and shape - it can and will be used to calculate and correlate different pieces of data.

The next parts of this document will present **Integrations For Observability** which has a key concept of Observability schema.

It will overview the concepts of observability, will describe the current issues customers are facing with observability and continue to elaborate on how to mitigate them using Integrations and structured schemas.

---

### Creating An Integration

```yaml

reousce-name
config.json
display`
Application.json
Maps.json
Dashboard.json
queries
Query.json
schemas
transformation.json
samples
resource.access logs
resource.error logs
resource.stats metrics
expected_results
info
documentation
```

**Definitions**

- `config.json` defines the general configuration for the entire integration component.
- `display` this is the folder in which the actual visualization components are stored
- `queries` this is the folder in which the actual PPL queries are stored
- `schemas` this is the folder in which the schemas are stored - schema for mapping translations or index mapping.
- `samples` this folder contains sample logs and translated logs are present
- `metadata` this folder contains additional metadata definitions such as security and policies
- `info` this folder contains documentations, licences and external references

---

#### Config

`Config.json` file includes the following Integration configuration see [NginX config](nginx/config.json)

Additional information on the config structure see [Structure](docs/Integration-structure.md)

#### Display:

Visualization contains the relevant visual components associated with this integration.

The visual display component will need to be validated to the schema that it is expected to work on - this may be part of the Integration validation flow...

#### Queries

Queries contains specific PPL queries that precisely demonstrates some common and useful use-case .

*Example:*

*-- The visual display component will need to be validated to the schema that it is expected to work on*

```
source = sso_logs-default-prod | ... where ...
```

---
Loading