Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .editorconfig
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,9 @@ root = true
max_line_length = 100
indent_size = 2

[*.md]
max_line_length = 120

[renovate.json5]
max_line_length = 300

Expand Down
2 changes: 1 addition & 1 deletion .markdownlint.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -8,4 +8,4 @@ MD013:
line_length: 120

MD024:
allow_different_nesting: true
siblings_only: true
1 change: 1 addition & 0 deletions .prettierignore
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
**/*.md
19 changes: 13 additions & 6 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Contributing

It's recommended to use the [mise](https://mise.jdx.dev/) for development.
It's recommended to use the [mise][mise] for development.

## Building locally

Expand All @@ -20,20 +20,20 @@ It's recommended to use the [mise](https://mise.jdx.dev/) for development.
Acceptance test cases are defined in `oats.yaml` files in the examples directory.
The test cases are run by [oats].

If a test case fails (lets say `examples/nodejs`), follows these steps:
If a test case fails (let's say `examples/nodejs`), follow these steps:

1. Build a new image: `mise run build-lgtm dev1`
2. `oats -timeout 2h -lgtm-version dev1 examples/nodejs` (automatically installed by `mise`)
3. go to <http://localhost:3000>

You can run all everything together using `mise run test`.

[oats]: https://github.com/grafana/oats

## Architecture diagram

The source code for the architecture diagram is a
[google slide](https://docs.google.com/presentation/d/1txMBBitezscvtJIXRHNSXnCekjMRM29GmHufUSI0NRw/edit?slide=id.g26040f0db78_0_0#slide=id.g26040f0db78_0_0). <!-- editorconfig-checker-disable-line -->
> [!NOTE]
> The architecture diagram is only accessible to Grafana employees.

The source code for the architecture diagram is a [Google slide][architecture].
Take a screenshot of the slide and save it as `img/overview.png`.

## OTel Collector
Expand All @@ -44,3 +44,10 @@ Take a screenshot of the slide and save it as `img/overview.png`.
./otelcol-contrib --config docker/otelcol-config.yaml --config docker/otelcol-export-http.yaml \
print-initial-config --feature-gates otelcol.printInitialConfig > merged.yaml
```

<!-- editorconfig-checker-disable -->
<!-- markdownlint-disable MD013 -->
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

that's a neat trick!


[architecture]: https://docs.google.com/presentation/d/1txMBBitezscvtJIXRHNSXnCekjMRM29GmHufUSI0NRw/edit?slide=id.g26040f0db78_0_0#slide=id.g26040f0db78_0_0
[mise]: https://github.com/jdx/mise
[oats]: https://github.com/grafana/oats
124 changes: 81 additions & 43 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,32 +2,42 @@

An OpenTelemetry backend in a Docker image.

<!-- markdownlint-disable-next-line MD013 -->
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why do we need annotations for prettier, markdownlint and editorconfig-checker now?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added the markdownlint one to remove the squiggles I get in VS Code, but then prettier complains that there isn't a new line between the markdownlint-disable-next-line, but adding that would make the next-line not work, so it needs disabling around that block.

If there's a neater way to make all three happy for this one line I can change it (or I can just make the alt text short enough to not go over 120).

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe we can let only one linter check the line length by telling the others to ignore the rule

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've added a .prettierignore to ignore Markdown sotwo of the extra lines can be removed. For the sake of just the one inline suppression I reckon that's good enough.

![Components included in the Docker image: OpenTelemetry collector, Prometheus, Tempo, Loki, Grafana, Pyroscope](img/overview.png) <!-- editorconfig-checker-disable-line -->

The `grafana/otel-lgtm` Docker image is an open source backend for OpenTelemetry
thats intended for development, demo, and testing environments.
that's intended for development, demo, and testing environments.
If you are looking for a production-ready, out-of-the box solution to monitor applications
and minimize MTTR (mean time to resolution) with OpenTelemetry and Prometheus,
you should try [Grafana Cloud Application Observability](https://grafana.com/products/cloud/application-observability/). <!-- editorconfig-checker-disable-line -->
you should try [Grafana Cloud Application Observability][app-o11y].

## Documentation

- Blog post: [An OpenTelemetry backend in a Docker image: Introducing grafana/otel-lgtm](https://grafana.com/blog/2024/03/13/an-opentelemetry-backend-in-a-docker-image-introducing-grafana/otel-lgtm/) <!-- editorconfig-checker-disable-line -->
- Blog post: [_An OpenTelemetry backend in a Docker image: Introducing grafana/otel-lgtm_][otel-lgtm]

## Get the Docker image

The Docker image is available on Docker hub: <https://hub.docker.com/r/grafana/otel-lgtm>

## Run the Docker image

### Linux/Unix

```sh
# Unix/Linux
./run-lgtm.sh
```

### Windows (PowerShell)

# Windows (PowerShell)
```powershell
./run-lgtm
```

### Linux/Unix Using mise

You can also use [mise][mise] to run the Docker image:

# Using mise (Unix/Linux)
```sh
mise run lgtm
```

Expand All @@ -37,52 +47,50 @@ mise run lgtm

You can enable logging for troubleshooting:

| Environment Variable | Enable Logging in |
| ---------------------- | ----------------------- |
| ENABLE_LOGS_GRAFANA | Grafana |
| ENABLE_LOGS_LOKI | Loki |
| ENABLE_LOGS_PROMETHEUS | Prometheus |
| ENABLE_LOGS_TEMPO | Tempo |
| ENABLE_LOGS_PYROSCOPE | Pyroscope |
| ENABLE_LOGS_OTELCOL | OpenTelemetry Collector |
| ENABLE_LOGS_ALL | all of the above |
| Environment Variable | Enables Logging in: |
| ------------------------ | ----------------------- |
| `ENABLE_LOGS_GRAFANA` | Grafana |
| `ENABLE_LOGS_LOKI` | Loki |
| `ENABLE_LOGS_PROMETHEUS` | Prometheus |
| `ENABLE_LOGS_TEMPO` | Tempo |
| `ENABLE_LOGS_PYROSCOPE` | Pyroscope |
| `ENABLE_LOGS_OTELCOL` | OpenTelemetry Collector |
| `ENABLE_LOGS_ALL` | All of the above |

This has nothing to do with the application logs, which are collected by OpenTelemetry.
This has nothing to do with any application logs, which are collected by OpenTelemetry.

### Send data to vendors

In addition to the built-in observability tools, you can also send data to vendors.
That way, you can easily try and switch between different backends.

If the [OTEL_EXPORTER_OTLP_ENDPOINT](https://opentelemetry.io/docs/languages/sdk-configuration/otlp-exporter/#otel_exporter_otlp_endpoint) <!-- editorconfig-checker-disable-line -->
If the [`OTEL_EXPORTER_OTLP_ENDPOINT`][otlp-endpoint]
variable is set, the OpenTelemetry Collector will send data (logs, metrics, and traces)
to the specified endpoint using "OTLP/HTTP".

In addition, you can provide
[OTEL_EXPORTER_OTLP_HEADERS](https://opentelemetry.io/docs/languages/sdk-configuration/otlp-exporter/#otel_exporter_otlp_headers), <!-- editorconfig-checker-disable-line -->
In addition, you can provide [`OTEL_EXPORTER_OTLP_HEADERS`][otlp-headers],
for example, to authenticate with the backend.

#### Send data to Grafana Cloud

You can find the values for the environment variables in your
[Grafana Cloud account](https://grafana.com/docs/grafana-cloud/send-data/otlp/send-data-otlp/#manual-opentelemetry-setup-for-advanced-users). <!-- editorconfig-checker-disable-line -->
You can find the values for the environment variables in your [Grafana Cloud account][otel-setup].

### Persist data across container instantiation

The various components in the repository are configured to write their data to the /data
The various components in the repository are configured to write their data to the `/data`
directory. If you need to persist data across containers being created and destroyed,
you can mount a volume to the /data directory. Note that this image is intended for
you can mount a volume to the `/data` directory. Note that this image is intended for
development, demo, and testing environments and persisting data to an external volume
doesn't change that. However, this feature could be useful in certain cases for
some users even in testing situations.

## Run lgtm in kubernetes
## Run lgtm in Kubernetes

```sh
# create k8s resources
# Create k8s resources
kubectl apply -f k8s/lgtm.yaml

# port forwarding
# Configure port forwarding
kubectl port-forward service/lgtm 3000:3000 4317:4317 4318:4318

# Using mise
Expand All @@ -92,17 +100,17 @@ mise k8s-port-forward

## Send OpenTelemetry Data

There's no need to configure anything: The Docker image works with OpenTelemetry's defaults.
There's no need to configure anything: the Docker image works with OpenTelemetry's defaults.

```sh
# Not needed as these are the defaults in OpenTelemetry:
# Not needed, but these are the defaults in OpenTelemetry
export OTEL_EXPORTER_OTLP_PROTOCOL=http/protobuf
export OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4318
```

## View Grafana

Log in to [http://localhost:3000](http://localhost:3000) with user _admin_ and password _admin_.
Navigate to <http://localhost:3000> and log in with the default built-in user `admin` and password `admin`.

## Build the Docker image from scratch

Expand All @@ -116,41 +124,57 @@ mise build-lgtm

## Build and run the example app

> **_NOTE:_** You can run everything together using
> [mise](https://mise.jdx.dev/) with `mise run all`
> [!TIP]
> You can run everything together using [mise][mise] with `mise run all`.

### Run

Run the example REST service:

#### Unix/Linux

```sh
# Unix/Linux
./run-example.sh
```

#### Windows (PowerShell)

# Windows (PowerShell)
```powershell
./run-example
```

#### Unix/Linux Using mise

# Using mise (Unix/Linux)
```sh
mise run example
```

Generate traffic:
### Generate traffic

#### Unix/Linux

```sh
# Unix/Linux
./generate-traffic.sh
```

#### Windows (PowerShell)

# Windows (PowerShell)
```powershell
./generate-traffic
```

#### Unix/Linux Using mise

# Using mise (Unix/Linux)
```sh
mise run generate-traffic
```

> **_NOTE:_** You can use [OTel Checker](https://github.com/grafana/otel-checker/)
> to check if the instrumentation is correct.
> [!TIP]
> You can use [OTel Checker][otel-checker] to check if the instrumentation is correct.

## Run example apps in different languages

The example apps are in the `examples/` directory.
The example apps are in the [`examples/`][examples] directory.
Each example has a `run.sh` or `run.cmd` script to start the app.

Every example implements a rolldice service, which returns a random number between 1 and 6.
Expand All @@ -163,8 +187,22 @@ Each example uses a different application port
| Java | `curl http://localhost:8080/rolldice` |
| Go | `curl http://localhost:8081/rolldice` |
| Python | `curl http://localhost:8082/rolldice` |
| dotnet | `curl http://localhost:8083/rolldice` |
| .NET | `curl http://localhost:8083/rolldice` |
| Node.js | `curl http://localhost:8084/rolldice` |

## Related Work

- Metrics, Logs, Traces and Profiles in Grafana: <https://github.com/grafana/intro-to-mltp>
- [Metrics, Logs, Traces and Profiles in Grafana][mltp]

<!-- editorconfig-checker-disable -->
<!-- markdownlint-disable MD013 -->

[app-o11y]: https://grafana.com/products/cloud/application-observability/
[examples]: https://github.com/grafana/docker-otel-lgtm/tree/main/examples
[mise]: https://github.com/jdx/mise
[mltp]: https://github.com/grafana/intro-to-mltp
[otel-checker]: https://github.com/grafana/otel-checker/
[otel-lgtm]: https://grafana.com/blog/2024/03/13/an-opentelemetry-backend-in-a-docker-image-introducing-grafana/otel-lgtm/
[otel-setup]: https://grafana.com/docs/grafana-cloud/send-data/otlp/send-data-otlp/#manual-opentelemetry-setup-for-advanced-users
[otlp-endpoint]: https://opentelemetry.io/docs/languages/sdk-configuration/otlp-exporter/#otel_exporter_otlp_endpoint
[otlp-headers]: https://opentelemetry.io/docs/languages/sdk-configuration/otlp-exporter/#otel_exporter_otlp_headers
1 change: 1 addition & 0 deletions generate-traffic.cmd
Original file line number Diff line number Diff line change
Expand Up @@ -5,5 +5,6 @@
curl -s http://localhost:8081/rolldice
curl -s http://localhost:8082/rolldice
curl -s http://localhost:8083/rolldice
curl -s http://localhost:8084/rolldice
timeout /t 2
goto loop