diff --git a/examples/kubernetes/README.md b/examples/kubernetes/README.md new file mode 100644 index 000000000000..3aba4248c4f1 --- /dev/null +++ b/examples/kubernetes/README.md @@ -0,0 +1,59 @@ +# OpenTelemetry Collector Demo + +This demo is a sample app to build the collector and exercise its kubernetes logs scrapping functionality. + +## Build and Run + +### Kubernetes + +Switch to this directory and run following command: `kubectl apply -n -f otel-collector.yaml` + +#### Include/Exclude Specific Logs + +Kubernetes logs are being stored in `/var/log/pods`. +Path to container logs is constructed using following pattern: + +`/var/log/pods/__//.log` + +You can use it to manage from which containers do you want to include and exclude logs. + +For example, to include all logs from `default` namespace, you can use following configuration: + +```yaml +include: + - /var/log/pods/default_*/*/*.log +exclude: [] +``` + +### Docker Compose + +Two steps are required to build and run the demo: + +1. Build latest docker image in main repository directory `make docker-otelcontribcol` +1. Switch to this directory and run `docker-compose up` + +#### Description + +`varlogpods` contains example log files placed in kubernetes-like directory structure. +Each of the directory has different formatted logs in one of three formats (either `CRI-O`, `CRI-Containerd` or `Docker`). +This directory is mounted to standard location (`/var/log/pods`). + +`otel-collector-config` is a configuration to autodetect and parse logs for all of three mentioned formats + +## Performance Tests + +There are multiple tests for various configurations in [`testbed`](../../testbed/tests/log_test.go). +Following table shows result of example run: + +Test |Result|Duration|CPU Avg%|CPU Max%|RAM Avg MiB|RAM Max MiB|Sent Items|Received Items| +----------------------------------------|------|-------:|-------:|-------:|----------:|----------:|---------:|-------------:| +Log10kDPS/OTLP |PASS | 15s| 15.2| 15.7| 69| 73| 149900| 149900| +Log10kDPS/filelog |PASS | 15s| 16.5| 18.0| 61| 74| 150000| 150000| +Log10kDPS/kubernetes_containers |PASS | 15s| 42.3| 44.0| 66| 80| 150000| 150000| +Log10kDPS/k8s_CRI-Containerd |PASS | 15s| 36.7| 38.0| 64| 78| 150000| 150000| +Log10kDPS/k8s_CRI-Containerd_no_attr_ops|PASS | 15s| 28.9| 29.7| 64| 77| 150000| 150000| +Log10kDPS/CRI-Containerd |PASS | 15s| 19.0| 21.0| 63| 77| 150000| 150000| + +## ToDo + +To cover kubernetes system logs, logs from journald should be supported as well. diff --git a/examples/kubernetes/docker-compose.yml b/examples/kubernetes/docker-compose.yml new file mode 100644 index 000000000000..6bb749ad801d --- /dev/null +++ b/examples/kubernetes/docker-compose.yml @@ -0,0 +1,8 @@ +version: "3" +services: + opentelemetry-collector-contrib: + image: otelcontribcol + command: ["--config=/etc/otel-collector-config.yml"] + volumes: + - ./otel-collector-config.yml:/etc/otel-collector-config.yml + - ./varlogpods:/var/log/pods diff --git a/examples/kubernetes/otel-collector-config.yml b/examples/kubernetes/otel-collector-config.yml new file mode 100644 index 000000000000..e40b27ff973a --- /dev/null +++ b/examples/kubernetes/otel-collector-config.yml @@ -0,0 +1,78 @@ +receivers: + filelog: + include: + - /var/log/pods/*/*/*.log + exclude: + # Exclude logs from all containers named otel-collector + - /var/log/pods/*/otel-collector/*.log + start_at: beginning + include_file_path: true + include_file_name: false + operators: + # Find out which format is used by kubernetes + - type: router + id: get-format + routes: + - output: parser-docker + expr: '$$record matches "^\\{"' + - output: parser-crio + expr: '$$record matches "^[^ Z]+ "' + - output: parser-containerd + expr: '$$record matches "^[^ Z]+Z"' + # Parse CRI-O format + - type: regex_parser + id: parser-crio + regex: '^(?P