-
Notifications
You must be signed in to change notification settings - Fork 2.5k
/
Copy pathREADME.md
373 lines (305 loc) · 19.7 KB
/
README.md
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
# Receiver Creator
<!-- status autogenerated section -->
| Status | |
| ------------- |-----------|
| Stability | [alpha]: logs, traces |
| | [beta]: metrics |
| Distributions | [contrib] |
| Issues | [](https://github.com/open-telemetry/opentelemetry-collector-contrib/issues?q=is%3Aopen+is%3Aissue+label%3Areceiver%2Freceivercreator) [](https://github.com/open-telemetry/opentelemetry-collector-contrib/issues?q=is%3Aclosed+is%3Aissue+label%3Areceiver%2Freceivercreator) |
| [Code Owners](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/CONTRIBUTING.md#becoming-a-code-owner) | [@rmfitzpatrick](https://www.github.com/rmfitzpatrick) |
[alpha]: https://github.com/open-telemetry/opentelemetry-collector#alpha
[beta]: https://github.com/open-telemetry/opentelemetry-collector#beta
[contrib]: https://github.com/open-telemetry/opentelemetry-collector-releases/tree/main/distributions/otelcol-contrib
<!-- end autogenerated section -->
This receiver can instantiate other receivers at runtime based on whether
observed endpoints match a configured rule. To use the receiver creator, you
must first configure one or more
[observers](../../extension/observer/README.md) that will discover networked
endpoints that you may be interested in. The configured rules will be
evaluated for each endpoint discovered. If the rule evaluates to true then
the receiver for that rule will be started against the matched endpoint.
If you use the receiver creator in multiple pipelines of differing telemetry types,
but a given dynamically instantiated receiver doesn't support one of the pipeline's type,
it will effectively lead to a logged no-op that won't cause a collector service failure.
## Configuration
**watch_observers**
A list of observers previously defined to be run in the `extensions` section.
receiver_creator will watch for endpoints generated by these observers.
**receivers**
A map of receiver names (e.g. `redis/1`) to a template for when and how to
instantiate that receiver.
**receivers.<receiver_type/id>.rule**
Rule expression using [expvar
syntax](https://github.com/expr-lang/expr/blob/master/docs/language-definition.md).
Variables available are detailed below in [Rule
Expressions](#rule-expressions).
Note: The built-in `type` function introduced in v1.14.1 has been relocated to `typeOf`.
**receivers.<receiver_type/id>.config**
This is configuration that will be used when creating the receiver at
runtime.
This option can use static and dynamic configuration values. Static values
are normal YAML values. However, the value can also be dynamically constructed
from the discovered endpoint object. Dynamic values are surrounded by
backticks (\`). If a literal backtick is needed use \\` to escape it. Dynamic
values can be used with static values in which case they are concatenated.
For example:
```yaml
config:
secure_url: https://`pod.labels["secure_host"]`
```
The value of `secure_url` will be `https://` concatenated with the value of
the `secure_host` label.
This can also be used when the discovered endpoint needs to be changed
dynamically. For instance, suppose the IP `1.2.3.4` is discovered without a
port but the port needs to be set inside endpoint. You could do:
```yaml
config:
endpoint: '`endpoint`:8080'
```
If your target receiver provides an `endpoint` config field and you aren't
manually setting it like the above example, the observer endpoint target value
will automatically be sourced. If no `endpoint` field is available you are
required to specify any necessary fields.
**receivers.resource_attributes**
```yaml
resource_attributes:
<endpoint type>:
<attribute>: <attribute value>
```
This setting controls what resource attributes are set on telemetry emitted from the created receiver. These attributes can be set from [values in the endpoint](#rule-expressions) that was matched by the `rule`. These attributes vary based on the endpoint type. These defaults can be disabled by setting the attribute to be removed to an empty value. Note that the values can be dynamic and processed the same as in `config`.
Note that the backticks below are not typos--they indicate the value is set dynamically.
`type == "pod"`
| Resource Attribute | Default |
|--------------------|---------------|
| k8s.pod.name | \`name\` |
| k8s.pod.uid | \`uid\` |
| k8s.namespace.name | \`namespace\` |
`type == "port"`
| Resource Attribute | Default |
|--------------------|-------------------|
| k8s.pod.name | \`pod.name\` |
| k8s.pod.uid | \`pod.uid\` |
| k8s.namespace.name | \`pod.namespace\` |
`type == "container"`
| Resource Attribute | Default |
|----------------------|-------------------|
| container.name | \`name\` |
| container.image.name | \`image\` |
`type == "hostport"`
None
`type == "k8s.service"`
| Resource Attribute | Default |
|--------------------|-------------------|
| k8s.namespace.name | \`namespace\` |
`type == "k8s.node"`
| Resource Attribute | Default |
|--------------------|-------------------|
| k8s.node.name | \`name\` |
| k8s.node.uid | \`uid\` |
See `redis/2` in [examples](#examples).
**receivers.<receiver_type/id>.resource_attributes**
```yaml
receivers:
<receiver_type>:
resource_attributes:
<attribute>: <attribute string value>
```
Similar to the per-endpoint type `resource_attributes` described above but for individual receiver instances. Duplicate attribute entries (including the empty string) in this receiver-specific mapping take precedence. These attribute values also support expansion from endpoint environment content. At this time their values must be strings.
## Rule Expressions
Each rule must start with `type == ("pod"|"port"|"hostport"|"container"|"k8s.service"|"k8s.node") &&` such that the rule matches
only one endpoint type. Depending on the type of endpoint the rule is
targeting it will have different variables available.
### Pod
| Variable | Description | Data Type |
|-------------|-----------------------------------|-------------------------------|
| type | `"pod"` | String |
| id | ID of source endpoint | String |
| name | name of the pod | String |
| namespace | namespace of the pod | String |
| uid | unique id of the pod | String |
| labels | map of labels set on the pod | Map with String key and value |
| annotations | map of annotations set on the pod | Map with String key and value |
### Port
| Variable | Description | Data Type |
|-----------------|-----------------------------------------|-------------------------------|
| type | `"port"` | String |
| id | ID of source endpoint | String |
| name | container port name | String |
| port | port number | Integer |
| protocol | The transport protocol ("TCP" or "UDP") | String |
| pod.name | name of the owning pod | String |
| pod.namespace | namespace of the pod | String |
| pod.uid | unique id of the pod | String |
| pod.labels | map of labels of the owning pod | Map with String key and value |
| pod.annotations | map of annotations of the owning pod | Map with String key and value |
### Host Port
| Variable | Description | Data Type |
|---------------|--------------------------------------------------|-------------------------------|
| type | `"hostport"` | String |
| id | ID of source endpoint | String |
| process_name | Name of the process | String |
| command | Command line with the used to invoke the process | String |
| is_ipv6 | true if endpoint is IPv6, otherwise false | Boolean |
| port | Port number | Integer |
| transport | The transport protocol ("TCP" or "UDP") | String |
### Container
| Variable | Description | Data Type |
|----------------|-------------------------------------------------------------------|-------------------------------|
| type | `"container"` | String |
| id | ID of source endpoint | String |
| name | Primary name of the container | String |
| image | Name of the container image | String |
| port | Exposed port of the container | Integer |
| alternate_port | Exposed port accessed through redirection, such as a mapped port | Integer |
| command | The command used to invoke the process of the container | String |
| container_id | ID of the container | String |
| host | Hostname or IP of the underlying host the container is running on | String |
| transport | Transport protocol used by the endpoint (TCP or UDP) | String |
| labels | User-specified metadata labels on the container | Map with String key and value |
### Kubernetes Service
| Variable | Description | Data Type |
|----------------|---------------------------------------------------------------------------------------|-------------------------------|
| type | `"k8s.service"` | String |
| id | ID of source endpoint | String |
| name | The name of the Kubernetes service | String |
| namespace | The namespace of the service | String |
| uid | The unique ID for the service | String |
| labels | The map of labels set on the service | Map with String key and value |
| annotations | The map of annotations set on the service | Map with String key and value |
| service_type | The type of the kubernetes service: ClusterIP, NodePort, LoadBalancer, ExternalName | String |
| cluster_ip | The cluster IP assigned to the service | String |
### Kubernetes Ingress
| Variable | Description | Data Type |
|----------------|---------------------------------------------------------------------------------------|-------------------------------|
| type | `"k8s.ingress"` | String |
| id | ID of source endpoint | String |
| name | The name of the Kubernetes ingress | String |
| namespace | The namespace of the ingress | String |
| uid | The unique ID for the ingress | String |
| labels | The map of labels set on the ingress | Map with String key and value |
| annotations | The map of annotations set on the ingress | Map with String key and value |
| scheme | Scheme represents whether the ingress path is accessible via HTTPS or HTTP. | String |
| host | Host is the FQDN that map to backends | String |
| path | Path that map requests to backends | String |
### Kubernetes Node
| Variable | Description | Data Type |
|-----------------------|----------------------------------------------------------------------|-------------------------------|
| type | `"k8s.node"` | String |
| id | ID of source endpoint | String |
| name | The name of the Kubernetes node | String |
| uid | The unique ID for the node | String |
| hostname | The node's hostname as reported by its Status object | String |
| external_ip | The node's external IP address as reported by its Status object | String |
| internal_ip | The node's internal IP address as reported by its Status object | String |
| external_dns | The node's external DNS record as reported by its Status object | String |
| internal_dns | The node's internal DNS record as reported by its Status object | String |
| annotations | A key-value map of non-identifying, user-specified node metadata | Map with String key and value |
| labels | A key-value map of user-specified node metadata | Map with String key and value |
| kubelet_endpoint_port | The node Status object's DaemonEndpoints.KubeletEndpoint.Port value | Integer |
## Examples
```yaml
extensions:
# Configures the Kubernetes observer to watch for pod start and stop events.
k8s_observer:
observe_nodes: true
observe_services: true
observe_ingresses: true
host_observer:
receivers:
receiver_creator/1:
# Name of the extensions to watch for endpoints to start and stop.
watch_observers: [k8s_observer]
receivers:
prometheus_simple:
# Configure prometheus scraping if standard prometheus annotations are set on the pod.
rule: type == "pod" && annotations["prometheus.io/scrape"] == "true"
config:
metrics_path: '`"prometheus.io/path" in annotations ? annotations["prometheus.io/path"] : "/metrics"`'
endpoint: '`endpoint`:`"prometheus.io/port" in annotations ? annotations["prometheus.io/port"] : 9090`'
resource_attributes:
an.attribute: a.value
# Dynamic configuration values
app.version: '`labels["app_version"]`'
redis/1:
# If this rule matches an instance of this receiver will be started.
rule: type == "port" && port == 6379
config:
# Static receiver-specific config.
password: secret
# Dynamic configuration value.
collection_interval: '`pod.annotations["collection_interval"]`'
redis/2:
# Set a resource attribute based on endpoint value.
rule: type == "port" && port == 6379
sqlserver:
rule: type == "port" && pod.name matches "(?i)mssql"
config:
server: '`host`'
port: '`port`'
username: sa
password: password
resource_attributes:
# Dynamic configuration values, overwriting default attributes`
pod:
service.name: '`labels["service_name"]`'
app: '`labels["app"]`'
port:
service.name: '`pod.labels["service_name"]`'
app: '`pod.labels["app"]`'
receiver_creator/2:
# Name of the extensions to watch for endpoints to start and stop.
watch_observers: [host_observer]
receivers:
redis/on_host:
# If this rule matches an instance of this receiver will be started.
rule: type == "port" && port == 6379 && is_ipv6 == true
resource_attributes:
service.name: redis_on_host
receiver_creator/3:
watch_observers: [k8s_observer]
receivers:
kubeletstats:
rule: type == "k8s.node"
config:
auth_type: serviceAccount
collection_interval: 10s
endpoint: '`endpoint`:`kubelet_endpoint_port`'
extra_metadata_labels:
- container.id
metric_groups:
- container
- pod
- node
httpcheck:
# Configure probing if standard prometheus annotations are set on the pod.
rule: type == "k8s.service" && annotations["prometheus.io/probe"] == "true"
config:
targets:
- endpoint: 'http://`endpoint`:`"prometheus.io/port" in annotations ? annotations["prometheus.io/port"] : 9090``"prometheus.io/path" in annotations ? annotations["prometheus.io/path"] : "/health"`'
method: GET
collection_interval: 10s
receiver_creator/4:
watch_observers: [k8s_observer]
receivers:
httpcheck:
# Configure probing if standard prometheus annotations are set on the pod.
rule: type == "k8s.ingress" && annotations["prometheus.io/probe"] == "true"
config:
targets:
- endpoint: '`scheme`://`endpoint`:`port``"prometheus.io/path" in annotations ? annotations["prometheus.io/path"] : "/health"`'
method: GET
collection_interval: 10s
processors:
exampleprocessor:
exporters:
exampleexporter:
service:
pipelines:
metrics:
receivers: [receiver_creator/1, receiver_creator/2, receiver_creator/3, receiver_creator/4]
processors: [exampleprocessor]
exporters: [exampleexporter]
extensions: [k8s_observer, host_observer]
```
The full list of settings exposed for this receiver are documented [here](./config.go)
with detailed sample configurations [here](./testdata/config.yaml).