Skip to content

Commit 4282ff8

Browse files
CodePrometheusCodePrometheus
authored
Add go agent so11y dashboard (#13074)
1 parent b2c50e4 commit 4282ff8

File tree

20 files changed

+542
-30
lines changed

20 files changed

+542
-30
lines changed

Diff for: ‎docs/en/changes/changes-10.1.0.md

+1
Original file line numberDiff line numberDiff line change
@@ -83,6 +83,7 @@
8383
* Optimize `network_address_alias` table to reduce the number of the index.
8484
* PromQL service: support round brackets operator.
8585
* Support query Alarm message Tag for auto-complete.
86+
* Add SkyWalking Go Agent self observability dashboard.
8687

8788
#### UI
8889

Diff for: ‎docs/en/setup/backend/dashboards-so11y-go-agent.md

+32
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,32 @@
1+
# Go Agent self observability dashboard
2+
3+
SkyWalking go agent reports its metrics using Meter APIS to measure tracing performance.
4+
it also provides a dashboard to visualize the agent metrics.
5+
6+
## Data flow
7+
1. SkyWalking go agent reports metrics data internally and automatically.
8+
2. SkyWalking OAP receives these meters through native protocols.
9+
3. The SkyWalking OAP Server parses the expression with [MAL](../../concepts-and-designs/mal.md) to filter/calculate/aggregate and store the results.
10+
11+
## Set up
12+
Go Agent so11y is a build-in feature, it reports meters automatically after boot.
13+
14+
## Self observability monitoring
15+
Self observability monitoring provides monitoring of the runtime performance of the go agent itself. `agent.service_name` is a `Service` in Agent so11y, and land on the `Layer: SO11Y_GO_AGENT`.
16+
17+
### Self observability metrics
18+
19+
| Unit | Metric Name | Description | Data Source |
20+
|------------------|-------------------------------------------------------|---------------------------------------------|---------------------|
21+
| Count Per Minute | meter_sw_go_created_tracing_context_count | Created Tracing Context Count (Per Minute) | SkyWalking Go Agent |
22+
| Count Per Minute | meter_sw_go_finished_tracing_context_count | Finished Tracing Context Count (Per Minute) | SkyWalking Go Agent |
23+
| Count Per Minute | meter_sw_go_created_ignored_context_count | Created Ignored Context Count (Per Minute) | SkyWalking Go Agent |
24+
| Count Per Minute | meter_sw_go_finished_ignored_context_count | Finished Ignored Context Count (Per Minute) | SkyWalking Go Agent |
25+
| Count Per Minute | meter_sw_go_possible_leaked_context_count | Possible Leak Context Count (Per Minute) | SkyWalking Go Agent |
26+
| Count Per Minute | meter_sw_go_interceptor_error_count | Interceptor Error Count (Per Minute) | SkyWalking Go Agent |
27+
| ns | meter_sw_go_tracing_context_execution_time_percentile | Tracing Context Execution Time (ns) | SkyWalking Go Agent |
28+
29+
## Customizations
30+
You can customize your own metrics/expression/dashboard panel.
31+
The metrics definition and expression rules are found in `/meter-analyzer-config/go-agent.yaml`
32+
The self observability dashboard panel configurations are found in `/config/ui-initialized-templates/so11y_go_agent`.

Diff for: ‎docs/menu.yml

+2
Original file line numberDiff line numberDiff line change
@@ -150,6 +150,8 @@ catalog:
150150
path: "/en/setup/backend/dashboards-so11y-satellite"
151151
- name: "SkyWalking Java Agent self telemetry"
152152
path: "/en/setup/backend/dashboards-so11y-java-agent"
153+
- name: "SkyWalking Go Agent self telemetry"
154+
path: "/en/setup/backend/dashboards-so11y-go-agent"
153155
- name: "Configuration Vocabulary"
154156
path: "/en/setup/backend/configuration-vocabulary"
155157
- name: "Advanced Setup"

Diff for: ‎oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/analysis/Layer.java

+7-1
Original file line numberDiff line numberDiff line change
@@ -245,7 +245,13 @@ public enum Layer {
245245
/**
246246
* Kong is Cloud-Native API Gateway and AI Gateway.
247247
*/
248-
KONG(40, true);
248+
KONG(40, true),
249+
250+
/**
251+
* The self observability of SkyWalking Go Agent,
252+
* which provides the abilities to measure the tracing performance and error statistics of plugins.
253+
*/
254+
SO11Y_GO_AGENT(41, true);
249255

250256
private final int value;
251257
/**

Diff for: ‎oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/management/ui/template/UITemplateInitializer.java

+1
Original file line numberDiff line numberDiff line change
@@ -78,6 +78,7 @@ public class UITemplateInitializer {
7878
Layer.CILIUM_SERVICE.name(),
7979
Layer.SO11Y_JAVA_AGENT.name(),
8080
Layer.KONG.name(),
81+
Layer.SO11Y_GO_AGENT.name(),
8182
"custom"
8283
};
8384
private final UITemplateManagementService uiTemplateManagementService;

Diff for: ‎oap-server/server-starter/src/main/resources/application.yml

+1-1
Original file line numberDiff line numberDiff line change
@@ -293,7 +293,7 @@ agent-analyzer:
293293
# Nginx and Envoy agents can't get the real remote address.
294294
# Exit spans with the component in the list would not generate the client-side instance relation metrics.
295295
noUpstreamRealAddressAgents: ${SW_NO_UPSTREAM_REAL_ADDRESS:6000,9000}
296-
meterAnalyzerActiveFiles: ${SW_METER_ANALYZER_ACTIVE_FILES:datasource,threadpool,satellite,go-runtime,python-runtime,continuous-profiling,java-agent} # Which files could be meter analyzed, files split by ","
296+
meterAnalyzerActiveFiles: ${SW_METER_ANALYZER_ACTIVE_FILES:datasource,threadpool,satellite,go-runtime,python-runtime,continuous-profiling,java-agent,go-agent} # Which files could be meter analyzed, files split by ","
297297
slowCacheReadThreshold: ${SW_SLOW_CACHE_SLOW_READ_THRESHOLD:default:20,redis:10} # The slow cache read operation thresholds. Unit ms.
298298
slowCacheWriteThreshold: ${SW_SLOW_CACHE_SLOW_WRITE_THRESHOLD:default:20,redis:10} # The slow cache write operation thresholds. Unit ms.
299299

Diff for: ‎oap-server/server-starter/src/main/resources/meter-analyzer-config/go-agent.yaml

+32
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,32 @@
1+
# Licensed to the Apache Software Foundation (ASF) under one or more
2+
# contributor license agreements. See the NOTICE file distributed with
3+
# this work for additional information regarding copyright ownership.
4+
# The ASF licenses this file to You under the Apache License, Version 2.0
5+
# (the "License"); you may not use this file except in compliance with
6+
# the License. You may obtain a copy of the License at
7+
#
8+
# http://www.apache.org/licenses/LICENSE-2.0
9+
#
10+
# Unless required by applicable law or agreed to in writing, software
11+
# distributed under the License is distributed on an "AS IS" BASIS,
12+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13+
# See the License for the specific language governing permissions and
14+
# limitations under the License.
15+
16+
expSuffix: instance(['service'], ['instance'], Layer.SO11Y_GO_AGENT)
17+
metricPrefix: meter
18+
metricsRules:
19+
- name: sw_go_created_tracing_context_count
20+
exp: sw_go_created_tracing_context_counter.sum(['created_by', 'service', 'instance']).increase('PT1M')
21+
- name: sw_go_finished_tracing_context_count
22+
exp: sw_go_finished_tracing_context_counter.sum(['service', 'instance']).increase('PT1M')
23+
- name: sw_go_created_ignored_context_count
24+
exp: sw_go_created_ignored_context_counter.sum(['created_by', 'service', 'instance']).increase('PT1M')
25+
- name: sw_go_finished_ignored_context_count
26+
exp: sw_go_finished_ignored_context_counter.sum(['service', 'instance']).increase('PT1M')
27+
- name: sw_go_possible_leaked_context_count
28+
exp: sw_go_possible_leaked_context_counter.sum(['source', 'service', 'instance']).increase('PT1M')
29+
- name: sw_go_interceptor_error_count
30+
exp: sw_go_interceptor_error_counter.sum(['plugin_name', 'service', 'instance']).increase('PT1M')
31+
- name: sw_go_tracing_context_execution_time_percentile
32+
exp: sw_go_tracing_context_performance.sum(['le', 'service', 'instance']).histogram().histogram_percentile([50,70,90,99])

Diff for: ‎oap-server/server-starter/src/main/resources/ui-initialized-templates/menu.yaml

+5
Original file line numberDiff line numberDiff line change
@@ -252,3 +252,8 @@ menus:
252252
description: The Java Agent for Apache SkyWalking, which provides the native tracing/metrics/logging/event/profiling abilities for Java projects.
253253
documentLink: https://skywalking.apache.org/docs/main/next/en/setup/backend/dashboards-so11y-java-agent/
254254
i18nKey: self_observability_java_agent
255+
- title: SkyWalking Go Agent
256+
layer: SO11Y_GO_AGENT
257+
description: The Go Agent for Apache SkyWalking, which provides the native tracing/metrics/logging abilities for Golang projects.
258+
documentLink: https://skywalking.apache.org/docs/main/next/en/setup/backend/dashboards-so11y-go-agent/
259+
i18nKey: self_observability_go_agent

Diff for: ‎oap-server/server-starter/src/main/resources/ui-initialized-templates/so11y_go_agent/so11y-instance.json

+202
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,202 @@
1+
[
2+
{
3+
"id": "Self-Observability-Go-Agent-Instance",
4+
"configuration": {
5+
"children": [
6+
{
7+
"x": 0,
8+
"y": 0,
9+
"w": 6,
10+
"h": 13,
11+
"i": "14",
12+
"type": "Widget",
13+
"widget": {
14+
"title": "Tracing Context Creation (Per Minute)",
15+
"tips": "The number of created tracing contexts, including a label created_by(value=sampler,propagated)."
16+
},
17+
"graph": {
18+
"type": "Line",
19+
"step": false,
20+
"smooth": false,
21+
"showSymbol": true,
22+
"showXAxis": true,
23+
"showYAxis": true
24+
},
25+
"expressions": [
26+
"meter_sw_go_created_tracing_context_count"
27+
]
28+
},
29+
{
30+
"x": 6,
31+
"y": 0,
32+
"w": 6,
33+
"h": 13,
34+
"i": "6",
35+
"type": "Widget",
36+
"widget": {
37+
"title": "Tracing Context Creation and Completion (Per Minute)",
38+
"tips": "The number of created tracing contexts and finished tracing contexts."
39+
},
40+
"graph": {
41+
"type": "Line",
42+
"step": false,
43+
"smooth": false,
44+
"showSymbol": true,
45+
"showXAxis": true,
46+
"showYAxis": true
47+
},
48+
"metricConfig": [
49+
{
50+
"label": "Creation"
51+
},
52+
{
53+
"label": "Completion"
54+
}
55+
],
56+
"expressions": [
57+
"aggregate_labels(meter_sw_go_created_tracing_context_count,sum)",
58+
"meter_sw_go_finished_tracing_context_count"
59+
]
60+
},
61+
{
62+
"x": 12,
63+
"y": 0,
64+
"w": 6,
65+
"h": 13,
66+
"i": "1",
67+
"type": "Widget",
68+
"widget": {
69+
"title": "Ignored Context Creation (Per Minute)",
70+
"tips": "The number of created ignored contexts, including a label created_by(value=sampler,propagated)."
71+
},
72+
"graph": {
73+
"type": "Line",
74+
"step": false,
75+
"smooth": false,
76+
"showSymbol": true,
77+
"showXAxis": true,
78+
"showYAxis": true
79+
},
80+
"expressions": [
81+
"meter_sw_go_created_ignored_context_count"
82+
]
83+
},
84+
{
85+
"x": 18,
86+
"y": 0,
87+
"w": 6,
88+
"h": 13,
89+
"i": "2",
90+
"type": "Widget",
91+
"widget": {
92+
"title": "Ignored Context Creation and Completion (Per Minute)",
93+
"tips": "The number of created ignored contexts and finished ignored contexts."
94+
},
95+
"graph": {
96+
"type": "Line",
97+
"step": false,
98+
"smooth": false,
99+
"showSymbol": true,
100+
"showXAxis": true,
101+
"showYAxis": true
102+
},
103+
"expressions": [
104+
"aggregate_labels(meter_sw_go_created_ignored_context_count,sum)",
105+
"meter_sw_go_finished_ignored_context_count"
106+
],
107+
"metricConfig": [
108+
{
109+
"label": "Creation"
110+
},
111+
{
112+
"label": "Completion"
113+
}
114+
]
115+
},
116+
{
117+
"x": 0,
118+
"y": 13,
119+
"w": 6,
120+
"h": 13,
121+
"i": "11",
122+
"type": "Widget",
123+
"widget": {
124+
"title": "Possible Leaked Context (Per Minute)",
125+
"tips": "The number of detected leaked contexts, including a label source(value=tracing, ignore)."
126+
},
127+
"graph": {
128+
"type": "Line",
129+
"step": false,
130+
"smooth": false,
131+
"showSymbol": true,
132+
"showXAxis": true,
133+
"showYAxis": true
134+
},
135+
"expressions": [
136+
"meter_sw_go_possible_leaked_context_count"
137+
],
138+
"metricConfig": [
139+
{
140+
"label": "count"
141+
}
142+
]
143+
},
144+
{
145+
"x": 12,
146+
"y": 13,
147+
"w": 12,
148+
"h": 13,
149+
"i": "8",
150+
"type": "Widget",
151+
"widget": {
152+
"title": "Interceptor Error Count (Per Minute)",
153+
"tips": "The number of errors happened in the interceptor logic, including the label plugin_name"
154+
},
155+
"graph": {
156+
"type": "Line",
157+
"step": false,
158+
"smooth": false,
159+
"showSymbol": true,
160+
"showXAxis": true,
161+
"showYAxis": true
162+
},
163+
"expressions": [
164+
"meter_sw_go_interceptor_error_count"
165+
],
166+
"metricConfig": [
167+
{
168+
"label": "count"
169+
}
170+
]
171+
},
172+
{
173+
"x": 6,
174+
"y": 13,
175+
"w": 6,
176+
"h": 13,
177+
"i": "15",
178+
"type": "Widget",
179+
"graph": {
180+
"type": "Line",
181+
"step": false,
182+
"smooth": false,
183+
"showSymbol": true,
184+
"showXAxis": true,
185+
"showYAxis": true
186+
},
187+
"widget": {
188+
"title": "Tracing Context Execution time (ms)",
189+
"tips": "For successfully finished tracing context, it measures every interceptor's time cost."
190+
},
191+
"expressions": [
192+
"relabels(meter_sw_go_tracing_context_execution_time_percentile,p='50,75,90,95,99',p='50,75,90,95,99')/1000000"
193+
]
194+
}
195+
],
196+
"layer": "SO11Y_GO_AGENT",
197+
"entity": "ServiceInstance",
198+
"name": "Self-Observability-Go-Agent-Instance",
199+
"isRoot": false
200+
}
201+
}
202+
]

Diff for: ‎oap-server/server-starter/src/main/resources/ui-initialized-templates/so11y_go_agent/so11y-service.json

+62
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,62 @@
1+
[
2+
{
3+
"id": "Self-Observability-Go-Agent-Service",
4+
"configuration": {
5+
"children": [
6+
{
7+
"x": 0,
8+
"y": 2,
9+
"w": 24,
10+
"h": 38,
11+
"i": "0",
12+
"type": "Widget",
13+
"graph": {
14+
"type": "InstanceList",
15+
"dashboardName": "Self-Observability-Go-Agent-Instance",
16+
"fontSize": 12
17+
},
18+
"metricConfig": [
19+
{
20+
"label": "Context Creation",
21+
"detailLabel": "context_creation",
22+
"unit": "Per Minute"
23+
},
24+
{
25+
"label": "Context Completion",
26+
"unit": "Per Minute",
27+
"detailLabel": "context_completion"
28+
}
29+
],
30+
"expressions": [
31+
"avg(aggregate_labels(meter_sw_go_created_tracing_context_count,sum)+aggregate_labels(meter_sw_go_created_ignored_context_count,sum))",
32+
"avg(meter_sw_go_finished_tracing_context_count+meter_sw_go_finished_ignored_context_count)"
33+
],
34+
"subExpressions": [
35+
"aggregate_labels(meter_sw_go_created_tracing_context_count,sum)+aggregate_labels(meter_sw_go_created_ignored_context_count,sum)",
36+
"meter_sw_go_finished_tracing_context_count+meter_sw_go_finished_ignored_context_count"
37+
]
38+
},
39+
{
40+
"x": 0,
41+
"y": 0,
42+
"w": 24,
43+
"h": 2,
44+
"i": "100",
45+
"type": "Text",
46+
"graph": {
47+
"fontColor": "theme",
48+
"backgroundColor": "theme",
49+
"content": "The self observability of SkyWalking Go Agent, which provides the abilities to measure the tracing performance and error statistics of plugins.",
50+
"fontSize": 14,
51+
"textAlign": "left",
52+
"url": "https://skywalking.apache.org/docs/main/next/en/setup/backend/dashboards-so11y-go-agent/"
53+
}
54+
}
55+
],
56+
"layer": "SO11Y_GO_AGENT",
57+
"entity": "Service",
58+
"name": "Self-Observability-Go-Agent-Service",
59+
"isRoot": true
60+
}
61+
}
62+
]

Diff for: ‎skywalking-ui

0 commit comments

Comments
 (0)