Skip to content

Commit 8c08b1e

Browse files
rodrigopexrodrigopex
committed
samples: zbus: Add benchmark sample
The sample measures the time to transfer 256KB from the producer to the consumers. Signed-off-by: Rodrigo Peixoto <[email protected]>
1 parent d9d60f5 commit 8c08b1e

File tree

8 files changed

+521
-0
lines changed

8 files changed

+521
-0
lines changed

‎samples/subsys/zbus/benchmark/CMakeLists.txt‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
# SPDX-License-Identifier: Apache-2.0
2+
cmake_minimum_required(VERSION 3.20.0)
3+
4+
find_package(Zephyr REQUIRED HINTS $ENV{ZEPHYR_BASE})
5+
project(benchmark)
6+
7+
FILE(GLOB app_sources src/*.c)
8+
target_sources(app PRIVATE ${app_sources})

‎samples/subsys/zbus/benchmark/Kconfig‎

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
# Copyright (c) 2022 Rodrigo Peixoto <[email protected]>
2+
# SPDX-License-Identifier: Apache-2.0
3+
4+
config BM_MESSAGE_SIZE
5+
int "Message size"
6+
default 256
7+
8+
config BM_ONE_TO
9+
int "Number of consumers"
10+
default 8
11+
12+
config BM_ASYNC
13+
bool "Consuming in asynchronous mode"
14+
default false
15+
16+
source "Kconfig.zephyr"

‎samples/subsys/zbus/benchmark/README.rst‎

Lines changed: 132 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,132 @@
1+
.. _zbus-benchmark-sample:
2+
3+
Benchmark sample
4+
################
5+
6+
This sample implements an application to measure the time for sending 256KB from the producer to the consumers.
7+
8+
Building and Running
9+
********************
10+
11+
.. zephyr-app-commands::
12+
:zephyr-app: samples/subsys/zbus/dyn_channel
13+
:host-os: unix
14+
:board: qemu_cortex_m3
15+
:gen-args: -DCONFIG_BM_MESSAGE_SIZE=1 -DCONFIG_BM_ONE_TO=1 -DCONFIG_BM_ASYNC=0
16+
:goals: build run
17+
18+
Notice we have the following parameters:
19+
20+
* **CONFIG_BM_MESSAGE_SIZE** the size of the message to be transferred;
21+
* **CONFIG_BM_ONE_TO** number of consumers to send;
22+
* **CONFIG_BM_ASYNC** if the execution must be asynchronous or synchronous. Use y to async and n to sync;
23+
24+
Sample Output
25+
=============
26+
The result would be something like:
27+
28+
.. code-block:: console
29+
30+
*** Booting Zephyr OS build zephyr-v3.2.0 ***
31+
I: Benchmark 1 to 8: Dynamic memory, ASYNC transmission and message size 256
32+
I: Bytes sent = 262144, received = 262144
33+
I: Average data rate: 1872457.14B/s
34+
I: Duration: 140ms
35+
36+
@140
37+
38+
39+
Running the benchmark automatically
40+
===================================
41+
42+
There is a Robot script called ``benchmark_256KB.robot`` which runs all the input combinations as the complete benchmark.
43+
The resulting file, ''zbus_dyn_benchmark_256KB.csv`` is generated in the project root folder. It takes a long time to execute. In the CSV file, we have the following columns:
44+
45+
+------------+---------------------+--------------------------+---------------+-------------+-------------+
46+
| Style | Number of consumers | Message size (bytes) | Duration (ms) | RAM (bytes) | ROM (bytes) |
47+
+============+=====================+==========================+===============+=============+=============+
48+
| SYNC/ASYNC | 1,2,4,8 | 1,2,4,8,16,32,64,128,256 | float | int | int |
49+
+------------+---------------------+--------------------------+---------------+-------------+-------------+
50+
51+
The complete benchmark command using Robot framework is:
52+
53+
.. code-block:: console
54+
55+
robot --variable serial_port:/dev/ttyACM0 -d /tmp/benchmark_out benchmark_256KB.robot
56+
57+
An example of execution using the ``hifive1_revb`` board would generate a file like this:
58+
59+
.. code-block::
60+
61+
SYNC,1,1,8534.0,6856,17434
62+
SYNC,1,2,4469.0,6856,17440
63+
SYNC,1,4,2362.3333333333335,6856,17438
64+
SYNC,1,8,1307.6666666666667,6864,17448
65+
SYNC,1,16,768.6666666666666,6872,17478
66+
SYNC,1,32,492.0,6888,17506
67+
SYNC,1,64,391.0,6920,17540
68+
SYNC,1,128,321.0,6984,17600
69+
SYNC,1,256,258.0,7112,17758
70+
SYNC,2,1,4856.666666666667,6880,17490
71+
SYNC,2,2,2464.0,6880,17494
72+
SYNC,2,4,1367.0,6880,17494
73+
SYNC,2,8,778.6666666666666,6888,17504
74+
SYNC,2,16,477.0,6896,17534
75+
SYNC,2,32,321.0,6912,17562
76+
SYNC,2,64,266.0,6944,17592
77+
SYNC,2,128,203.0,7008,17662
78+
SYNC,2,256,188.0,7136,17814
79+
SYNC,4,1,3021.3333333333335,6920,17536
80+
SYNC,4,2,1505.3333333333333,6920,17542
81+
SYNC,4,4,860.0,6920,17542
82+
SYNC,4,8,521.3333333333334,6928,17552
83+
SYNC,4,16,328.0,6936,17582
84+
SYNC,4,32,227.0,6952,17606
85+
SYNC,4,64,180.0,6984,17646
86+
SYNC,4,128,164.0,7048,17710
87+
SYNC,4,256,149.0,7176,17854
88+
SYNC,8,1,2044.3333333333333,7000,17632
89+
SYNC,8,2,1002.6666666666666,7000,17638
90+
SYNC,8,4,586.0,7000,17638
91+
SYNC,8,8,383.0,7008,17648
92+
SYNC,8,16,250.0,7016,17674
93+
SYNC,8,32,203.0,7032,17708
94+
SYNC,8,64,156.0,7064,17742
95+
SYNC,8,128,141.0,7128,17806
96+
SYNC,8,256,133.0,7256,17958
97+
ASYNC,1,1,22187.666666666668,7312,17776
98+
ASYNC,1,2,11424.666666666666,7312,17782
99+
ASYNC,1,4,5823.0,7312,17778
100+
ASYNC,1,8,3071.0,7312,17790
101+
ASYNC,1,16,1625.0,7328,17832
102+
ASYNC,1,32,966.3333333333334,7344,17862
103+
ASYNC,1,64,578.0,7376,17896
104+
ASYNC,1,128,403.6666666666667,7440,17956
105+
ASYNC,1,256,352.0,7568,18126
106+
ASYNC,2,1,18547.333333333332,7792,18030
107+
ASYNC,2,2,9563.0,7792,18034
108+
ASYNC,2,4,4831.0,7792,18030
109+
ASYNC,2,8,2497.3333333333335,7792,18044
110+
ASYNC,2,16,1377.6666666666667,7824,18098
111+
ASYNC,2,32,747.3333333333334,7856,18132
112+
ASYNC,2,64,492.0,7920,18162
113+
ASYNC,2,128,321.0,8048,18232
114+
ASYNC,2,256,239.33333333333334,8304,18408
115+
ASYNC,4,1,16823.0,8744,18474
116+
ASYNC,4,2,8604.333333333334,8744,18480
117+
ASYNC,4,4,4325.666666666667,8744,18472
118+
ASYNC,4,8,2258.0,8744,18490
119+
ASYNC,4,16,1198.3333333333333,8808,18572
120+
ASYNC,4,32,696.0,8872,18610
121+
ASYNC,4,64,430.0,9000,18650
122+
ASYNC,4,128,289.0,9256,18714
123+
ASYNC,4,256,227.0,9768,18906
124+
ASYNC,8,1,15976.666666666666,10648,19366
125+
ASYNC,8,2,7929.666666666667,10648,19372
126+
ASYNC,8,4,4070.6666666666665,10648,19356
127+
ASYNC,8,8,2158.6666666666665,10648,19382
128+
ASYNC,8,16,1119.6666666666667,10776,19506
129+
ASYNC,8,32,619.6666666666666,10904,19566
130+
ASYNC,8,64,391.0,11160,19600
131+
ASYNC,8,128,273.0,11672,19686
132+
ASYNC,8,256,211.0,12696,19934

‎samples/subsys/zbus/benchmark/benchmark_256KB.robot‎

Lines changed: 83 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,83 @@
1+
*** Settings ***
2+
Library Process
3+
Library String
4+
Library SerialLibrary
5+
Library CSVLibrary
6+
7+
Suite Teardown Terminate All Processes kill=True
8+
9+
10+
*** Variables ***
11+
${csv_file} zbus_dyn_benchmark_256kb.csv
12+
${board} hifive1_revb
13+
${serial_port} /dev/ttyACM0
14+
15+
*** Tasks ***
16+
Clear Old CSV File
17+
Empty Csv File ${csv_file}
18+
19+
Zbus Benchmark
20+
FOR ${async} IN "n" "y"
21+
FOR ${consumers} IN 1 2 4 8
22+
FOR ${msg_size} IN 1 2 4 8 16 32 64 128 256
23+
Benchmark Report For message_size=${msg_size} one_to=${consumers} asynchronous=${async}
24+
END
25+
END
26+
END
27+
28+
29+
*** Keywords ***
30+
Run Memory Report
31+
[Arguments] ${type}
32+
${result} Run Process west build -t ${type}_report shell=True
33+
Should Be Equal As Integers ${result.rc} 0
34+
${mem} Get Substring ${result.stdout} -20
35+
${mem} Strip String ${mem}
36+
${mem} Convert To Integer ${mem}
37+
RETURN ${mem}
38+
39+
Measure Results
40+
${total} Set Variable 0
41+
Add Port ${serial_port} timeout=120 baudrate=115200
42+
Set Encoding ascii
43+
FOR ${count} IN RANGE 3
44+
${result} Run Process west flash shell=True
45+
Should Be Equal As Integers ${result.rc} 0
46+
${val} Read Until expected=@ encoding=ascii
47+
${val} Read Until encoding=ascii
48+
${val} Strip String ${val}
49+
${val} Convert To Integer ${val}
50+
${total} Evaluate ${total}+${val}
51+
END
52+
${duration} Evaluate ${total}/3.0
53+
RETURN ${duration}
54+
[Teardown] Delete All Ports
55+
56+
Benchmark
57+
[Arguments] ${message_size}=256 ${one_to}=1 ${asynchronous}=n
58+
${result} Run Process
59+
... west build -b ${board} -p always -- -DCONFIG_BM_MESSAGE_SIZE\=${message_size} -DCONFIG_BM_ONE_TO\=${one_to} -DCONFIG_BM_ASYNC\=${asynchronous}
60+
... shell=True
61+
Should Be Equal As Integers ${result.rc} 0
62+
${duration} Measure Results
63+
RETURN ${duration}
64+
65+
Benchmark Report For
66+
[Arguments] ${message_size}=256 ${one_to}=1 ${asynchronous}=n
67+
${duration} Benchmark message_size=${message_size} one_to=${one_to} asynchronous=${asynchronous}
68+
${ram_amount} Run Memory Report ram
69+
${rom_amount} Run Memory Report rom
70+
IF ${asynchronous} == "y"
71+
${async_str} Set Variable ASYNC
72+
ELSE
73+
${async_str} Set Variable SYNC
74+
END
75+
@{results} Create List
76+
... ${async_str}
77+
... ${one_to}
78+
... ${message_size}
79+
... ${duration}
80+
... ${ram_amount}
81+
... ${rom_amount}
82+
Log To Console \n${results}
83+
Append To Csv File ${csv_file} ${results}

‎samples/subsys/zbus/benchmark/prj.conf‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
CONFIG_LOG=y
2+
CONFIG_LOG_MODE_MINIMAL=y
3+
CONFIG_HEAP_MEM_POOL_SIZE=1024
4+
CONFIG_ASSERT=n
5+
CONFIG_ZBUS=y
6+
CONFIG_ZBUS_LOG_LEVEL_INF=y

‎samples/subsys/zbus/benchmark/sample.yaml‎

Lines changed: 41 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,41 @@
1+
sample:
2+
name: Benchmark
3+
tests:
4+
sample.zbus.benchmark_async:
5+
tags: zbus
6+
min_ram: 16
7+
filter: CONFIG_SYS_CLOCK_EXISTS
8+
harness: console
9+
harness_config:
10+
type: multi_line
11+
ordered: true
12+
regex:
13+
- "I: Benchmark 1 to 8: Dynamic memory, ASYNC transmission and message size 256"
14+
- "I: Bytes sent = 262144, received = 262144"
15+
- "I: Average data rate: (.*)B/s"
16+
- "I: Duration: (.*)ms"
17+
- "@(.*)"
18+
extra_configs:
19+
- CONFIG_BM_ONE_TO=8
20+
- CONFIG_BM_MESSAGE_SIZE=256
21+
- CONFIG_BM_ASYNC=y
22+
platform_exclude: qemu_x86 qemu_x86_64 qemu_riscv32_smp native_posix native_posix_64 qemu_riscv32_smp qemu_cortex_a53_smp qemu_riscv64_smp qemu_leon3 qemu_xtensa qemu_cortex_a53 qemu_riscv32 qemu_malta qemu_malta_be qemu_arc_hs6x qemu_riscv64 m2gl025_miv hifive_unleashed
23+
sample.zbus.benchmark_sync:
24+
tags: zbus
25+
min_ram: 16
26+
filter: CONFIG_SYS_CLOCK_EXISTS
27+
harness: console
28+
harness_config:
29+
type: multi_line
30+
ordered: true
31+
regex:
32+
- "I: Benchmark 1 to 8: Dynamic memory, SYNC transmission and message size 256"
33+
- "I: Bytes sent = 262144, received = 262144"
34+
- "I: Average data rate: (.*)B/s"
35+
- "I: Duration: (.*)ms"
36+
- "@(.*)"
37+
extra_configs:
38+
- CONFIG_BM_ONE_TO=8
39+
- CONFIG_BM_MESSAGE_SIZE=256
40+
- CONFIG_BM_ASYNC=n
41+
platform_exclude: qemu_x86 qemu_x86_64 qemu_riscv32_smp native_posix native_posix_64 qemu_riscv32_smp qemu_cortex_a53_smp qemu_riscv64_smp qemu_leon3 qemu_xtensa qemu_cortex_a53 qemu_riscv32 m2gl025_miv m2gl025_miv

0 commit comments

Comments
 (0)