Skip to content

Conversation

@rodrigopex
Copy link
Contributor

@rodrigopex rodrigopex commented Jul 31, 2022

Pull request for adding zbus to Zephyr.

The Zephyr message bus - Zbus is a lightweight and flexible message bus enabling a simple way for threads to talk to one another. Threads can broadcast messages to all interested observers using zbus. Many-to-many communication is possible. The bus implements message-passing and publish/subscribe communication paradigms that enable threads to communicate synchronously or asynchronously through shared memory. The communication through zbus channel-based, where threads publish and read to and from using messages. Additionally, threads can observe channels and receive notifications when the channels are modified.

Note for reviewers - Remaining compliance warnings ⚠️

The only one I added is the _ZBUS_ASSERT macro which is necessary to verify the asserts coverage during tests. I have discussed it with @yperess about it. The remaining compliance errors seem to be related to checkpatch and/or clang-format issues.

-:517: WARNING:MACRO_WITH_FLOW_CONTROL: Macros with flow control statements should be avoided
#517: FILE: include/zephyr/zbus/zbus.h:104:
+#define _ZBUS_ASSERT(_cond, _fmt, ...)                                                             \
+	({                                                                                         \
+		if (!(_cond)) {                                                                    \
+			LOG_ERR(_fmt, ##__VA_ARGS__);                                              \
+			return -EFAULT;                                                            \
+		}                                                                                  \
+	})

-:602: WARNING:SUSPECT_CODE_INDENT: suspect code indent for conditional statements (8, 8)
#602: FILE: include/zephyr/zbus/zbus.h:189:
+	FOR_EACH_NONEMPTY_TERM(_ZBUS_OBS_EXTERN, (;), _observers)                                 \
+	_ZBUS_STRUCT_DECLARE(zbus_channel, _name) = {                                              \

-:604: WARNING:LONG_LINE: line length of 110 exceeds 100 columns
#604: FILE: include/zephyr/zbus/zbus.h:191:
+		.name = STRINGIFY(_name), .read_only = (_read_only),                                         \

Zbus presentation slides

Zbus documentation

Features table

Characteristic zbus
Made for Increase code quality by enabling reuse and increasing testability
Metaphor Messaging bus
Message definition time Compile-time
Message definition approach Decentralized, developers can define channels in different files
Message allocation style Static (compile-time). It is possible to use dynamically allocated messages. See dyn_channel sample
Message persistency Persistent, it still exists after processed
Message distribution pattern Hybrid: Message-passing (listeners) and Publish/subscribe (subscribers)
Subscription time Compile-time (it can be disabled in execution time by masking the subscriber) and runtime registration is possible
Message transmission style Direct transmission when using callbacks style. Two-factor for asynchronous transmission where, first, the event dispatcher sends the id of the changed channel to the subscriber, and second the subscriber decides if it reads the content or not. The transmission order is defined by the position of the subscriber on the subscribing list
Observer execution styles Synchronous for listeners (by callback), asynchronous for subscribers (by queue)
Event dispatcher There is no centralized event dispatcher. The dispatcher logic is executed by the publisher thread context. It enables the execution in different priority contexts and avoids preemption of the centralized approach
Implementation approach Static memory, mutexes, sys_slists, and queues
Use code generation (tools/scripts) No, only macros
Extension mechanism (you can add your functionality) Yes (see uart_bridge and remote_mock samples)
Maturity Feature-complete

This pull request aims to add zbus (message bus/event manager) discussed at RFC #45910 to Zephyr. With the implemented tests, it was possible to reach 90%+ of line and 80%+ of branch coverage. All the APIs were designed to be like the Zephyr's. For example, the channel publish is almost the same as a message queue put function. There is a claim/finish like the ring buffer and so on. I guess zbus would help several developers to create even more reusable and decoupled code. The pull request commits are separated into feature code, tests, and samples. I have made individual commits to help reviewers.

Base feature code commit:

  • zbus: zbus: Add message bus subsystem to Zephyr

Zbus tests:

  • tests: zbus: Add zbus unittests
  • tests: zbus: Add zbus integration tests
  • tests: zbus: Add zbus user_data usage tests
  • tests: zbus: Add zbus dynamic channels tests
  • tests: zbus: Add zbus runtime observer registration tests

Zbus samples:

  • samples: zbus: Add the hello world sample to zbus
  • samples: zbus: Add work queue sample
  • samples: zbus: Add dynamic channels sample
  • samples: zbus: Add the UART bridge sample
  • samples: zbus: Add the remote mock sample
  • samples: zbus: Add runtime observer registration sample
  • samples: zbus: Add benchmark sample

Activities during the PR:

  • Tests coverage (results: Line 93%, Branch 81%);
  • Compliance issues, there are some, but I guess they are not related to the submission;
  • General documentation;
  • Samples documentation.

@carlescufi carlescufi requested review from anangl and gmarull August 1, 2022 10:48
@rodrigopex rodrigopex force-pushed the rfc_subsys_ipc_zephyr_message_bus branch 5 times, most recently from 2611560 to fd3cb95 Compare August 1, 2022 20:45
Copy link
Member

@gmarull gmarull left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for your contribution, I've done a first pass review. My main concern is on how channels/messages are defined. Could you take a look at iterable sections? I think they would simplify things and would allow for a cleaner solution.

@rodrigopex
Copy link
Contributor Author

@gmarull

Thanks for your contribution, I've done a first pass review.

Thank you for the comments.

My main concern is on how channels/messages are defined. Could you take a look at iterable sections? I think they would simplify things and would allow for a cleaner solution.

I will take a look at that. This is, in fact, a tricky part of the code.

@rodrigopex rodrigopex force-pushed the rfc_subsys_ipc_zephyr_message_bus branch from fd3cb95 to 411927d Compare August 3, 2022 01:13
@rodrigopex
Copy link
Contributor Author

I have made some heavy changes to the way of generating channels. Now it is using Iterable Sections. The code is cleaner and smaller. I really good improvement. Thanks a lot, @gmarull, for the advice. There is a lot of work to do. I will address that in the next few days.

@gmarull
Copy link
Member

gmarull commented Aug 3, 2022

@rodrigopex please make sure CI passes and that commit structure is ok (for example, 9adf191 misses proper commit description)

@rodrigopex
Copy link
Contributor Author

rodrigopex commented Aug 3, 2022

Sorry for that. I will do. Could you take a look at the new way of defining channels?

@rodrigopex rodrigopex force-pushed the rfc_subsys_ipc_zephyr_message_bus branch 4 times, most recently from be9e00b to f465c3e Compare August 7, 2022 18:01
@rodrigopex
Copy link
Contributor Author

@gmarull The issues I am facing with CI now seem to be something other them my submission. Checkpatch and clang-format are fighting each other. The warnings below I could not remove:

-:455: WARNING:SUSPECT_CODE_INDENT: suspect code indent for conditional statements (8, 8)
#455: FILE: include/zephyr/zbus/zbus.h:163:
+	FOR_EACH_NONEMPTY_TERM(_ZBUS_OBS_EXTERN, (;), _observers)                                 \
+	STRUCT_SECTION_ITERABLE(zbus_channel, _name) = {                                           \

-:457: WARNING:LONG_LINE: line length of 110 exceeds 100 columns
#457: FILE: include/zephyr/zbus/zbus.h:165:
+		.name = STRINGIFY(_name), .flag = {false, _on_changed, _read_only},                          \

- total: 0 errors, 2 warnings, 3718 lines checked

The building errors seem related to iterable sections. I could not replicate those locally.

: && ccache /opt/toolchains/zephyr-sdk-0.14.2/xtensa-intel_apl_adsp_zephyr-elf/bin/xtensa-intel_apl_adsp_zephyr-elf-gcc   zephyr/CMakeFiles/zephyr_pre0.dir/misc/empty_file.c.obj -o zephyr/zephyr_pre0.elf  -fuse-ld=bfd  -Wl,-T  zephyr/linker_zephyr_pre0.cmd  -Wl,-Map=/__w/zephyr/zephyr/twister-out/intel_adsp_cavs15/samples/subsys/zbus/hello_world/sample.zbus.hello_world/zephyr/zephyr_pre0.map  -Wl,--whole-archive  app/libapp.a  zephyr/libzephyr.a  zephyr/arch/common/libarch__common.a  zephyr/arch/arch/xtensa/core/libarch__xtensa__core.a  zephyr/arch/arch/xtensa/core/startup/libarch__xtensa__core__startup.a  zephyr/lib/libc/minimal/liblib__libc__minimal.a  zephyr/lib/posix/liblib__posix.a  zephyr/soc/xtensa/intel_adsp/common/libintel_adsp_common.a  zephyr/subsys/zbus/libsubsys__zbus.a  zephyr/drivers/interrupt_controller/libdrivers__interrupt_controller.a  zephyr/drivers/timer/libdrivers__timer.a  modules/xtensa/libmodules_xtensa_hal.a  -Wl,--no-whole-archive  zephyr/kernel/libkernel.a  zephyr/CMakeFiles/offsets.dir/./arch/xtensa/core/offsets/offsets.c.obj  -L"/opt/toolchains/zephyr-sdk-0.14.2/xtensa-intel_apl_adsp_zephyr-elf/bin/../lib/gcc/xtensa-intel_apl_adsp_zephyr-elf/10.3.0"  -L/__w/zephyr/zephyr/twister-out/intel_adsp_cavs15/samples/subsys/zbus/hello_world/sample.zbus.hello_world/zephyr  -lgcc  zephyr/arch/common/libisr_tables.a  zephyr/soc/xtensa/intel_adsp/common/libintel_adsp_common.a  -no-pie  -Wl,--gc-sections  -Wl,--build-id=none  -Wl,--sort-common=descending  -Wl,--sort-section=alignment  -Wl,-u,_OffsetAbsSyms  -Wl,-u,_ConfigAbsSyms  -nostdlib  -static  -Wl,-X  -Wl,-N  -Wl,--orphan-handling=warn  -Wl,--fatal-warnings && cd /__w/zephyr/zephyr/twister-out/intel_adsp_cavs15/samples/subsys/zbus/hello_world/sample.zbus.hello_world/zephyr && /usr/local/bin/cmake -E echo
/opt/toolchains/zephyr-sdk-0.14.2/xtensa-intel_apl_adsp_zephyr-elf/bin/../lib/gcc/xtensa-intel_apl_adsp_zephyr-elf/10.3.0/../../../../xtensa-intel_apl_adsp_zephyr-elf/bin/ld.bfd: warning: orphan section `._zbus_channel.static.acc_data' from `app/libapp.a(channels.c.obj)' being placed in section `._zbus_channel.static.acc_data'
/opt/toolchains/zephyr-sdk-0.14.2/xtensa-intel_apl_adsp_zephyr-elf/bin/../lib/gcc/xtensa-intel_apl_adsp_zephyr-elf/10.3.0/../../../../xtensa-intel_apl_adsp_zephyr-elf/bin/ld.bfd: warning: orphan section `._zbus_channel.static.version' from `app/libapp.a(channels.c.obj)' being placed in section `._zbus_channel.static.version'
/opt/toolchains/zephyr-sdk-0.14.2/xtensa-intel_apl_adsp_zephyr-elf/bin/../lib/gcc/xtensa-intel_apl_adsp_zephyr-elf/10.3.0/../../../../xtensa-intel_apl_adsp_zephyr-elf/bin/ld.bfd: warning: orphan section `._zbus_observer.static.my_subscriber' from `app/libapp.a(main.c.obj)' being placed in section `._zbus_observer.static.my_subscriber'
/opt/toolchains/zephyr-sdk-0.14.2/xtensa-intel_apl_adsp_zephyr-elf/bin/../lib/gcc/xtensa-intel_apl_adsp_zephyr-elf/10.3.0/../../../../xtensa-intel_apl_adsp_zephyr-elf/bin/ld.bfd: warning: orphan section `._zbus_observer.static.my_listener' from `app/libapp.a(main.c.obj)' being placed in section `._zbus_observer.static.my_listener'
/opt/toolchains/zephyr-sdk-0.14.2/xtensa-intel_apl_adsp_zephyr-elf/bin/../lib/gcc/xtensa-intel_apl_adsp_zephyr-elf/10.3.0/../../../../xtensa-intel_apl_adsp_zephyr-elf/bin/ld.bfd: zephyr/subsys/zbus/libsubsys__zbus.a(zbus.c.obj):(.literal.zbus_iterate_over_channels+0x0): undefined reference to `_zbus_channel_list_start'
/opt/toolchains/zephyr-sdk-0.14.2/xtensa-intel_apl_adsp_zephyr-elf/bin/../lib/gcc/xtensa-intel_apl_adsp_zephyr-elf/10.3.0/../../../../xtensa-intel_apl_adsp_zephyr-elf/bin/ld.bfd: zephyr/subsys/zbus/libsubsys__zbus.a(zbus.c.obj):(.literal.zbus_iterate_over_channels+0x4): undefined reference to `_zbus_channel_list_end'
/opt/toolchains/zephyr-sdk-0.14.2/xtensa-intel_apl_adsp_zephyr-elf/bin/../lib/gcc/xtensa-intel_apl_adsp_zephyr-elf/10.3.0/../../../../xtensa-intel_apl_adsp_zephyr-elf/bin/ld.bfd: zephyr/subsys/zbus/libsubsys__zbus.a(zbus.c.obj):(.literal.zbus_iterate_over_observers+0x0): undefined reference to `_zbus_observer_list_start'
/opt/toolchains/zephyr-sdk-0.14.2/xtensa-intel_apl_adsp_zephyr-elf/bin/../lib/gcc/xtensa-intel_apl_adsp_zephyr-elf/10.3.0/../../../../xtensa-intel_apl_adsp_zephyr-elf/bin/ld.bfd: zephyr/subsys/zbus/libsubsys__zbus.a(zbus.c.obj):(.literal.zbus_iterate_over_observers+0x4): undefined reference to `_zbus_observer_list_end'
collect2: error: ld returned 1 exit status
ninja: build stopped: subcommand failed.

The documentation is not ready yet and needs a lot of revision. I am working on that this week.

The code I guess it is in a stable version. Please take a look at that.

@rodrigopex rodrigopex requested a review from gmarull August 8, 2022 11:36
@rodrigopex rodrigopex force-pushed the rfc_subsys_ipc_zephyr_message_bus branch 2 times, most recently from 68b411c to ba3f794 Compare August 9, 2022 19:13
@rodrigopex rodrigopex marked this pull request as ready for review August 9, 2022 19:13
@rodrigopex rodrigopex changed the title [WIP] zbus: Add Zephyr message bus zbus: Add Zephyr message bus Aug 9, 2022
Add zbus message bus as a Zephyr subsystem. No message bus
or communication abstraction other than the usual (message queues,
mailboxes, etc.) enabled developers to implement event-driven systems in
Zephyr quickly. Zbus would fill that gap by providing the community with
a lightweight and flexible message bus. The implementation tries to be
closest as possible to the existing ones. We use the claim/finish
approach, and the API for publishing and reading channels are similar
in message queues. Zbus is about channels, messages, and observers.

Signed-off-by: Rodrigo Peixoto <[email protected]>
Add zbus subsystem test to verify read-only and hard channels.

Signed-off-by: Rodrigo Peixoto <[email protected]>
Add zbus subsystem test to verify threads integration based on usual
interaction, masking observers, and delayed interaction to verify semaphore
and queue event dispatcher timeout.

Signed-off-by: Rodrigo Peixoto <[email protected]>
Add zbus subsystem test to verify the user_data usage does not affect
the normal behavior.

Signed-off-by: Rodrigo Peixoto <[email protected]>
Add zbus subsystem test to verify the dynamic channel usage.
It demonstrates the use of static and dynamic external messages.

Signed-off-by: Rodrigo Peixoto <[email protected]>
Add zbus subsystem test to verify the channel's runtime observers list
usage. It demonstrates the use of dynamically add and remove observer to
and from channels at runtime.

Signed-off-by: Rodrigo Peixoto <[email protected]>
The hello world sample illustrates a simple way of using zbus. There is
a listener and a subscriber interacting each other by using a read-only
and a regular channel. The APIs are called by using the function
and macro variation.

Signed-off-by: Rodrigo Peixoto <[email protected]>
The work_queue sample illustrates three reaction styles available for
using zbus. The first is a listener that reacts by callback; use it should
for urgent reaction. The second is a listener that responds by callback;
instead of executing the code, it pushes a job to a work queue that will
be executed shortly but not immediately. The last one is the subscriber
that reacts using a queue; use it for an asynchronous reaction where the
developer would like to control the flow.

Signed-off-by: Rodrigo Peixoto <[email protected]>
The sample illustrates a way of using dynamically allocated messages in
static channels.

Signed-off-by: Rodrigo Peixoto <[email protected]>
The sample illustrates a way of send all the channel event to the host
by using the UART, a user_data portion, and the claim/finish APIs.

Signed-off-by: Rodrigo Peixoto <[email protected]>
The sample illustrates a way of exchange message with a host mock
running in a Python script. It can be used to improve the testability
and controllability of the tests.

Signed-off-by: Rodrigo Peixoto <[email protected]>
The sample illustrates a way of using the runtime observer registration
feature. With this sample the developer can understand how to use static
and runtime observer registration.

Signed-off-by: Rodrigo Peixoto <[email protected]>
The sample measures the time to transfer 256KB from the
producer to the consumers.

Signed-off-by: Rodrigo Peixoto <[email protected]>
@rodrigopex rodrigopex force-pushed the rfc_subsys_ipc_zephyr_message_bus branch from 7a08ade to 8c08b1e Compare November 14, 2022 11:48
@rodrigopex rodrigopex requested review from anangl and cfriedt and removed request for anangl and cfriedt November 14, 2022 12:41
@rodrigopex rodrigopex requested a review from cfriedt November 14, 2022 12:51
@nashif nashif merged commit 27ec69e into zephyrproject-rtos:main Nov 14, 2022
@rodrigopex
Copy link
Contributor Author

Thank you very much to everyone involved who gave suggestions and discussed the feature and pitfalls of the solution. I am happy to have that merged. Now let's search for bugs and fix them if they happen. Regards.

@M1cha
Copy link
Contributor

M1cha commented Nov 3, 2023

@rodrigopex How did you create those insanely pretty SVGs? 👀

@rodrigopex
Copy link
Contributor Author

@rodrigopex How did you create those insanely pretty SVGs? 👀

Thanks @M1cha. I used Figma. The assets and images are public. You can use that by follwing the link. Feel free to change. Regards.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.