Skip to content

Commit c1afc45

Browse files
ngphibangngphibang
committed
samples: subsys: Add libMP video examples
Add video examples for libMP which includes two pipelines: - camera source and display sink - camera source, video transform and display sink Signed-off-by: Phi Bang Nguyen <[email protected]> Signed-off-by: Trung Hieu Le <[email protected]>
1 parent 55969fa commit c1afc45

File tree

11 files changed

+438
-0
lines changed

11 files changed

+438
-0
lines changed

‎samples/subsys/libmp/libmp.rst‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
.. zephyr:code-sample-category:: LibMP
2+
:name: LibMP
3+
:show-listing:
4+
:glob: **/*

‎samples/subsys/libmp/video_examples/camera_display/CMakeLists.txt‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
CMAKE_MINIMUM_REQUIRED (VERSION 3.20.0)
2+
3+
find_package(Zephyr REQUIRED HINTS $ENV{ZEPHYR_BASE})
4+
5+
project(video_example)
6+
7+
target_sources(app PRIVATE ${CMAKE_CURRENT_SOURCE_DIR}/camera_display.c)

‎samples/subsys/libmp/video_examples/camera_display/README.rst‎

Lines changed: 77 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,77 @@
1+
.. zephyr:code-sample:: libmp_camera_display
2+
:name: Camera Display Example
3+
4+
A sample pipeline composed of 3 elements: a camera source, a capsfilter and a display sink.
5+
6+
Description
7+
***********
8+
9+
::
10+
11+
+-----------------+ +--------------+ +----------------+
12+
| Camera Source | --> | Capsfilter | --> | Display Sink |
13+
+-----------------+ +--------------+ +----------------+
14+
15+
This example demonstrates a pipeline consisting of 3 elements. The camera source element generates
16+
video frames. The capsfilter is used to enforce a video format and/or resolution and/or framerate.
17+
This element is optional, without it, the pipeline is still working but with the default negotiated
18+
format. The display sink then renders the video frames on the screen.
19+
20+
Requirements
21+
************
22+
23+
* A board with input camera support
24+
* A board with output display support
25+
* Sufficient RAM for video buffering
26+
27+
This sample has been tested on the following boards:
28+
29+
- :zephyr:board:`native_sim`
30+
31+
- :zephyr:board:`mimxrt1170_evk`
32+
33+
Building and Running
34+
********************
35+
36+
For :zephyr:board:`native_sim`, build the sample with the following command:
37+
38+
.. zephyr-app-commands::
39+
:zephyr-app: samples/subsys/libmp/video_examples/camera_display
40+
:board: native_sim/native/64
41+
:goals: build
42+
:snippets: video-sw-generator
43+
:compact:
44+
45+
For :zephyr:board:`mimxrt1170_evk`, build the sample with the following command:
46+
47+
.. zephyr-app-commands::
48+
:zephyr-app: samples/subsys/libmp/video_examples/camera_display
49+
:board: mimxrt1170_evk@B/mimxrt1176/cm7
50+
:shield: nxp_btb44_ov5640,rk055hdmipi4ma0
51+
:goals: build
52+
:compact:
53+
54+
Wiring
55+
******
56+
57+
On :zephyr:board:`native_sim`, no wiring is needed since it uses a software video generator
58+
as source, however for the display output you need to ensure that your host system supports
59+
SDL2 to emulate the display.
60+
61+
62+
On :zephyr:board:`mimxrt1170_evk`, the OV5640 camera module should be plugged into the J2
63+
camera connector, the display module RK055HDMIPI4MA0 should be plugged into the J48 MIPI LCD connector.
64+
An additional USB cable is needed to connect USB debug port (J11) on the board to the host PC.
65+
66+
67+
Sample Output
68+
*************
69+
70+
The application will start the video pipeline and display the camera frames on the screen.
71+
Check for any error messages during initialization:
72+
73+
.. code-block:: console
74+
75+
*** Booting Zephyr OS build ***
76+
[00:00:00.321,000] <inf> mp_zdisp_sink: Display device: display-controller@40804000
77+
[00:00:00.373,000] <inf> mp_zvid_buffer_pool: Started streaming

‎samples/subsys/libmp/video_examples/camera_display/boards/mimxrt1170_evk_mimxrt1176_cm7.conf‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
CONFIG_VIDEO_BUFFER_POOL_SZ_MAX=3686800

‎samples/subsys/libmp/video_examples/camera_display/camera_display.c‎

Lines changed: 120 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,120 @@
1+
/*
2+
* Copyright 2025 NXP
3+
*
4+
* SPDX-License-Identifier: Apache-2.0
5+
*/
6+
7+
#include <zephyr/drivers/video-controls.h>
8+
#include <zephyr/kernel.h>
9+
#include <zephyr/logging/log.h>
10+
11+
#include <mp.h>
12+
13+
LOG_MODULE_REGISTER(main);
14+
15+
#define LOG_LEVEL LOG_LEVEL_DBG
16+
17+
#define PIPELINE_ID 0
18+
#define CAM_SRC_ID 1
19+
#define CAPS_FILTER_ID 2
20+
#define DISP_SINK_ID 3
21+
22+
int main(void)
23+
{
24+
int ret;
25+
26+
/* Create a new pipeline */
27+
struct mp_element *pipeline = mp_pipeline_new(PIPELINE_ID);
28+
29+
if (pipeline == NULL) {
30+
goto err;
31+
}
32+
33+
/* Create elements */
34+
struct mp_element *source = mp_element_factory_create(MP_ZVID_SRC_ELEM, CAM_SRC_ID);
35+
36+
if (source == NULL) {
37+
goto err;
38+
}
39+
40+
struct mp_element *caps_filter =
41+
mp_element_factory_create(MP_CAPS_FILTER_ELEM, CAPS_FILTER_ID);
42+
43+
if (caps_filter == NULL) {
44+
goto err;
45+
}
46+
47+
struct mp_element *sink = mp_element_factory_create(MP_ZDISP_SINK_ELEM, DISP_SINK_ID);
48+
49+
if (sink == NULL) {
50+
goto err;
51+
}
52+
53+
/* Set elements' properties */
54+
ret = mp_object_set_properties(MP_OBJECT(source), PROP_NUM_BUFS, 300, VIDEO_CID_HFLIP, 1,
55+
PROP_LIST_END);
56+
if (ret < 0) {
57+
goto err;
58+
}
59+
60+
struct mp_caps *filtered_caps =
61+
mp_caps_new("video/x-raw", "width", MP_TYPE_UINT, 320, "height", MP_TYPE_UINT, 240,
62+
"framerate", MP_TYPE_UINT_FRACTION, 30, 1, NULL);
63+
64+
if (filtered_caps == NULL) {
65+
goto err;
66+
}
67+
68+
ret = mp_object_set_properties(MP_OBJECT(caps_filter), PROP_CAPS, filtered_caps,
69+
PROP_LIST_END);
70+
mp_caps_unref(filtered_caps);
71+
if (ret < 0) {
72+
goto err;
73+
}
74+
75+
/* Add elements to the pipeline - order does not matter */
76+
if (mp_bin_add(MP_BIN(pipeline), source, caps_filter, sink, NULL) == false) {
77+
LOG_ERR("Failed to add elements");
78+
goto err;
79+
}
80+
81+
/* Link elements together - order does matter */
82+
if (mp_element_link(source, caps_filter, sink, NULL) == false) {
83+
LOG_ERR("Failed to link elements");
84+
goto err;
85+
}
86+
87+
/* Start playing */
88+
if (mp_element_set_state(pipeline, MP_STATE_PLAYING) != MP_STATE_CHANGE_SUCCESS) {
89+
LOG_ERR("Failed to start pipeline");
90+
goto err;
91+
}
92+
93+
/* Handle message from the pipeline */
94+
struct mp_bus *bus = mp_element_get_bus(pipeline);
95+
/* Wait until an Error or an EOS - blocking */
96+
struct mp_message *msg = mp_bus_pop_msg(bus, MP_MESSAGE_ERROR | MP_MESSAGE_EOS);
97+
98+
if (msg != NULL) {
99+
switch (MP_MESSAGE_TYPE(msg)) {
100+
case MP_MESSAGE_ERROR:
101+
LOG_INF("ERROR message from element %d", msg->src->id);
102+
break;
103+
case MP_MESSAGE_EOS:
104+
LOG_INF("EOS message from element %d", msg->src->id);
105+
break;
106+
default:
107+
LOG_ERR("Unexpected message from element %d", msg->src->id);
108+
break;
109+
}
110+
}
111+
112+
/* TODO: Stop pipeline and free allocated resources */
113+
mp_message_destroy(msg);
114+
115+
return 0;
116+
117+
err:
118+
LOG_ERR("Aborting sample");
119+
return 0;
120+
}

‎samples/subsys/libmp/video_examples/camera_display/prj.conf‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
CONFIG_LOG=y
2+
3+
CONFIG_MP=y
4+
CONFIG_MP_CAPSFILTER=y
5+
CONFIG_MP_PLUGIN_ZVID=y
6+
CONFIG_MP_PLUGIN_ZDISP=y
7+
8+
CONFIG_VIDEO=y
9+
CONFIG_DISPLAY=y
10+
11+
CONFIG_SHELL=y
12+
CONFIG_DEVICE_SHELL=y
13+
14+
CONFIG_HEAP_MEM_POOL_SIZE=10000

‎samples/subsys/libmp/video_examples/camera_transform_display/CMakeLists.txt‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
CMAKE_MINIMUM_REQUIRED (VERSION 3.20.0)
2+
3+
find_package(Zephyr REQUIRED HINTS $ENV{ZEPHYR_BASE})
4+
5+
project(video_example)
6+
7+
target_sources(app PRIVATE ${CMAKE_CURRENT_SOURCE_DIR}/camera_transform_display.c)

‎samples/subsys/libmp/video_examples/camera_transform_display/README.rst‎

Lines changed: 59 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,59 @@
1+
.. zephyr:code-sample:: libmp_camera_transform_display
2+
:name: Camera Transform Display Example
3+
4+
A sample pipeline composed of 4 elements: a camera source, a capsfilter,
5+
a video transform and a display sink.
6+
7+
Description
8+
***********
9+
10+
::
11+
12+
+-----------------+ +--------------+ +-------------------+ +----------------+
13+
| Camera Source | --> | Capsfilter | --> | Video Transform | --> | Display Sink |
14+
+-----------------+ +--------------+ +-------------------+ +----------------+
15+
16+
This example demonstrates a pipeline consisting of 4 elements. The camera source element generates
17+
video frames. The capsfilter is used to enforce a video format and/or resolution and/or framerate.
18+
This element is optional, without it, the pipeline is still working but with the default negotiated
19+
format. The video transformer then processes the video frames and forwards them to the display sink
20+
to render them on the screen.
21+
22+
Requirements
23+
************
24+
25+
* A board with input camera support
26+
* A board with output display support
27+
* Sufficient RAM for video buffering
28+
29+
This sample has been tested on the following boards:
30+
31+
- :zephyr:board:`mimxrt1170_evk`
32+
33+
Building and Running
34+
********************
35+
36+
For :zephyr:board:`mimxrt1170_evk`, build this sample application with the following commands:
37+
38+
.. zephyr-app-commands::
39+
:zephyr-app: samples/subsys/libmp/video_examples/camera_transform_display
40+
:board: mimxrt1170_evk@B/mimxrt1176/cm7
41+
:shield: nxp_btb44_ov5640,rk055hdmipi4ma0
42+
:goals: build flash
43+
:compact:
44+
45+
Wiring
46+
******
47+
On :zephyr:board:`mimxrt1170_evk`, the OV5640 camera module should be plugged into the J2
48+
camera connector, the display module RK055HDMIPI4MA0 should be plugged into the J48 MIPI LCD connector.
49+
An additional USB cable is needed to connect USB debug port (J11) on the board to the host PC.
50+
51+
52+
Sample Output
53+
*************
54+
55+
.. code-block:: console
56+
57+
*** Booting Zephyr OS build ***
58+
[00:00:00.321,000] <inf> mp_zdisp_sink: Display device: display-controller@40804000
59+
[00:00:00.373,000] <inf> mp_zvid_buffer_pool: Started streaming

‎samples/subsys/libmp/video_examples/camera_transform_display/boards/mimxrt1170_evk_mimxrt1176_cm7.conf‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
CONFIG_VIDEO_BUFFER_POOL_SZ_MAX=3686800
2+
CONFIG_VIDEO_BUFFER_POOL_NUM_MAX=3

0 commit comments

Comments
 (0)