[BUG] ping failed and rtps test failed. #12

ZhenshengLee · 2022-04-07T13:04:33Z

Thanks for all authors! But I just cannot reproduce the behavior of the ertps-stm32.

Here is the issue.

Env

hardware: stm32f767zi
toolchain: gcc8-2019-q3
cubeIDE: v6.4 with FW_V7 v1.16.0
network: ubuntu2004(192.168.1.104), stm32(192.168.1.103), router(192.168.1.1)
source: ZhenshengLee@3b983a3

Steps

change setting according to https://github.com/embedded-software-laboratory/embeddedRTPS-STM32/blob/master/readme.md

delete auto-generated Src/main.c (not main.cpp)
edit Middlewares/Third_Party/LwIP/system/arc/cc.h l41 as follows
//#define LWIP_PROVIDE_ERRNO
#define LWIP_ERRNO_INCLUDE <sys/errno.h>
edit Drivers/STM32F7xx_HAL_Driver/Src/stm32f7xx_eth.c l1877 as follows
macinit.PromiscuousMode = ETH_PROMISCUOUS_MODE_ENABLE;

change thread prio in stm32/rtps/include/rtps/config.h

       const uint8_t SPDP_WRITER_PRIO = 24;
        const uint8_t SPDP_MAX_NUMBER_FOUND_PARTICIPANTS = 5;
        const uint8_t SPDP_MAX_NUM_LOCATORS = 5;
        const Duration_t SPDP_LEASE_DURATION = {100, 0};

        const int MAX_NUM_UDP_CONNECTIONS = 10;

        const int THREAD_POOL_NUM_WRITERS = 1;
        const int THREAD_POOL_NUM_READERS = 1;
        const int THREAD_POOL_WRITER_PRIO = 24;
        const int THREAD_POOL_READER_PRIO = 24;

change ip setting in stm32/rtps/thirdparty/lwip/lwipcfg.h

#define LWIP_PORT_INIT_IPADDR(addr)   IP4_ADDR((addr), 192,168,1,103)
#define LWIP_PORT_INIT_GW(addr)       IP4_ADDR((addr), 192,168,1,1)
#define LWIP_PORT_INIT_NETMASK(addr)  IP4_ADDR((addr), 255,255,255,0)

change ip setting in stm32/rtps/include/rtps/config.h

const std::array<uint8_t, 4> IP_ADDRESS = {192,168,1,103};  // Needs to be set in lwipcfg.h too.

change ip setting in stm32/Src/lwip.c

void MX_LWIP_Init(void)
{
  /* IP addresses initialization */
  IP_ADDRESS[0] = 192;
  IP_ADDRESS[1] = 168;
  IP_ADDRESS[2] = 1;
  IP_ADDRESS[3] = 103;
  NETMASK_ADDRESS[0] = 255;
  NETMASK_ADDRESS[1] = 255;
  NETMASK_ADDRESS[2] = 255;
  NETMASK_ADDRESS[3] = 0;
  GATEWAY_ADDRESS[0] = 192;
  GATEWAY_ADDRESS[1] = 168;
  GATEWAY_ADDRESS[2] = 1;
  GATEWAY_ADDRESS[3] = 1;

add printf to serial according to microROS project https://github.com/micro-ROS/freertos_apps/blob/galactic/microros_nucleo_f767zi_extensions/Src/main.c

Expected

../embedded_rtps_test show something/

Actuall

../embedded_rtps_test waiting

Creating RTPS Participant...
Creating Participant...
Created Participant succesfully.
Creating RTPS reader on topic:TOLINUX
Creating FastRTPS Writer on topic TOSTM
Successfully created writer.
Waiting for reader match with STM32...

ping failed
arp -a show something with 192.168.1.103

? (169.254.169.254) at <incomplete> on enx000ec6cac70c
? (192.168.1.105) at 00:e0:4c:36:0f:51 [ether] on enx000ec6cac70c
? (192.168.1.101) at <incomplete> on enx000ec6cac70c
_gateway (192.168.157.254) at 00:00:5e:00:01:02 [ether] on wlxe84e0663b980
? (192.168.1.103) at <incomplete> on enx000ec6cac70c

disable the wifi makes no different of the result.

wireshark capturing no packets from 192.168.1.103
I added the tasklist function to show the thread report below

The text was updated successfully, but these errors were encountered:

akampmann · 2022-04-08T12:45:01Z

Hi,

do I understand correctly that you are not able to ping the STM?

If you are unable to ping the STM then something with your network or lwip config on the STM side is wrong. Can run your project without starting embeddedRTPS (comment out

embeddedRTPS-STM32/stm32/Src/main.cpp

Line 535 in f21d3fa

startRTPStest();

) and see if you are able to ping the STM?

ZhenshengLee · 2022-04-10T00:23:33Z

@akampmann thanks for your quick reply!

do I understand correctly that you are not able to ping the STM?

Yes, ping 192.168.1.103(stm32) outputs unreachable

If you are unable to ping the STM then something with your network or lwip config on the STM side is wrong. Can run your project without starting embeddedRTPS

Yes, I can ping the stm32 without starting embeddedRTPS, is there something wrong with the prio setting of threads? I've print the task info above.

This behavior happens also in microROS-ertps demo in https://github.com/micro-ROS/micro_ros_stm32cubemx_utils
this issue is related micro-ROS/micro_ros_stm32cubemx_utils#46

As for this repo, I've changed some code, but I believe the changing is about

static ip resetting
cubemx upgrade
print2serial and gettaskinfo function.

You can check with this ZhenshengLee@3b983a3

Thanks.

akampmann · 2022-04-11T07:53:40Z

I will have a look and come back to you. I haven't been running embeddedRTPS on the STM32 code for a while. I've been mainly developing for the Xilinx UltraScale+ Cortex R5.

akampmann · 2022-04-20T12:40:33Z

@ZhenshengLee I've created a new branch that tracks the latest embeddedRTPS, CubeIDE, fw v1.16.2, FastDDS v2.3.2 (probably also works with the latest version):

https://github.com/embedded-software-laboratory/embeddedRTPS-STM32/tree/feature/track_latest

STM32 is now able to talk again to the Linux side, could you give it a try on your side?

I seem to have sporadic issues with pinging the STM, but that occurs regardless of including embeddedRTPS, please let me know if you observe the same behavior.

jayvanmali · 2022-04-28T14:56:37Z

Hi @akampmann

Great Library! Extremely cool to see the RTPS-DDS running on a Micro-controller

I have been trying to get the example project working (the new branch called /feature/track_latest) and i have had some issues trying to get the example working. I have used the branch as is and set my Unix machine Networking Information to:

Address: 192.168.1.104
Network: 255.255.0.0
Gateway: 192.168.1.104
DNS: 192.168.1.104

I have also been using the latest version of FastDDS

I have connected the STM32F7 directly to my computer and noticed that it was hanging when it was trying to match the publisher and subscriber.

I did some debugging and found that it was failing to join IGMP multicast group (The printf verbose debugging is extremely useful). Inside the STM32F7 in the stm32f7xx_hal_eth.c file i had to change

macinit.PromiscuousMode = ETH_PROMISCUOUS_MODE_DISABLE;
macinit.MulticastFramesFilter = ETH_MULTICASTFRAMESFILTER_PERFECT;

to
macinit.PromiscuousMode = ETH_PROMISCUOUS_MODE_ENABLE;
macinit.MulticastFramesFilter = ETH_MULTICASTFRAMESFILTER_NONE;

After changing this i was able to successfully join the IGMP group however i now get the error "ParticipantProxyData deserializtaion failed" when the handleSPDPPackage function is called.

I haven't found the cause of the "ParticipantProxyData deserializtaion failed" but am continuing to debug. If you have any comments on potential causes or tests to help narrow the problem down it would be great to hear them!

Cheers

Jay

ZhenshengLee · 2022-04-29T06:38:48Z

STM32 is now able to talk again to the Linux side, could you give it a try on your side?

@akampmann Thanks for your quick reply!

I am on my vacation now, I'll give it a test in 10 days.

akampmann · 2022-04-29T12:03:32Z

@jayvanmali thank you for trying out the new branch and providing feedback. I'm curious why the changes to the MAC configuration were not necessary on my side.

Regarding the deserialization failure: Could you please try FastDDS v2.3.2? That is the exact version that I tested on my side. There are probably changes to the SPDP messages in newer FastDDS version which have not been incorporated in embeddedRTPS yet. Such changes were also necessary in the past. I will try to find time to look into this. Meanwhile, could you try 2.3.2 on your side and come back to me?

jayvanmali · 2022-05-02T21:12:25Z

Hi @akampmann, Thanks for your prompt reply.

Im currently away at the moment, but will test it with the FastDDS v2.3.2 version when i'm back (with in a day or so).

In regards to the MAC configuration changes, Once i get the example working, I will change it back to the same settings are provided in the example and see if it works. This should help understand the MAC configuration change.

Cheers

Jay

jayvanmali · 2022-05-04T10:01:36Z

Hi @akampmann

So i have installed FastDDS v2.3.4 (I couldn't find the files for v2.3.2) and i still get the error:

[SPDP] ParticipantProxyData deserializtaion failed

Just to be safe, i cloned the files from GitHub (And swapped to the /feature/track_latest branch) and did the following (Determined by trial and error as well as the github read-me):

Setup my Unix IP to 192.168.1.104.
Deleted the G_Test folder (It was causing compilation issues).
Changed the name if the file config_stm to config and updated the IP address to 192.168.1.103 (The STM32 IP set by LwIP).
Added the IGMP flag to netif->flags in ethernetif.
Changed SPDP_WRITER_PRIO, THREAD_POOL_WRITER_PRIO, THREAD_POOL_READER_PRIO to be 24.
Updated Drivers/STM32F7xx_HAL_Driver/Src/stm32f7xx_eth.c to change:

macinit.PromiscuousMode = ETH_PROMISCUOUS_MODE_DISABLE;
macinit.MulticastFramesFilter = ETH_MULTICASTFRAMESFILTER_PERFECT;

to

macinit.PromiscuousMode = ETH_PROMISCUOUS_MODE_ENABLE;
macinit.MulticastFramesFilter = ETH_MULTICASTFRAMESFILTER_NONE;

Here is a copy of the log from terminal: debuglog.txt

If you have any comments on potential causes or tests to help narrow the problem down it would be great to hear them!

I'm still making my way through debugging and will let you know when i manage to get it working!

Cheers

Jay

akampmann · 2022-05-05T14:12:58Z

@jayvanmali

this has most likely to do with the recent changes introduced that avoids adding remote locators that are not on the same subnet as embeddedRTPS. Could you please try to change this line to:

https://github.com/embedded-software-laboratory/embeddedRTPS/blob/68915c8ccc744db171a202182dce2ef55b7728ca/src/discovery/ParticipantProxyData.cpp#L187

to

return true;

The function returns false to indicate that the locator is not added, but that return is incorrectly causes the deserialization to stop. It worked on my side because since the first remote locator transmitted by remote participants was on the local network....

Do you mind confirming that this fixes the issue on your side?

I will then create a fix on the master branch shortly.

akampmann · 2022-05-06T08:22:37Z

@jayvanmali

I've updated https://github.com/embedded-software-laboratory/embeddedRTPS-STM32/tree/feature/track_latest to incorporate the fix. It works for me on the Cortex R5 now. Would you mind giving it a try on your side?

jayvanmali · 2022-05-06T09:10:13Z

Hi @akampmann

Great! ill give it a test a bit later this afternoon, Thanks for looking into it so quickly!

ZhenshengLee · 2022-05-09T04:46:07Z

@ZhenshengLee I've created a new branch that tracks the latest embeddedRTPS, CubeIDE, fw v1.16.2, FastDDS v2.3.2 (probably also works with the latest version):

https://github.com/embedded-software-laboratory/embeddedRTPS-STM32/tree/feature/track_latest

STM32 is now able to talk again to the Linux side, could you give it a try on your side?

@akampmann

The code has been altered to https://github.com/ZhenshengLee/embeddedRTPS-STM32/tree/outdoor/feature/track_latest/ to get a better debug experience.

Tested with fastdds2.3.4, everything is OK
linux side receiving well, sending well
ping stm well
stm side, doesn't konw if it works well

issue

the serial output behavior(the callback of rtpstest thread) is abnormal

expected

received data from Linux is output per second

actual

a lot of received data from Linux flushes out for once

screenshot

see the timestamps from the ternimal

screenshot

comment

is the serial or the callback work abnoraml?

Great! ill give it a test a bit later this afternoon, Thanks for looking into it so quickly!

EDIT: the test happens in 2022-0509-08:41 of your timezone, after your branch being updated for @jayvanmali 's issue

zs@zs-3630:~$ TZ='Europe/Berlin' date
2022年 05月 09日 星期一 08:41:15 CEST

ZhenshengLee · 2022-05-09T06:15:12Z

@ZhenshengLee I've created a new branch that tracks the latest embeddedRTPS, CubeIDE, fw v1.16.2, FastDDS v2.3.2 (probably also works with the latest version):

https://github.com/embedded-software-laboratory/embeddedRTPS-STM32/tree/feature/track_latest

STM32 is now able to talk again to the Linux side, could you give it a try on your side?

@akampmann

issue

oh, hello_world coredump after around 10 minutes

screenshot

ZhenshengLee · 2022-05-09T06:22:31Z

I seem to have sporadic issues with pinging the STM, but that occurs regardless of including embeddedRTPS, please let me know if you observe the same behavior.

@akampmann

after a 2-hour test of ping, I think there's no issue of the ping function of the stm

here is my board and toolchain info:

nucleo stm32f767zi
stm32ide 1.9.0
gcc: 10.3-2021.10 (builtin of the ide 1.9.0)
stm32cubemx 6.2
stm32cubeFW_F7 v1.16.2

screenshot

ZhenshengLee · 2022-05-09T06:32:26Z

@akampmann

I think your new branch https://github.com/embedded-software-laboratory/embeddedRTPS-STM32/tree/feature/track_latest of work can be merged as the basic function(ping and rtpstest) has been recovered.

So, after the feature branch being merged into master

this issue be closed
two new issues be created to track the received data from Linux issue and "hello_world coredump" issue I mentioned above
@jayvanmali would you agree that the ParticipantProxyData deserializtaion failed issue being tracked standalone?

akampmann · 2022-05-09T11:53:27Z

issue

the serial output behavior(the callback of rtpstest thread) is abnormal

expected

received data from Linux is output per second

actual

a lot of received data from Linux flushes out for once

screenshot

see the timestamps from the ternimal
screenshot

comment

is the serial or the callback work abnoraml?

@ZhenshengLee Can you add a newline character at the end of the output? Maybe the output buffer is only cleared and transmitted once full or after occurrence of a new line:

printf("Received data from Linux\n");

@jayvanmali would you agree that the ParticipantProxyData deserializtaion failed issue being tracked standalone?

@ZhenshengLee @jayvanmali This should be fixed on the latest master.

oh, hello_world coredump after around 10 minutes

@ZhenshengLee Could you please run with gdb and provide the output of backtrace once it runs into the segmentation fault?

jayvanmali · 2022-05-09T15:14:40Z

Hi @akampmann @ZhenshengLee

I have tried the new commit and the error for ParticipantProxyData deserializtaion failed has disappeared, however i still cant seem to get an output on either side. Wireshark shows that both the Unix machine and the STM32 are sending RTPS packets to 239.255.0.1 so i am not completely sure why i am not getting an output. The debug log however seems to freeze just after the ./hello_world linux application is started so i suspect it may be something to do with the RTOS threads.

Ive attached the RTPS debug log: RTPS_DEBUG.txt

Im also curious to know why i have to adjust the stm32f7xx_eth.c and the ethernetif file to successfully join the IGMP group. @ZhenshengLee did you have this issue when running this example?

akampmann · 2022-05-09T19:23:40Z

@jayvanmali Are you running this by any chance in a virtual machine? Could you provide a wireshark recording of the entire interaction?

ZhenshengLee · 2022-05-10T03:02:49Z

Im also curious to know why i have to adjust the stm32f7xx_eth.c and the ethernetif file to successfully join the IGMP group. @ZhenshengLee did you have this issue when running this example?

Nope, during my test, I did not change anything in eth setting of the stm32.

jayvanmali · 2022-05-10T09:02:25Z

Hi @akampmann

I'm running this on a Linux laptop with a HP USB-C to RJ45 Adaptor as the laptop does not have a Ethernet Port.

I have attached the RTPS Log from the STM32F7 and the Wireshark Recording:

RTPS Log: STM32_Log.txt
Wireshark Log: Wireshark_Recording.zip

After it appears to freeze, the pinging does not work anymore, I am doing some debugging to workout where its locking up but i cant immediately see anything failing.

Be happy to try any suggestions you recommend.

Cheers

Jay

jayvanmali · 2022-05-10T09:06:20Z

Hi @ZhenshengLee

That is interesting, Are you able to confirm if your stm32f7xx_eth has promiscuous mode enabled?

macinit.PromiscuousMode = ETH_PROMISCUOUS_MODE_ENABLE;
macinit.MulticastFramesFilter = ETH_MULTICASTFRAMESFILTER_NONE;

Can you also check your ethernetif and see if you have the IGMP flag set: NETIF_FLAG_IGMP

if i don't enable these i cant seem to join and IGMP group.

Much Appreciated

Jay

akampmann · 2022-05-10T09:23:35Z

Hi @akampmann

I'm running this on a Linux laptop with a HP USB-C to RJ45 Adaptor as the laptop does not have a Ethernet Port.

I have attached the RTPS Log from the STM32F7 and the Wireshark Recording:

RTPS Log: STM32_Log.txt Wireshark Log: Wireshark_Recording.zip

After it appears to freeze, the pinging does not work anymore, I am doing some debugging to workout where its locking up but i cant immediately see anything failing.

Be happy to try any suggestions you recommend.

Cheers

Jay

@jayvanmali

Something is off with the IP address configuration. The subnet assumed by embeddedRTPS is wrong (it sends messages to 192.168.42.44 (thus it assumes that this locator is on its own subnet)), but IP packets are actually sent from 192.168.1.103.

Can you confirm that the IP addresses here:

embeddedRTPS-STM32/stm32/LWIP/App/lwip.c

Line 69 in 6f9e84a

IP_ADDRESS[0] = 192;

And here:

embeddedRTPS-STM32/stm32/Core/Inc/rtps/config.h

Line 39 in 6f9e84a

const std::array<uint8_t, 4> IP_ADDRESS = {

match?

Currently configuring the IP address is a bit messy, as it has to be changed in two places. That is mainly because there does not seem to be a consistent way of configuring the IP address of the lwIP interface on different platforms (its different on the Cortex R5). I will have to add an assert at some point to warn of this misconfiguration during embeddedRTPS init.

takasehideki · 2022-05-14T09:23:02Z

This is just an FYI. I've confirmed feature/track_latest branch could work well in my environment as is (just changing the IP address). I have the utmost respect for @akampmann 's work!!

akampmann · 2022-05-16T14:33:16Z

@takasehideki thank you so much!

@jayvanmali did you have a chance to look at the IP config?

jayvanmali · 2022-05-17T07:30:42Z

Hi @akampmann

Apologies for the delayed response. I have been away for the past week and will back end of this week, Ill confirm the branch then and let you know my findings.

Cheers

akampmann · 2022-06-05T09:05:00Z

Hi @jayvanmali did you have a chance to try the fix?

jayvanmali · 2022-06-06T22:59:10Z

Hi @akampmann, apologies for the delay, An Illness left me out of action for a while. I'll test it within the next day and report back, I'll attach the trace log and Wireshark log as well

Cheers

Jay

jayvanmali · 2022-06-09T10:29:59Z

Hi @akampmann

I've started trying the fixed, but have had network issues on the F7 which i have narrowed down to hardware issues, I'll continue testing the fix when i get access to a second F7 (within a day or so).

Cheers

ZhenshengLee · 2022-06-09T16:23:59Z

@jayvanmali

That is interesting, Are you able to confirm if your stm32f7xx_eth has promiscuous mode enabled?

macinit.PromiscuousMode = ETH_PROMISCUOUS_MODE_ENABLE; macinit.MulticastFramesFilter = ETH_MULTICASTFRAMESFILTER_NONE;

yes, you can check in https://github.com/ZhenshengLee/ros2_mcu/blob/0cdedf3f9f07599bb58fbd42e61ac5204fdfddd0/libs/custom_stm32f767zi_ertps/Drivers/STM32F7xx_HAL_Driver/Src/stm32f7xx_hal_eth.c#L1877-L1878

Can you also check your ethernetif and see if you have the IGMP flag set: NETIF_FLAG_IGMP

yes, you can check in https://github.com/ZhenshengLee/ros2_mcu/blob/0cdedf3f9f07599bb58fbd42e61ac5204fdfddd0/libs/custom_stm32f767zi_ertps/LWIP/Target/ethernetif.c#L241

akampmann · 2022-07-26T14:27:59Z

Hi @akampmann

I've started trying the fixed, but have had network issues on the F7 which i have narrowed down to hardware issues, I'll continue testing the fix when i get access to a second F7 (within a day or so).

Cheers

hey @jayvanmali, did you have a chance to try with a new F7?

jayvanmali · 2022-08-11T23:29:05Z

hey @akampmann, Apologies for the radio silence, I have been away. Sadly due to supply issues, i have not been able to get an F7 but i do have a H723. I'm going to retry and get this working on the H7.

I'm fully back on deck so i have more time and ill keep you updated on the progress, ill also use the newest version of embeddedRTPS. I have done a quick integration of it but it seems to fail joining the IGMP group so i need to check the MAC filtering.

Cheers for everything so far!

jayvanmali · 2022-10-12T08:23:10Z

Hey @akampmann

I have managed to test the new code on a STM32H723, Sorry for the late response. I managed to fix the IP Config

I still seem to have a problem with the matching of the Publishers and Subscribers. I have attached the Wireshark Log and the Embedded RTPS DDS Log.

If you have some thoughts, Happy to try any solution.

Cheers

Jay

RTPS_Wireshark.zip

RTPS_LOG.txt

akampmann · 2022-10-12T14:00:09Z

@jayvanmali

Hi,

the problem is simple, your interface IP address does not match the embeddedRTPS config. You are sending data from 192.168.1.47, but the discovery advertises 192.168.1.103 as the IP address of your STM.

You need to adapt this parameter to 192.168.1.47

embeddedRTPS-STM32/stm32/Core/Inc/rtps/config.h

Line 39 in b9a8bed

const std::array<uint8_t, 4> IP_ADDRESS = {

Unfortunately, you are not the first one to encounter this issue. I will try to include a warning or assert in the embeddedRTPS code to catch this problem.

jayvanmali · 2022-10-13T10:57:39Z

Hi @akampmann

Good catch! I fixed that change, but i still seem to have the STM32 code remain waiting for the Publisher and Subscribers to match.

Any thoughts on what might be causing that?

Cheers

Jay

akampmann · 2022-10-13T12:44:47Z

Can you provide a Wireshark recording with the adapted config?

Are you able to ping the STM?

jayvanmali · 2022-10-13T13:16:51Z

Hey @akampmann

I can successfully ping the STM32H7 on 192.168.1.47.

Here are the new New Logs

MCU.txt
Wrieshark Log.zip

Here is the RTPS config file:
config.zip

What factors would stop the publishers and subscribers from matching?

Cheers

Jay

akampmann · 2022-10-13T18:15:19Z

Hi @jayvanmali

your machine does not respond to the STM32.

Here are some thoughts:

Are you using the FastDDS commit incorporate as a submodule in this repo?
Could you provide a trace from the beginning of the communication?
Also, do you have a router in between, that might block multicast? Could you try hooking up the STM directly with your machine?

Best regards
Alex

jayvanmali · 2022-10-20T13:25:11Z

Hey @akampmann

I think i have manged to get something working, It appears to be a issue with the version of FastDDS used in my ROS distribution (Locked to 2.6) and a second issue with the Firewall on my laptop

On a downgrade to 2.3 and just using FastDDS directly, it appears to be working.

Do you know what changes are potentially required to enable it working with FastDDS 2.6? Happy to look into it if you have any comments.

Best Regards

Jaynesh

akampmann · 2022-10-24T11:05:56Z

hey @jayvanmali

in the past, the main issues were increased discovery message sizes, due to additional fields. I will try to replicate this problem.

Have you tried turning verbosity on?

Best regards
Alex

jayvanmali · 2022-11-01T00:32:29Z

Hey @akampmann

I've turned on the debugging and am slowly going through the different versions of FastDDS and seeing when and if it fails and finding out why it does at those points, I will keep you updated with what i find.

One thing i have noticed is that when i try to add another participant in the same domain, i seem to keep getting a hard-fault even though my MAX_NUM_PARTICIPANTS in the config file is 2. I have made sure my FreeRTOS threads have a stack size of 5000. After stepping though the debugger it seems to fail at the "addBuiltInEndpoints". Do you have any idea of why this could be the case? or could i be missing a config setting to have multiple publishers and subscribers?

Cheers for all the help

Best Regards

Jay

akampmann · 2022-11-02T10:03:38Z

Hey @jayvanmali

I've finalized my thesis and now had some time to look into this.

I have tested embeddedRTPS both with FastDDS v2.6.0 and v2.8.0 - It worked with both versions on my side without requiring modifications to embeddedRTPS

Just to be clear: The issue on your side occurs in conjunction with ROS on top of embeddedRTPS, right? Can you confirm that FastDDS 2.6.0 and embeddedRTPS (without ROS2) works on your side? From my experience of using embeddedRTPS in a different context - these kind of issues have almost always been related to insufficient stack sizes of the reader workers. Depending on what you are doing in the context of the callback...

Can you try to use the following code snippet to print the stack consumption? Make sure to run it periodically or at least close to where you see your application crashing.

void printStackUsage() {
  UBaseType_t uxArraySize = uxTaskGetNumberOfTasks();
  unsigned long ulTotalRunTime, ulStatsAsPercentage;

  static TaskStatus_t* pxTaskStatusArray = static_cast<TaskStatus_t*>(
      pvPortMalloc(uxArraySize * sizeof(TaskStatus_t)));

  if (pxTaskStatusArray != NULL) {
    uxArraySize =
        uxTaskGetSystemState(pxTaskStatusArray, uxArraySize, &ulTotalRunTime);

    for (int x = 0; x < uxArraySize; x++) {
      printf("%s %u \n", pxTaskStatusArray[x].pcTaskName,
             static_cast<int>(
                 uxTaskGetStackHighWaterMark(pxTaskStatusArray[x].xHandle)));
    }
  }
}

Most likely you will have to increase (at least) the following stack sizes

const int THREAD_POOL_WRITER_STACKSIZE; 
const int THREAD_POOL_READER_STACKSIZE;

akampmann · 2022-11-02T10:54:20Z

@ZhenshengLee the core dump is also fixed on master, I was not using FastDDS correctly.

ZhenshengLee mentioned this issue May 9, 2022

Follow the latest update of embeddedRTPS #11

Closed

[BUG] ping failed and rtps test failed. #12

[BUG] ping failed and rtps test failed. #12

Comments

ZhenshengLee commented Apr 7, 2022

Env

Steps

Expected

Actuall

akampmann commented Apr 8, 2022

ZhenshengLee commented Apr 10, 2022

akampmann commented Apr 11, 2022 • edited Loading

akampmann commented Apr 20, 2022 • edited Loading

jayvanmali commented Apr 28, 2022

ZhenshengLee commented Apr 29, 2022

akampmann commented Apr 29, 2022 • edited Loading

jayvanmali commented May 2, 2022

jayvanmali commented May 4, 2022

akampmann commented May 5, 2022

akampmann commented May 6, 2022

jayvanmali commented May 6, 2022

ZhenshengLee commented May 9, 2022 • edited Loading

issue

expected

actual

screenshot

comment

ZhenshengLee commented May 9, 2022

issue

screenshot

ZhenshengLee commented May 9, 2022

screenshot

ZhenshengLee commented May 9, 2022

akampmann commented May 9, 2022 • edited Loading

issue

expected

actual

screenshot

comment

jayvanmali commented May 9, 2022

akampmann commented May 9, 2022

ZhenshengLee commented May 10, 2022

jayvanmali commented May 10, 2022

jayvanmali commented May 10, 2022

akampmann commented May 10, 2022 • edited Loading

takasehideki commented May 14, 2022

akampmann commented May 16, 2022

jayvanmali commented May 17, 2022

akampmann commented Jun 5, 2022

jayvanmali commented Jun 6, 2022

jayvanmali commented Jun 9, 2022

ZhenshengLee commented Jun 9, 2022

akampmann commented Jul 26, 2022

jayvanmali commented Aug 11, 2022

jayvanmali commented Oct 12, 2022

akampmann commented Oct 12, 2022

jayvanmali commented Oct 13, 2022

akampmann commented Oct 13, 2022 • edited Loading

jayvanmali commented Oct 13, 2022 • edited Loading

akampmann commented Oct 13, 2022

jayvanmali commented Oct 20, 2022

akampmann commented Oct 24, 2022

jayvanmali commented Nov 1, 2022

akampmann commented Nov 2, 2022

akampmann commented Nov 2, 2022

akampmann commented Apr 11, 2022 •

edited

Loading

akampmann commented Apr 20, 2022 •

edited

Loading

akampmann commented Apr 29, 2022 •

edited

Loading

ZhenshengLee commented May 9, 2022 •

edited

Loading

akampmann commented May 9, 2022 •

edited

Loading

akampmann commented May 10, 2022 •

edited

Loading

akampmann commented Oct 13, 2022 •

edited

Loading

jayvanmali commented Oct 13, 2022 •

edited

Loading