Basics of inter core communication on STM32

[marcoaccame]

This PR adds the basics of communication between the two cores of our dual core boards.

In the following post I attach a document which explains how everything works. The document also shows how I have used inter core communication to implement the request from the CM4 core to the CM7 core to print a string.

The features have been tested using a dedicated application in board\amc\examples\appl-test which exchanges data with the project board\amc2c\application with a specific dedicated macro.

[marcoaccame]

Inter core communication on STM32

The exchange of signals and data across the two cores of the H7 can be implemented using the available peripherals of the MPU: the HSEM with associated IRQHandler and a bank of shared memory.

In here I present the inter core communication utilities of the embot:hw framework developed on top of the available HW peripherals.

The simplest utility is the embot::hw::MTX which acts as a mutex which also offers notification across cores. This one is the brick that we can use for implementing more powerful services such as:

• more refined inter-core signaling with the embot::hw::icc::SIG;
• protected memory exchange with the embot::hw::icc:MEM;
• advanced memory exchange with notification (and possible ack) with the embot::hw::icc::LTR.

erDiagram
  
    "embot::hw" ||..|| MTX : contains
    "embot::hw" ||..|| "embot::hw::icc" : contains
    "embot::hw::icc" ||..|| SIG : contains
    "embot::hw::icc" ||..|| MEM : contains
    "embot::hw::icc" ||..|| LTR : contains

Loading

Figure. The inter core communication utilities inside embot::hw.

erDiagram
STM32 ||..||  "embot::hw::MTX" : allows
    STM32 {
        HSEM thirtytwo
        IRQHandler two
    }
 "Shared RAM" ||..|| "embot::hw::icc::MEM" : allows
    

Loading

Figure. Their HW requirements

Finally, I will show the first application: the CM4 sends strings to the CM7 core which prints them over SWO. We need one object on each core, the embot::hw::icc::printer::theClient and the embot::hw::icc::printer ::theServer which use the same embot::hw::icc::LTR.

The embot::hw::MTX

The embot::hw::MTX is a mutual exclusion HW utility that works across cores.

Usage

It offers two very important services: mutual exclusion and signaling.

• Mutual exclusion. A thread in a core can take possession of a embot::hw::MTX so that it cannot be taken by the other core at the same time. Note that the embot::hw::MTX offers mutual exclusions only across different cores, but not across threads running on the same core. For that, we need an RTOS mutex such as the embot::os::rtos::mutex_t or its C++ wrapper object embot::os::Mutex.

sequenceDiagram
    Core 1 ->> MTX: take()
    activate MTX
    activate Core 1
    Note left of Core 1: MTX is taken by Core 1
    Core 2 ->> MTX : take()
    %% activate Core 2
    Note right of Core 2: MTX is already taken, so Core 2 waits
    Core 1 ->> MTX: release()
    deactivate MTX
    deactivate Core 1
    %% deactivate Core 2
    activate MTX
    activate Core 2
    Note right of Core 2: MTX can now be taken by Core 2
    Core 2 ->> MTX : release()
    deactivate MTX
    deactivate Core 2    
    

Loading

• Signaling. A thread running on a given core can subscribe for the execution of a callback when the embot::hw::MTX is released by the other core. Note that the activation is technically possible also across threads running on the same core, but for that we typically use RTOS utilities such as embot::os::rtos::event_t etc.

  sequenceDiagram
  Core 1 ->> MTX: subscribe(callback)
  Note left of Core 1: we ask to execute callback() on release
  Core 2 ->> MTX: take()
  Core 2 ->> MTX: release()
  MTX -->> Core 1: signal via IRQHandler
  %% Core 1 -->> Core 1: 
  activate Core 1
  Note left of Core 1: the callback() is executed
  deactivate Core 1
  
  

  Loading

Dependencies

The emboth::hw::MTX is implemented with the following HW resources provided by STMicroelectronics: the HSEM peripheral which hosts up to 32 individual items and the associated IRQHandler.

erDiagram
  

    "embot::hw::MTX" ||..|| HSEM : uses
    "embot::hw::MTX" ||..|| IRQHandler : uses

Loading

Figure. The embot::hw::MTX uses the HSEM peripheral plus the associated IRQHandler.

The used API

We have implemented the embot::hw::MTX in the HW namespace, so we have followed the conventions which collects the functions inside a namespace with the first argument telling on which item to operate. The reason is that we cannot just create as many items as we want. We just can get them if available.

In here are the API of use.

namespace embot::hw {
     enum class MTX : uint8_t { one = 0, two = 1, three = 2, four = 3, five = 4, 
                                six = 5, seven = 6, eight = 7, nine = 8, ten = 9, 
                                eleven = 10, twelve = 11, thirteen = 12, fourteen = 13, fifteen = 14,
                                sixteen = 15, seventeen = 16, eighteen = 17, nineteen = 18, twenty = 19,
                                twentyone = 20, twentytwo = 21, twentythree = 22, twentyfour = 23, twentyfive = 24, 
                                twentysix = 25, twentyseven = 26, twentyeight = 27, twentynine = 28, thirty = 29, 
                                thirthyone = 30, maxnumberof = 31, none = 31 };   
} 

Code listing. We can use at most 31 embot::hw::MTX items of of the possible 32 HSEM in the H7 MPU. We have reserved one HSEM, the one with value 0, to synchronize the start of the CM4 vs the CM7.

namespace embot::hw::mtx {
            
    bool supported(embot::hw::MTX m);   
    result_t init(embot::hw::MTX m);
    bool initialised(embot::hw::MTX m);
    
    bool take(embot::hw::MTX m, embot::core::relTime timeout = embot::core::reltimeWaitForever);    
    void release(embot::hw::MTX m);
    bool check(embot::hw::MTX m);
    
    bool subscribe(embot::hw::MTX m, const embot::hw::Subscription &onrelease);     
        
} 

Code listing. If the embot::hw::MTX is supported and initialized by a core it can be used. The methods take() and release() are self-explanatory, check() returns false if the other core has taken the mutex, and subscribe() just .. executes a callback on the release of the mutex as described by embot::hw::Subscription.

namespace embot::hw {
            
    struct Subscription
    {
        enum class MODE : uint8_t { none = 0, oneshot = 1, permanent = 2 };       
        embot::core::Callback callback {};
        MODE mode {MODE::none};  
        constexpr Subscription() = default;  
        constexpr Subscription(const embot::core::Callback &cbk, MODE mo) : callback(cbk), mode(mo) {} 
        void clear() { callback.clear(); mode = MODE::none; }        
        void execute() 
        { 
            if(mode != MODE::none)
            {   
                callback.execute();            
                if(mode == MODE::oneshot) { clear(); } 
            } 
        }        
        constexpr bool isvalid() const { return callback.isvalid() && (MODE::none != mode); }
    };   
        
}   

Code listing. The object embot::hw::Subscription configures a callback to be executed in MODE::oneshot or MODE::permanent. It is used by one core so that it can be alerted when the other core releases the embot::hw::MTX.

The embot::hw::icc:SIG

The embot::hw::icc::SIG is a specialization of the embot::hw::MTX for the purpose of signaling.

Usage

They must be used in pairs. A core initializes a given item, say embot::hw::icc::SIG:one, as a transmitter (or receiver) and the other core must initialize the same item as the opposite, so receiver (or transmitter). Then the transmitter can set() the signal which will be notified to the receiver. The receiver can be notified by execution of a callback or by checking the notification in polling.

See following diagrams.

sequenceDiagram
    Core 1 ->> SIG: init(SIG::one, {DIR::rx})
    Core 1 ->> SIG: subscribe(SIG::one, {callback, MODE::permanent })
    Core 2 ->> SIG: init(SIG::one, {DIR::tx})
    Core 2 ->> SIG: set(SIG::one)
    %% SIG -->> Core 1: signal via IRQHandler
    SIG -->> Core 1: 
    activate Core 1
    Note left of Core 1: the callback() is executed straight away
    deactivate Core 1

Loading

Sequence Diagram. Subscription mode. Core 1 initializes the embot::hw::icc::SIG::one in reception mode and subscribes to execute a callback at reception of a signaling. Core 2 initializes the same embot::hw::icc::SIG::one in transmission mode and then it sends a signaling. At this point Core 1 is immediately notified by execution of a callback.

sequenceDiagram
    Core 1 ->> SIG: init(SIG::one, {DIR::rx})
    Core 1 ->> SIG: bool rx = check(SIG::one)
    Core 2 ->> SIG: init(SIG::one, {DIR::tx})
    activate Core 1
    Note left of Core 1: doing something
    deactivate Core 1
    Core 1 ->> SIG: bool rx = check(SIG::one)
    Core 2 ->> SIG: set(SIG::one)

    activate Core 1
    Note left of Core 1: doing something
    deactivate Core 1
    Core 1 ->> SIG: bool rx = check(SIG::one)
    SIG -->> Core 1: 
    activate Core 1
    Note left of Core 1: rx is true, so the SIG is cleared and activity is executed
    deactivate Core 1

Loading

Sequence Diagram. Polling mode. Core 1 initializes the embot::hw::icc::SIG::one in reception mode and starts to periodically check vs a received signal. Core 2 initializes the same embot::hw::icc::SIG::one in transmission mode. ad then it sends a signaling. Core 1 can take action only when it verifies the reception.

The embot::hw::icc::SIG has also a third experimental mode called DIR::txrx which is so far reserved for use by two threads of the same core. In this mode, one thread acts as a transmitter and the other as a receiver.

sequenceDiagram
    Thread 1 ->> SIG: init(SIG::one, {DIR::txrx})
    Note left of Thread 1: only one initialization
    Thread 1 ->> SIG: subscribe(SIG::one, {callback, MODE::permanent})
    Thread 2 ->> SIG: init(SIG::one, {DIR::tx})
    Thread 2 ->> SIG: set(SIG::one)
    %% SIG -->> Thread 1: signal via IRQHandler
    SIG -->> Thread 1: 
    activate Thread 1
    Note left of Thread 1: the callback() is executed straight away
    deactivate Thread 1

Loading

Sequence Diagram. DIR::txrx. To be used by two threads inside one core only

Dependencies

Each embot::hw::icc::SIG uses exactly one embot::hw::MTX.

erDiagram
  
    "embot::hw::icc::SIG" ||..|| "embot::hw::MTX" : uses

   

Loading

Figure. The embot::hw::icc::SIG uses exactly one embot::hw::MTX.

The used API

In here are the API of use.

namespace embot::hw::icc {
    enum class DIR : uint8_t { none = 0, tx = 1, rx = 2, txrx = 3 };
    
    enum class SIG : uint8_t {  one = 0, two = 1, three = 2, four = 3, five = 4, six = 5, 
                                seven = 6, eight = 7, 
                                none = 31, maxnumberof = 8 };
} 

Code listing. We can use at most 8 embot::hw::icc::SIG items.

namespace embot::hw::icc:;sig {
            
    struct Config
    {        
        DIR direction {DIR::none};       
        constexpr Config() = default;
        constexpr Config(DIR d) : direction(d) {}
    };
    
    bool supported(embot::hw::icc::SIG s);   
    result_t init(embot::hw::icc::SIG s, const Config &cfg);
    bool initialised(embot::hw::icc::SIG s);
    
    // -- transmitter methods. they can be used only for DIR::tx or Dir:txrx
    
    // one entity sends the signal set()
    bool set(embot::hw::icc::SIG s, embot::core::relTime timeout = embot::core::reltimeWaitForever);    
    
    // -- receiver methods.  they can be used only for DIR::rx or DIR::txrx
    
    // the entity which waits must subscribe 
    bool subscribe(embot::hw::icc::SIG s, const embot::hw::Subscription &onsignal);   
    // or it can check() in polling mode.   
    bool check(embot::hw::icc::SIG s, bool andclear = true);
    // we can also explicitly clear() the signal
    bool clear(embot::hw::icc::SIG s);   
        
} 

Code listing. The public interface of embot::hw::icc::SIG.

The embot::hw::icc:MEM

The embot::hw::icc::MEM used a embot::hw::MTX to protect access to a shared memory area.

Usage

They must be used in pairs. Each core initializes a given item, say embot::hw::icc::MEM:one, and then they begin using it for writing or for reading. Note that the mutual exclusion is guaranteed only between two cores, not between two threads of the same core.

See following diagrams.

sequenceDiagram
    Core 1 ->> MEM: init(MEM::one)
    Core 1 ->> MEM: set(MEM::one, {mem1, size})
    Note left of Core 1: it copies inside MEM::one size bytes of its own memory
    Core 2 ->> MEM: get(MEM::one, {mem2, size})
    Note right of Core 2: it copies inside its own memory size bytes from MEM::one
    

Loading

Sequence Diagram. Use of MEM in copy mode. Core 1 copies its own memory inside the shared memory owned byMEM::one and Core 2 retrieve it and copies that into its own local copy.

sequenceDiagram
    Core 1 ->> MEM: init(MEM::one)
    Core 1 ->> MEM: take(MEM::one)
    Core 1 ->> MEM: void *p = memory(MEM::one)
    activate Core 1
    Note left of Core 1: it directly manipulates the memory inside MEM::one
    Note left of Core 1: for instance with a push_back into a container  
    deactivate Core 1
    Core 1 ->> MEM: release(MEM::one)
    Core 2 ->> MEM: take(MEM::one)
    Core 2 ->> MEM: void *p = memory(MEM::one)
    activate Core 2
    Note right of Core 2: it directly manipulates the memory inside MEM::one
    Note right of Core 2: for instance by removing an item in front of a container  
    deactivate Core 2
    Core 2 ->> MEM: release(MEM::one)
    

Loading

Sequence Diagram. Use of MEM in modify mode. Core 1 locks the shared memory, changes it for instance by adding an item inside a container and then releases it.

Dependencies

Each embot::hw::icc::MEM uses exactly one embot::hw::MTX and a portion of shared memory.

erDiagram
  
    "embot::hw::icc::SIG" ||..|| "embot::hw::MTX" : uses
    "embot::hw::icc::SIG" ||..|| "shared RAM" : uses
   

Loading

Figure. The embot::hw::icc::MEM uses one embot::hw::MTX and a portion of shared memory.

The used API

In here are the API of use.

namespace embot::hw::icc {
    
    enum class MEM : uint8_t {  one = 0, two = 1, three = 2, four = 3, five = 4, six = 5, 
                                seven = 6, eight = 7, 
                                none = 31, maxnumberof = 8 };
} 

Code listing. We can use at most 8 embot::hw::icc::MEM items.

namespace embot::hw::icc::mem {
            
    bool supported(embot::hw::icc::MEM m); 
    size_t size(embot::hw::icc::MEM m);
    result_t init(embot::hw::icc::MEM m);
    bool initialised(embot::hw::icc::MEM m);
    
    // atomic set() / get()
    size_t set(embot::hw::icc::MEM m, const embot::core::Data &data, 
               embot::core::relTime timeout = embot::core::reltimeWaitForever);    
    size_t get(embot::hw::icc::MEM m, embot::core::Data &data, 
               embot::core::relTime timeout = embot::core::reltimeWaitForever); 
    
    // take(), modify memory(), release()
    // the usage is for instance ... manipulate a FIFO mapped inded the shared memory in mutual access mode  
    bool take(embot::hw::icc::MEM m, embot::core::relTime timeout = embot::core::reltimeWaitForever);
    void * memory(embot::hw::icc::MEM m);
    void release(embot::hw::icc::MEM m);  
        
} 

Code listing. The public interface of embot::hw::icc::MEM.

The embot::hw::icc:LTR

The embot::hw::icc::MEM modifies shared memory but does not alert the other core about a modification. For that we need an embot::hw::icc::LTR.

Usage

The embot::hw::icc::LTR allows to write in mutual exclusion mode a portion of shared memory and also to wait until the receiver has effectively received and read the memory. So it is effective in passing data that must be surely received.

Two reception modes are possible:

• by subscription, where the receiver register a callback so that it can be alerted that a read is required;
• by polling, where the receiver must continuously verify if a new letter is available before it can read it.

In here we show only the mode by subscription.

sequenceDiagram
    Core 1 ->> LTR: init(LTR::one, {DIR:tx})
    Core 2 ->> LTR: init(LTR::one, {DIR:rx})
    Core 2 ->> LTR: subscribe(LTR::one, {callback, MODE::permanent})
    Core 1 ->> LTR: post(LTR::one, {datacore1, size}, ackTimeout)
    LTR -->> Core 2: 
    activate Core 1   
    Note left of Core 1: waits until an ACK arrives  
    Core 2 ->> LTR: read(LTR::one, {datacore2, size})
    Note right of LTR: reading also generates an ack 
    LTR -->> Core 1: ack
    deactivate Core 1
    Note left of Core 1: it can continues execution
     
    

Loading

Sequence Diagram. Use of LTR in subscription mode. Core 1 posts a letter and waits until Core 2 confirms its reading.

Dependencies

Each embot::hw::icc::LTR uses one embot::hw::icc::MEM and an embot::hw::icc::SIG.

erDiagram
  
    "embot::hw::icc::LTR" ||..|| "embot::hw::icc::MEM" : uses
    "embot::hw::icc::LTR" ||..|| "embot::hw::icc::SIG" : uses
   

Loading

Figure. The embot::hw::icc::LTR uses one embot::hw::icc::MEM and ne embot::hw::icc::SIG .

The used API

In here are the API of use.

namespace embot::hw::icc {
    
    enum class LTR : uint8_t {  one = 0, two = 1, three = 2, four = 3, five = 4, six = 5, 
                                seven = 6, eight = 7, 
                                none = 31, maxnumberof = 8 };
} 

Code listing. We can use at most 8 embot::hw::icc::LTR items.

namespace embot::hw::icc::ltr {
            
    struct Config
    {        
        DIR direction {DIR::none};        
        constexpr Config() = default;
        constexpr Config(DIR d) : direction(d) {}            
        constexpr bool isvalid() const 
        {
            return (DIR::rx == direction) || (DIR::tx == direction);
        }
    };
    
    bool supported(embot::hw::icc::LTR l); 
    size_t size(embot::hw::icc::LTR l);
    result_t init(embot::hw::icc::LTR l, const Config &cfg);
    bool initialised(embot::hw::icc::LTR l);
    
    // tx methods
    size_t post(embot::hw::icc::LTR l, const embot::core::Data &data, embot::core::relTime acktimeout);
    size_t post(embot::hw::icc::LTR l, const embot::core::Data &data, const embot::core::Callback &onack); 
    bool acked(embot::hw::icc::LTR l);
    
    // rx methods
    bool subscribe(embot::hw::icc::LTR l, const embot::hw::Subscription &onreceived);
    bool available(embot::hw::icc::LTR l);   
    size_t read(embot::hw::icc::LTR l, embot::core::Data &data);             
} 

Code listing. The public interface of embot::hw::icc::LTR.

The embot::hw::icc::printer objects

The embot::hw::MTX is a mutual exclusion HW utility that works across cores.

The embot::hw::icc::printer::theClient and the embot::hw::icc::printer ::theServer can be instantiated one on each core of an amc board, the former on the CM4 and the second on the CM7 core.

At this point, the CM4 can send a string to the CM7 which prints it. How is done? Let's see.

namespace embot::hw::icc::printer {

    DIR _dir {DIR::none};
    
    void init(DIR dir)
    {
        _dir = dir;
        
        if(_dir == DIR::tx)
        {
            theClient::getInstance().initialise({embot::hw::icc::LTR::one});
        }
        else if(_dir == DIR::rx)
        {
            theServer::getInstance().initialise({embot::hw::icc::LTR::one});
        }            
    }
    
    void tick(const std::string &str)
    {        
        if(_dir == DIR::tx)
        {
            theClient::getInstance().getInstance().post(str);
        }      
    }
          
} 

Code listing. The test of namespace embot::hw::icc::printer.

void at_startup()
{
	embot::hw::icc::printer::init(DIR::rx);
}

Code listing. The calls on CM7.

void at_startup()
{
	embot::hw::icc::printer::init(DIR::tx);
}

void at_runtime()
{
    embot::core::TimeFormatter tf {embot::core::now()};
    embot::hw::icc::printer::init("i am the cm4 @ time = " + tf.to_string());    
}

Code listing. The calls on CM4.

The magic is done because theServer uses an embot::hw::icc::LTR in this way:

struct embot::hw::icc::printer::theServer::Impl
{
    embot::hw::icc::LTR _l {embot::hw::icc::LTR::one}; 
    bool _initted {false};    
    
    char _str[256] = {0};
    embot::core::Data _dd { &_str, sizeof(_str) };
    
    static void oninterrupt(void *p)
    {
        Impl *tHIS = reinterpret_cast<Impl*>(p);
        if(nullptr == tHIS)
        {
            return;
        }
                   
        embot::hw::icc::ltr::read(tHIS->_l, tHIS->_dd);
        embot::core::TimeFormatter tf(embot::core::now());
        embot::core::print("PS@" + tf.to_string() + ": " + tHIS->_str);   
    }
    
    
    Impl() = default;
            
    bool initialise(const Config &config)
    {
        if(true == _initted)
        {
            return false;;
        } 
        _l = config.l;        
        _initted = (resOK == embot::hw::icc::ltr::init(_l, {DIR::rx})) ? true : false;
        if(true == _initted)
        {
            embot::hw::icc::ltr::subscribe(_l, {{oninterrupt, this}, embot::hw::Subscription::MODE::permanent});
        }
        
        return true;
    }
    
};

Code listing. The internals of theServer. The LTR registers an interrupt which gets the string and prints it.

And theClient posts the string in this way.

struct embot::hw::icc::printer::theClient::Impl
{
    embot::hw::icc::LTR _l {embot::hw::icc::LTR::one}; 
    bool _initted {false};    
    
    char _str[256] = {0};
    embot::core::Data _dd { &_str, sizeof(_str) };
   
    
    Impl() = default;
            
    bool initialise(const Config &config)
    {
        if(true == _initted)
        {
            return false;;
        } 
        _l = config.l;        
        _initted = (resOK == embot::hw::icc::ltr::init(_l, {DIR::tx})) ? true : false;
        
        return true;
    }
        
    bool post(const std::string &str, embot::core::relTime tout)
    {
        if(false == _initted)
        {
            return false;
        }
        std::snprintf(_str, sizeof(_str), "%s", str.c_str());
        embot::hw::icc::ltr::post(_l, _dd, tout);
        return embot::hw::icc::ltr::acked(_l);           
    }
    
};

Code listing. The internals of theClient. The LTR sends a string and waits for its reading.