-
Notifications
You must be signed in to change notification settings - Fork 23
/
cfs_sbn.md
280 lines (228 loc) · 14.1 KB
/
cfs_sbn.md
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
Core Flight Software (cFS) Software Bus Networking (SBN)
========================================================
Documentation Version History
------------------------------
This document details the design and use of the SBN application. This is current
as of SBN version 1.11.
This document is maintained by [Chris Knight, NASA Ames Research Center]
SBN Version History
-------------------
- SBN 1.0 – UDP-only monolithic application.
- SBN 1.1 – Added a modular network layer for Spacewire, Serial.
- SBN 1.2 \@ce1b3ca – TCP module. Merged protocol and data traffic into
the same connections/sockets. Heartbeats only sent if no other traffic
sent in the last number of seconds. Bug-fix to ensure SBN ignores messages it
publishes on the SB, ensures all network messages are big-endian and aligned,
removes windowing/retransmit logic.
- SBN 1.3 \@15f3754 – Removed sync word. Simplified module API, added MID
remapping/filtering. Added the (compile-time) option of per-peer tasks for
watching pipes and net.
- SBN 1.4 \@2b6556a – DTN module. Pushed protocol handling (announce/heartbeat)
down into the modules that need it (UDP.)
- SBN 1.5 \@b5cb3d7 – When sending all subs, send them in one message.
- SBN 1.6 \@b0d0027 – Added “unload” method to modules.
- SBN 1.7 \@eff7047 – Polling occurs each cycle (modules are responsible for managing timeouts), re-added serial backend, SbnPeerData.dat uses a numeric network ID, not name.
- SBN 1.8 \@ae9a1f5 - Removed separate housekeeping status structs (moved housekeeping-related values to the main structs (`SBN_App_t`, `SBN_NetInterface_s` (`SBN_NetInterface_t`), `SBN_PeerInterface_t`). standardized on housekeeping being hand-packed big-endian.
- SBN 1.9 \@063ebf2 - Adds protocol packet to identify protocol version. For now SBN reports if the version matches or not. (Is backward-compatible.) Also modules communicate to the main SBN app when a connection is established and lost and message pipes are only created/maintained for connected peers.
- SBN 1.10 - Table-driven configuration, rather than flat-file. Removed net and peer names, instead should always use ProcessorID and NetIdx.
- SBN 1.11 - Modules loaded by ES as libraries, OSAL used for TCP + UDP.
Overview
--------
SBN is a cFS application that connects the local software bus to one or more
other cFS nodes (who are also running SBN) such that all messages sent by an
application on one bus will be received by an application on another bus. SBN
has a modular *network* architecture (TCP, UDP, Serial, SpaceWire, etc.) to
connect *peers* and supports multiple peer networks with a local *host*
connection affiliated with each. SBN also remaps and filters messages (cFS
table-configured.)
SBN Build and Configuration
---------------------------
SBN is built like any other cFS application, either via specifying it in
the the `TGT#_APPLISTS` parameter in the targets.cmake. Protocol modules
(`sbn_udp`, `sbn_tcp`, `sbn_dtn`, ...) must also be specified as an
application in the build process and loaded by ES after the core SBN
application.
SBN uses two tables, the "conf" table for configuring which modules to connect
and which networks and peers to communicate with and the "remap" table for
identifying which Message ID's should be remapped or filtered before sending
to specific peers.
### SBN Platform Configuration
The `sbn_platform_cfg.h` file contains a number of definitions that control
how SBN allocates and limits resources as well as controlling the behavior
of SBN. Most "max" definitions relate to in-memory static arrays, so increasing
the value increases the memory footprint of SBN (in some cases, non-linearly.)
### SBN Configuration Table
The SBN configuration table is a standard cFS table defining modules and
networks of peers.
See `sbn_tbl.h` and `sbn_conf_tbl.c`.
### SBN Remapping Table
The SBN remapping table is a standard cFS table defining, on a peer-by-peer
basis, which message ID's should be remapped to other ID's, or which
message ID's should be filtered (where the "To" field is 0).
See `sbn_tbl.h` and `sbn_remap_tbl.c`.
SBN Remapping and Filtering
---------------------------
In complex SBN peer networks, it may be useful to have each node have its own
configuration of message ID's that local apps publish and subscribe to, and
to limit traffic between the nodes or map from one ID space to another. SBN
supports this via the remapping and filtering capability, which can be
configured both by the remap table at initialization time and via commands
at runtime.
The SBN remapping subsystem, by default, does not affect traffic and all
messages published on one node is sent to any peers that have subscribed
to that ID. Alternatively, SBN remapping can be configured so that ONLY
those entries that are remapped are sent to peers; and, yes, you can remap
from an MID to the same MID.
SBN remapping may be enabled at start time via a compile-time macro, and
can be enabled and disabled at runtime via the remapping configuration
command.
SBN Control Commands
--------------------
SBN has a number of commands for managing the SBN application's configuration
and for requesting housekeeping telemetry.
Macro |CC |Command Description |Parameters
--------------------|------|-----------------------------------------|------
`SBN_NOOP_CC` |`0x00`|Do nothing beyond reporting an event. |<none>
`SBN_RESET_CC` |`0x01`|Resets the SBN network. |<none>
`SBN_RESET_PEER_CC` |`0x02`|Resets a connection to a particular peer. |`uint8 NetIdx, uint8 PeerIdx`
`SBN_REMAPCFG_CC` |`0x05`|Sets the remapping configuration. |`uint8 Enabled, uint8 RemapDefaultFlag`
`SBN_REMAPADD_CC` |`0x06`|Adds a remapping. |`uint32 ProcessorID, CFE_SB_MsgId_t FromMID, CFE_SB_MsgId_t ToMID`
`SBN_REMAPDEL_CC` |`0x07`|Removes a remapping. |`uint32 ProcessorID, CFE_SB_MsgId_t FromMID`
`SBN_HK_CC` |`0x0A`|Requests main housekeeping telemetry. |<none>
`SBN_HK_NET_CC` |`0x0B`|Requests housekeeping telemetry for a net. |`uint8 NetIdx`
`SBN_HK_PEER_CC` |`0x0C`|Requests housekeeping telemetry for a peer.|`uint8 NetIdx, uint8 PeerIdx`
`SBN_HK_PEERSUBS_CC`|`0x0D`|Requests hk telemetry for a peer's subs. |`uint8 NetIdx, uint8 PeerIdx`
`SBN_HK_MYSUBS_CC` |`0x0E`|Requests hk telemetry for my subs. |<none>
SBN Housekeeping Telemetry
--------------------------
Housekeeping command codes are used to request housekeeping telemetry messages. As all housekeeping is requested with command codes to the main (and only)
command MID, the command code used to request the housekeeping is returned
in the housekeeping payload so that they can be differentiated. All numeric
values are transmitted in big-endian order and no padding is used.
The following commands generate payloads in the following format:
*SBN_HK_CC*
Field |Type |Description
-----------|--------|-----------
`CC` |`uint8` |Command code of HK request.
`CmdCnt` |`uint16`|Number of commands received.
`CmdErrCnt`|`uint16`|Number of commands that generated errors.
`SubCnt` |`uint16`|Number of local subscriptions.
`NetCnt` |`uint16`|Number of networks configured.
*SBN_HK_NET_CC*
Field |Type |Description
------------|----------------------------|-----------
`CC` |`uint8` |Command code of HK request.
`ProtocolID`|`uint8` |The ID of the protocol of this network.
`PeerCnt` |`uint16` |The number of peers associated with this network.
*SBN_HK_PEER_CC*
Field |Type |Description
-------------|-----------------------------|-----------
`CC` |`uint8` |Command code of HK request.
`QoS` |`CFE_SB_Qos_t` |QoS flags for this peer.
`SubCnt` |`uint16` |Number of errors generated in trying to receive from this peer.
`ProcessorID`|`uint32` |The ProcessorID of the peer.
`LastSend` |`OS_time_t` |The last time I sent a message to this peer.
`LastRecv` |`OS_time_t` |The last time I received a message from this peer.
`SendCnt` |`uint16` |Number of messages sent to this peer.
`RecvCnt` |`uint16` |Number of messages received from this peer.
`SendErrCnt` |`uint16` |Number of errors generated in trying to send to this peer.
`RecvErrCnt` |`uint16` |Number of errors generated in trying to receive from this peer.
*SBN_HK_PEERSUBS_CC*
Field |Type |Description
-----------|------------------------|-----------
`CC` |`uint8` |Command code of HK request.
`PeerIdx` |`uint16` |Index of the peer in the request.
`SubCnt` |`uint16` |Number of local subscriptions.
`Subs` |`CFE_SB_MsgId_t[SubCnt]`|Subscriptions.
*SBN_HK_MYSUBS_CC*
Field |Type |Description
-----------|------------------------|-----------
`CC` |`uint8` |Command code of HK request.
`SubCnt` |`uint16` |Number of local subscriptions.
`Subs` |`CFE_SB_MsgId_t[SubCnt]`|Subscriptions.
SBN Interactions With the Software Bus (SB)
-------------------------------------------
SBN treats all nodes as peers and (by default) all subscriptions of local
applications should receive messages sent by publishers on other peers, and
all messages published on the local bus should be transmitted to any peers
who have applications subscribed to that message ID.
The Software Bus (SB), when an application subscribes to a message ID or
unsubscribes from a message ID, sends a message that SBN receives. Upon
receipt of these messages, SBN updates its internal state tables and sends
a message to the peers with the information on the update.
SBN Application-Level Network Protocol
--------------------------------------
In communicating with peers over a network protocol provided by a protocol
module, SBN uses an SBN message format which is comprised of the following
fields in big-endian, packed (no alignment) format:
Field |Type |Description
---------|--------|-----------
`MsgSz` |`uint16`|The total size of the message, excluding this header.
`MsgType`|`uint8` |The type of the message (see below).
`CpuID` |`uint32`|The ProcessorID of the sender.
Message types are an enumeration from below, although protocol modules may
send additional message types. Type values of 128 or higher (high bit set)
are reserved for module use.
MsgType |Value |Description
---------------|------|-----------
`SBN_NO_MSG` |`0x00`|No payload. (Unused.)
`SBN_SUB_MSG` |`0x01`|Payload is local subs for peer to add.
`SBN_UNSUB_MSG`|`0x02`|Payload is local unsubscriptions for peer to remove.
`SBN_APP_MSG` |`0x03`|Payload is a message from the local software bus.
`SBN_PROTO_MSG`|`0x04`|Payload is a protocol informational packet.
Currently protocol messages contain a single byte value representing the
current protocol version defined by `SBN_PROTO_VER`.
SBN Scheduling and Tasks
------------------------
SBN has two modes of operation (configured at compile time):
- A traditional scheduler (SCH)-driven mode where SCH sends a wakeup message
periodically and SBN polls pipes and network modules. SBN has a built-in
timeout period so that if SCH is somehow not functioning properly, or is
mis-configured, SBN will continue to function. This mode is preferable
in environments where resources are constrained, where network traffic
load is well understood, and deterministic behavior is expected.
- A per-peer task model where the local SBN instance creates two tasks for
every peer--one task blocks on reading the local pipe (waiting for messages
to send to the respective peer) and the other task blocks on the network
module's "receive" function, waiting for messages from the peer to send
to the local bus. This model is preferred in environments where network
traffic load may vary significantly (posing a risk of overloading pipes),
and where determinism is not expected and resources are not particularly
constrained.
Technically, the choice of task or SCH-driven processing is set for
each direction ("sending" local bus messages to the peer and "receiving"
messages from the peer to put on the local bus.) However, it's generally
best to stick with either SCH-driven processing or task-driven processing.
SBN Protocol Modules
--------------------
SBN requires the use of protocol libraries that provide a
defined set of functions to send and receive encapsulated software bus messages
and subscription updates. Protocol modules may use connection-less (UDP)
or connection-based (TCP) network technologies and network reliability and
connection maintenance is expected to be provided by the module or the
network technology it is using. SBN does benefit from knowing when a peer
is "connected" so that the local subscriptions can be sent (in bulk) to
that peer; otherwise SBN does not need to know the state of the network
the module is communicating with.
Currently SBN provides the following modules:
- UDP - Utilizing the UDP/IP connectionless protocol, the UDP module uses
"announce" and "heartbeat" internal messages to determine when a peer has
connected to the network (and that the subscriptions need to be sent.)
Otherwise no network reliability is provided by the UDP module, packets
may be lost or jumbled without the knowledge of SBN.
- TCP - The TCP module utilizes the Internet-standard, high reliability TCP
protocol, which provides for error correction and connection management.
- DTN - Integrating the ION-DTN 3.6.0 libraries, the DTN module provides
high reliability, multi-path transmission, and queueing. Effectively,
DTN peers are always connected.
- Serial - Supports SBN over standard serial devices.
SBN Datastructures
------------------
SBN utilizes a complex set of data structures in memory to track
the state of the local system and its state knowledge of the peers on
the network.
