Option to drop oldest message when msg limit on pipe reached (keep newest)

[skliper]

Is your feature request related to a problem? Please describe.
Looks like SB rejects newest message when message limit reached

cFE/fsw/cfe-core/src/sb/cfe_sb_api.c

Lines 1577 to 1588 in f1be048

	/* if Msg limit exceeded, log event, increment counter */
	/* and go to next destination */
	if(DestPtr->BuffCount >= DestPtr->MsgId2PipeLim){
	SBSndErr.EvtBuf[SBSndErr.EvtsToSnd].PipeId = DestPtr->PipeId;
	SBSndErr.EvtBuf[SBSndErr.EvtsToSnd].EventId = CFE_SB_MSGID_LIM_ERR_EID;
	SBSndErr.EvtsToSnd++;
	CFE_SB.HKTlmMsg.Payload.MsgLimitErrorCounter++;
	PipeDscPtr->SendErrors++;
	continue;
	}/* end if */

In some cases newest message is of higher priority

Describe the solution you'd like
Add option (maybe part of QOS?) to remove oldest message from queue when limit reached. Easy if the message limit is 1, likely need trade study if more than one (reorder queue? Replace oldest w/ newest would put them out of order, add new to queue remove oldest and remove gap? etc). Performance cost to find/replace/reorder/etc.

Describe alternatives you've considered
Just subscribe with enough space to hold all, and chew through entire pipe to get the latest then process.

Additional context
Requested by 587

Requester Info
Jacob Hageman - NASA/GSFC

[skliper]

Preference is to keep unique pipe processing on app side where possible to avoid general performance hits or added SB complexity, but entered here as it's worth discussion.

[jphickey]

Keeping the oldest message and discarding the new one is normal FIFO queue behavior and it is dictated by the OSAL queue implementation that underpins SB pipes.

The contract of a FIFO queue says that if it is accepted at the sender side (i.e. successfully placed into the queue) then it is guaranteed to be delivered at the receiver side.

OSAL does not implement priority queues, partly because they aren't widely/consistently implemented across RTOS's.

Maybe another reason to push ahead and implement nasa/osal#73, as this could offer all of the behavior options (Basic FIFO, Priority, Ring buffer).

[iamthad]

For some applications, it may be sufficient to have some sort of interface that exposes only the latest value. I imagine this may be easier to support than a general LIFO capability.

[jphickey]

For some applications, it may be sufficient to have some sort of interface that exposes only the latest value. I imagine this may be easier to support than a general LIFO capability.

Totally agree - but then this would no longer be a queue at all, its more like a table.

In projects I support we employ an intermediate library that implements a software bus listener that subscribes to and captures all telemetry, and presents an API where other applications can get the current/latest value of each telemetry point. It implements a scheduler-based page flipping so all values are coherent with each other and various (configurable) filtering/scrubbing/unit normalization features.

The good news is though this can all be done with a CFE library, one doesn't need to replace queues to get it. The regular software bus does not need to be aware of the extra layer.