Unit Test Coding Standards

cFS Core Flight Executive Unit Test Coding Standards

Version 1.0 - Apr. 22, 2020

The following are standards for writing functional and coverage unit tests for the Core Flight Software Core Flight Executive system. New unit test code development should follow these standards, and as we update old unit tests, they should be converted to this standard.

These requirements and the associated macros and code are intended to make unit test code compact, easy to write, and easy to maintain. These macros are semantically-informative, and produce output in logs that also indicate the semantics of what was the intent of the call that succeeded or failed. Most of these do not report results of their execution unless there is a failure and/or the user configures for verbose output at compile time.

Standards

All API's shall have functional and coverage tests.
A single API has a master function (called a "test block" function in this document) that calls a number of per-test subfunctions.
A test block function shall call the "STARTBLOCK()" macro at the beginning and "ENDBLOCK()" macro at the end of execution.
Each per-test function...
- ...shall begin with a call of the "START()" macro.
- ...shall wrap, with the SETUP() macro, any CFE API calls that are required for configuring the environment for the test and which return a status.
- ...shall wrap, with an ASSERT...() macro, any call that is the focus of the test function.
- ...shall wrap, with a TEARDOWN() macro, any cleanup function calls.
- ...shall be considered as a failed test if any step (setup, test, teardown) fails.
- ...shall end with a call of the "REPORT()" macro.

Semantic Unit Test Macros

void STARTBLOCK() and void ENDBLOCK() - Reports the start and end of a "test block" function, a function that calls a series of unit test functions.

void START() - Reports the start of a single-test function.

bool SETUP(FN) - Call the CFE function that sets up the environment for the main test. This check the result, reporting an error and considering the test failed if the return from FN is not CFE_SUCCESS.

bool ASSERT(FN) - Call the CFE function being tested and consider the test successful if the return of the FN is CFE_SUCCESS. Note this reports the return code if it is not CFE_SUCCESS.

bool ASSERT_EQ(FN,EXP) - Call the CFE function being tested and consider the test successful if the return of the FN is equal to the EXP expression (usually a CFE return status definition). Note this reports the value of FN and EXP.

bool ASSERT_TRUE(EXP) - Asserts that the expression is true (note that this does not report any values.)

bool EVTCNT(N) - Check the number of events generated and report a test failure if the event count is not N.

bool EVTSENT(EVT) - Check that the event identified was sent by the code being tested.

bool TEARDOWN(FN) - Call the CFE function that performs teardown of the environment post-test. Call the CFE function and check the result, reporting an error and considering the test failed if the return from FN is not CFE_SUCCESS.

void REPORT() - Call at the end of a per-test function, this generates a report of the result of the test (PASS/FAIL).

Internals

The ut_support.h and ut_support.c files contain a suite of macros to be used by unit test code as described above. This code uses a global variable TestStat which is set to the #define CFE_PASS when the START() macro is called, and each step checks whether the value remains CFE_PASS, and if so, it executes the step and checks the return value, changing TestStat to CFE_FAIL if the step has failed. REPORT() calls UT_Report() to output the results of the test.

An Example Test Suite

void Test_SendMsg_API(void)
{
    STARTBLOCK();

    Test_SendMsg_NullPtr();
    Test_SendMsg_InvalidMsgId();
    Test_SendMsg_NoSubscribers();
    Test_SendMsg_MaxMsgSizePlusOne();
    Test_SendMsg_BasicSend();
    Test_SendMsg_SequenceCount();
    Test_SendMsg_QueuePutError();
    Test_SendMsg_PipeFull();
    Test_SendMsg_MsgLimitExceeded();
    Test_SendMsg_GetPoolBufErr();
    Test_SendMsg_ZeroCopyGetPtr();
    Test_SendMsg_ZeroCopySend();
    Test_SendMsg_ZeroCopyPass();
    Test_SendMsg_ZeroCopyReleasePtr();
    Test_SendMsg_DisabledDestination();
    Test_SendMsg_SendWithMetadata();
    Test_SendMsg_InvalidMsgId_ZeroCopy();
    Test_SendMsg_MaxMsgSizePlusOne_ZeroCopy();
    Test_SendMsg_NoSubscribers_ZeroCopy();

    ENDBLOCK();
} /* end Test_SendMsg_API */

/* ... */

/*
** Test successfully sending a message on the software bus
*/
void Test_SendMsg_BasicSend(void)
{
    CFE_SB_PipeId_t  PipeId;
    CFE_SB_MsgId_t   MsgId = SB_UT_TLM_MID;
    SB_UT_Test_Tlm_t TlmPkt;
    CFE_SB_MsgPtr_t  TlmPktPtr = (CFE_SB_MsgPtr_t) &TlmPkt;

    START();

    SB_ResetUnitTest();

    SETUP(CFE_SB_CreatePipe(&PipeId, 2, "TestPipe"));
    SETUP(CFE_SB_Subscribe(MsgId, PipeId));

    CFE_SB_InitMsg(&TlmPkt, MsgId, sizeof(TlmPkt), true); /* no return val from InitMsg */

    ASSERT(CFE_SB_SendMsg(TlmPktPtr));

    EVTCNT(3);

    TEARDOWN(CFE_SB_DeletePipe(PipeId));

    REPORT();
} /* end Test_SendMsg_BasicSend */

/* ... */

/*
** Test send message response when the socket queue is full
*/
void Test_SendMsg_PipeFull(void)
{
    CFE_SB_PipeId_t  PipeId;
    CFE_SB_MsgId_t   MsgId = SB_UT_TLM_MID;
    SB_UT_Test_Tlm_t TlmPkt;
    CFE_SB_MsgPtr_t  TlmPktPtr = (CFE_SB_MsgPtr_t) &TlmPkt;
    int32            PipeDepth = 1;

    START();

    SB_ResetUnitTest();
    CFE_SB_InitMsg(&TlmPkt, MsgId, sizeof(TlmPkt), true);
    SETUP(CFE_SB_CreatePipe(&PipeId, PipeDepth, "PipeFullTestPipe"));
    SETUP(CFE_SB_Subscribe(MsgId, PipeId));

    SETUP(CFE_SB_SendMsg(TlmPktPtr)); /* should succeed */

    /* Tell the QueuePut stub to return OS_QUEUE_FULL on its next call */
    UT_SetDeferredRetcode(UT_KEY(OS_QueuePut), 1, OS_QUEUE_FULL);

    ASSERT(CFE_SB_SendMsg(TlmPktPtr)); /* Pipe overflow causes SendMsg to return CFE_SUCCESS */

    EVTCNT(5);
    EVTSENT(CFE_SB_Q_FULL_ERR_EID);

    TEARDOWN(CFE_SB_DeletePipe(PipeId));

    REPORT();
} /* end Test_SendMsg_PipeFull */

Provide feedback

Saved searches