Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Further simplify SB unit tests #734

Closed
jphickey opened this issue Jun 4, 2020 · 2 comments · Fixed by #737 or #743
Closed

Further simplify SB unit tests #734

jphickey opened this issue Jun 4, 2020 · 2 comments · Fixed by #737 or #743
Assignees
Milestone

Comments

@jphickey
Copy link
Contributor

jphickey commented Jun 4, 2020

Is your feature request related to a problem? Please describe.
Recently the software bus unit tests were updated to employ macros for common code bits, but this only amplifies the fact that the SB unit tests don't follow the typical UT assert model.

In particular, they "collect" a bunch of conditions together and then assert at the end that all the conditions where true. There is no reason to do this deferred reporting with UT assert, and it only serves to obfuscate the true failure because the UT assert failure message line number can be quite different than what actually failed.

Describe the solution you'd like
Now that the macros/wrappers are in place, rather than having them only mimic the old test logic, have them implement the preferred UT assert patterns.

  • Remove TestStat global variable. Do not keep global state outside UT assert.
  • Remove separate text messages inside "if" conditionals and replace with UT assert statements. That's what the UT assert API is for, after all.
  • Remove VERBOSE compile-time option. The UT assert has a runtime verbosity flag, just call UtDebug and the message will only be printed when set to verbose mode.

Additional context
This really just the next step along the path to using UT assert as it was intended, and removing the "compatibility crutches" that were put into place because CFE (and SB in particular) didn't employ the same test patterns as other apps/modules.

Requester Info
Joseph Hickey, Vantage Systems, Inc.

@jphickey jphickey self-assigned this Jun 4, 2020
@jphickey
Copy link
Contributor Author

jphickey commented Jun 4, 2020

There is also no need for separate "start"/"end" reporting, etc. The UT assert already has a way to identify the start of each test routine/block as well as designated setup and teardown phases. The SB test functions should follow this pattern rather than making their own.

@jphickey
Copy link
Contributor Author

jphickey commented Jun 5, 2020

Entertaining tidbit of information. We have a one-line abstraction function here:

CCSDS_WR_LEN(MsgPtr->Hdr,TotalLength);

The unit test for that function is here, weighing in at 124 lines:

void Test_CFE_SB_SetGetTotalMsgLength(void)
{
SB_UT_Test_Cmd_t SBCmd;
CFE_SB_MsgPtr_t SBCmdPtr = (CFE_SB_MsgPtr_t) &SBCmd;
SB_UT_Test_Tlm_t SBTlm;
CFE_SB_MsgPtr_t SBTlmPtr = (CFE_SB_MsgPtr_t) &SBTlm;
SB_UT_TstPktWoSecHdr_t SBNoSecHdrPkt;
CFE_SB_MsgPtr_t SBNoSecHdrPktPtr = (CFE_SB_MsgPtr_t) &SBNoSecHdrPkt;
int32 SetSize;
uint16 TotalMsgSizeReturned;
int16 ActualPktLenField;
int16 ExpPktLenField;
START();
/* CCSDS pkt length field = TotalPktSize - 7 */
/* Loop through all pkt length values for cmd pkts w/sec hdr */
SB_ResetUnitTest();
TestStat = CFE_PASS;
CFE_SB_SetMsgId(SBCmdPtr, SB_UT_CMD_MID);
for (SetSize = 0; SetSize < 0x10000; SetSize++)
{
CFE_SB_SetTotalMsgLength(SBCmdPtr, SetSize);
TotalMsgSizeReturned = CFE_SB_GetTotalMsgLength(SBCmdPtr);
ActualPktLenField = UT_GetActualPktLenField(SBCmdPtr);
ExpPktLenField = SetSize - 7; /* TotalPktSize - 7 */
if (TotalMsgSizeReturned != SetSize ||
ActualPktLenField != ExpPktLenField)
{
snprintf(cMsg, UT_MAX_MESSAGE_LENGTH,
"SzRet=%u, SetSz=%ld, ActPL=%d, ExpPL=%d",
(unsigned int) TotalMsgSizeReturned, (long) SetSize,
(int) ActualPktLenField, (int) ExpPktLenField);
UT_Text(cMsg);
TestStat = CFE_FAIL;
break;
}
}
REPORT();
/* Loop through all pkt length values for cmd pkts wo/sec hdr */
SB_ResetUnitTest();
TestStat = CFE_PASS;
CFE_SB_SetMsgId(SBNoSecHdrPktPtr, SB_UT_TLM_MID);
for (SetSize = 0; SetSize < 0x10000; SetSize++)
{
CFE_SB_SetTotalMsgLength(SBNoSecHdrPktPtr, SetSize);
TotalMsgSizeReturned = CFE_SB_GetTotalMsgLength(SBNoSecHdrPktPtr);
ActualPktLenField = UT_GetActualPktLenField(SBNoSecHdrPktPtr);
ExpPktLenField = SetSize - 7; /* TotalPktSize - 7 */
if (TotalMsgSizeReturned != SetSize ||
ActualPktLenField != ExpPktLenField)
{
snprintf(cMsg, UT_MAX_MESSAGE_LENGTH,
"SzRet=%u, SetSz=%ld, ActPL=%d, ExpPL=%d",
(unsigned int) TotalMsgSizeReturned, (long) SetSize,
(int) ActualPktLenField, (int) ExpPktLenField);
UT_Text(cMsg);
TestStat = CFE_FAIL;
break;
}
}
REPORT();
/* Loop through all pkt length values for tlm pkts w/sec hdr */
SB_ResetUnitTest();
TestStat = CFE_PASS;
CFE_SB_SetMsgId(SBTlmPtr, SB_UT_TLM_MID);
for (SetSize = 0; SetSize < 0x10000; SetSize++)
{
CFE_SB_SetTotalMsgLength(SBTlmPtr, SetSize);
TotalMsgSizeReturned = CFE_SB_GetTotalMsgLength(SBTlmPtr);
ActualPktLenField = UT_GetActualPktLenField(SBTlmPtr);
ExpPktLenField = SetSize - 7; /* TotalPktSize - 7 */
if (TotalMsgSizeReturned != SetSize ||
ActualPktLenField != ExpPktLenField)
{
snprintf(cMsg, UT_MAX_MESSAGE_LENGTH,
"SzRet=%u, SetSz=%ld, ActPL=%d, ExpPL=%d",
(unsigned int) TotalMsgSizeReturned, (long) SetSize,
(int) ActualPktLenField, (int) ExpPktLenField);
UT_Text(cMsg);
TestStat = CFE_FAIL;
break;
}
}
REPORT();
/* Loop through all pkt length values for tlm pkts wo/sec hdr */
SB_ResetUnitTest();
TestStat = CFE_PASS;
CFE_SB_SetMsgId(SBNoSecHdrPktPtr, SB_UT_TLM_MID);
for (SetSize = 0; SetSize < 0x10000; SetSize++)
{
CFE_SB_SetTotalMsgLength(SBNoSecHdrPktPtr, SetSize);
TotalMsgSizeReturned = CFE_SB_GetTotalMsgLength(SBNoSecHdrPktPtr);
ActualPktLenField = UT_GetActualPktLenField(SBNoSecHdrPktPtr);
ExpPktLenField = SetSize - 7; /* TotalPktSize - 7 */
if (TotalMsgSizeReturned != SetSize ||
ActualPktLenField != ExpPktLenField)
{
snprintf(cMsg, UT_MAX_MESSAGE_LENGTH,
"SzRet=%u, SetSz=%ld, ActPL=%d, ExpPL=%d",
(unsigned int) TotalMsgSizeReturned, (long) SetSize,
(int) ActualPktLenField, (int) ExpPktLenField);
UT_Text(cMsg);
TestStat = CFE_FAIL;
break;
}
}
REPORT();
} /* end Test_CFE_SB_SetGetTotalMsgLength */

But what makes it really interesting is that for some reason this is feeling the need to call this one line function with every possible uint16 value for every possible header type including header types that SB does not support even though its a one line function that does not even look at header type.

In all, there are 524,288 "assert" actions to cover this one line function. because it also checks the packet content itself in addition to the return code for every value.

Edit: In all fairness, it actually covers the "get" and "set" so there are two lines of FSW code being tested by these 500,000 asserts.

jphickey added a commit to jphickey/cFE that referenced this issue Jun 5, 2020
Makes the SB unit test closer to recommended UT assert patterns

Do not keep a separate "TestStat" state variable outside UT assert.
Do not report separate status messages from the asserts.  Use UT assert.
Do not "reset" in the middle of a test routine, split into separate
test routines where this is done.
No need for "START" and "STARTBLOCK" or "ENDBLOCK".  UT assert has
messages for these test actions.  Each block should be a separate test
routine and then these are unnecessary.
astrogeco added a commit that referenced this issue Jun 17, 2020
Fix #734, continue cleanup of SB unit test
@astrogeco astrogeco added this to the 6.8.0 milestone Sep 28, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
2 participants