Fix #596, scrub all UT_Report calls #1624

jphickey · 2021-06-16T14:49:56Z

Describe the contribution
Scrub through all UT reporting calls and replace with preferred macro where possible.

All calls to check status/return code are replaced with macro that logs all arguments and return value
All calls that involved multiple conditions AND'ed together are replaced with individual asserts on each condition.

Fixes #596
Fixes #470

Testing performed
Build and run unit tests, confirm logging is as intended, and coverage level is not reduced

Expected behavior changes
Better, more complete context information is logged into the file
All individual assert conditions are logged separately (no more checking of multiple AND'ed values/conditions in a single report)

System(s) tested on
Ubuntu

Additional context
This also removes all the "free-form" info that was in the logs, since it wasn't really providing value (a developer will go to the source file/line number and see full context, nor was it complete or consistent enough to provide any value for cross referencing or documentation).

Contributor Info - All information REQUIRED for consideration of pull request
Joseph Hickey, Vantage Systems, Inc.

jphickey · 2021-06-16T14:50:34Z

Pushing as draft for initial review/discussion, only "ES" module is done so far.

astrogeco · 2021-06-16T16:36:20Z

CCB:2021-06-16 APPROVED

There's a Ressource ID equal macro in cfe_test.h
Should we take out the notice of how the test failed?
What to do with the macros' prefixes? Keep them, having the prefix indicates that it is not part of the generic UTAssert set

Update ES coverage test to use preferred macros. Adds dedicated assert macros for checking fixed-length string buffers, and for checking memory offsets. Also adds an improved implemention of the syslog/printf check which filters out newlines (keeps log more parseable).

Update EVS coverage test to use preferred macros

Update SB coverage test to use preferred macros. Adds a dedicated assert macro for checking SB MsgId values.

Update MSG coverage test to use preferred macros

Update SBR coverage test to use preferred macros

Update TBL coverage test to use preferred macros

Update TIME coverage test to use preferred macros

Update FS coverage test to use preferred macros.

jphickey · 2021-06-22T13:49:03Z

Updated for review. This is based on the current "integration-candidate" (pending final baseline) but otherwise is ready to go.

Clean up the assert functions and macros which are no longer used after updating all coverage tests to use preferred macros.

Add required coverage test cases to achieve 100% line coverage in FS

jphickey · 2021-06-22T20:46:53Z

Added commit to fix #470 onto this PR to avoid conflict/baseline issue, as the FS test changes would not merge cleanly with the other scrub.

This gets 100% line and 100% function coverage, and about 96% branch coverage. There are 5 untested branches total, 4 of which are "Endian Check" therefore fixed in hardware (not possible to change except by running on other arch). The last flagged item was the while loop with two OR'ed conditions, they typically both become false at the same time, there is never a way in the current code that only the second condition will be false, but I prefer to keep both stop conditions just in case.

skliper · 2021-06-23T15:35:01Z

@astrogeco - AFAIK this should either be merged with IC or a discussion topic at the CCB?

astrogeco · 2021-06-23T15:58:04Z

Put it on the agenda, didn't merge because of draft status

astrogeco · 2021-06-23T16:17:35Z

CCB:2021-06-23

LESSON LEARNED: do not use a single assert for multiple things. This obfuscates things if there's actual problems. Use one condition per assert!

nasa/cFE#1624, scrub all UT_Report calls

*Combines* - nasa/elf2cfetbl#81 - nasa/tblCRCTool#52 - nasa/ci_lab#88 - nasa/sch_lab#83 - nasa/sample_app#150 - nasa/sample_lib#64 - nasa/to_lab#100 *Includes* - nasa/cFE#1630, correct path to users guide warning log - nasa/cFE#1621, add additional test cases for Child Tasks - nasa/cFE#1608, Add cfe functional tests to CI - nasa/cFE#1627, rename/clean CFE coverage assert macros - nasa/cFE#1623, Added UT tests for cFE ES Api - nasa/cFE#1634, Expand CDS Functional Tests. - nasa/cFE#1633, add test log file - nasa/cFE#1594, Event ID updates - nasa/cFE#1624, scrub all UT_Report calls - nasa/osal#1066, implement missing parameter/retcode test permutations - nasa/cFS-GroundSystem#182, Add test start command script for cmdUtil - nasa/tblCRCTool#51, add printf conversion casts ** Implement Coding Standard in CodeQL ** - nasa/cFS-GroundSystem#180 - nasa/elf2cfetbl#80 - nasa/tblCRCTool#49 - nasa/ci_lab#87 - nasa/sch_lab#79 - nasa/sample_app#149 - nasa/sample_lib#63 - nasa/to_lab#99

*Combines* - nasa/cFE#1632, v6.8.0-rc1+dev726 - nasa/osal#1079, v5.1.0-rc1+dev548 - nasa/ci_lab#88, v2.4.0-rc1+dev42 - nasa/sch_lab#83, v2.4.0-rc1+dev40 - nasa/sample_app#150, v1.2.0-rc1+dev66 - nasa/sample_lib#64, v1.2.0-rc1+dev38 - nasa/to_lab#100, v2.4.0-rc1+dev49 - nasa/elf2cfetbl#81, v3.2.0-rc1+dev30 - nasa/tblCRCTool#52, v1.2.0-rc1+dev33 - nasa/cFS-GroundSystem#183, v2.2.0-rc1+dev52 *Includes* - nasa/cFE#1630, correct path to users guide warning log - nasa/cFE#1621, add additional test cases for Child Tasks - nasa/cFE#1608, Add cfe functional tests to CI - nasa/cFE#1627, rename/clean CFE coverage assert macros - nasa/cFE#1623, Added UT tests for cFE ES Api - nasa/cFE#1634, Expand CDS Functional Tests. - nasa/cFE#1633, add test log file - nasa/cFE#1594, Event ID updates - nasa/cFE#1624, scrub all UT_Report calls - nasa/osal#1066, implement missing parameter/retcode test permutations - nasa/cFS-GroundSystem#182, Add test start command script for cmdUtil - nasa/tblCRCTool#51, add printf conversion casts ** Implement Coding Standard in CodeQL ** - nasa/cFS-GroundSystem#180 - nasa/elf2cfetbl#80 - nasa/tblCRCTool#49 - nasa/ci_lab#87 - nasa/sch_lab#79 - nasa/sample_app#149 - nasa/sample_lib#63 - nasa/to_lab#99 Co-authored-by: Jacob Hageman <[email protected]> Co-authored-by: Joseph Hickey <[email protected]> Co-authored-by: Ariel Adams <[email protected]> Co-authored-by: Alex Campbell <[email protected]> Co-authored-by: Jose F Martinez Pedraza <[email protected]>

jphickey added the CCB:Ready Ready for discussion at the Configuration Control Board (CCB) label Jun 16, 2021

jphickey force-pushed the fix-596-ut-report-macros branch 2 times, most recently from 204d2a9 to c55dba2 Compare June 16, 2021 15:24

astrogeco added CCB:Approved Indicates code review and approval by community CCB and removed CCB:Ready Ready for discussion at the Configuration Control Board (CCB) labels Jun 16, 2021

jphickey mentioned this pull request Jun 16, 2021

Fix #1626, rename/clean CFE coverage assert macros #1627

Merged

jphickey force-pushed the fix-596-ut-report-macros branch from c55dba2 to 90b9c39 Compare June 22, 2021 13:43

jphickey added 8 commits June 22, 2021 09:46

Partial nasa#596, UtAssert macros for EVS test

161d8da

Update EVS coverage test to use preferred macros

Partial nasa#596, UtAssert macros for SB test

a563e13

Update SB coverage test to use preferred macros. Adds a dedicated assert macro for checking SB MsgId values.

Partial nasa#596, UtAssert macros for MSG test

899dd6d

Update MSG coverage test to use preferred macros

Partial nasa#596, UtAssert macros for SBR test

72c1c8d

Update SBR coverage test to use preferred macros

Partial nasa#596, UtAssert macros for TBL test

6b21c14

Update TBL coverage test to use preferred macros

Partial nasa#596, UtAssert macros for TIME test

907f2ca

Update TIME coverage test to use preferred macros

Partial nasa#596, UtAssert macros for FS test

d12ab62

Update FS coverage test to use preferred macros.

jphickey force-pushed the fix-596-ut-report-macros branch from 90b9c39 to e313573 Compare June 22, 2021 13:46

jphickey requested review from skliper and pepepr08 June 22, 2021 13:49

skliper approved these changes Jun 22, 2021

View reviewed changes

Fix nasa#596, remove now-unused assert functions (final)

890cc87

Clean up the assert functions and macros which are no longer used after updating all coverage tests to use preferred macros.

jphickey force-pushed the fix-596-ut-report-macros branch from e313573 to 890cc87 Compare June 22, 2021 15:10

pepepr08 approved these changes Jun 22, 2021

View reviewed changes

jphickey mentioned this pull request Jun 22, 2021

Fix #470, complete coverage for FS subsystem #1622

Closed

Fix nasa#470, complete coverage for FS subsystem

442634d

Add required coverage test cases to achieve 100% line coverage in FS

jphickey force-pushed the fix-596-ut-report-macros branch from dd849d8 to 442634d Compare June 22, 2021 20:38

jphickey marked this pull request as ready for review June 23, 2021 16:30

astrogeco changed the base branch from main to integration-candidate June 23, 2021 16:31

astrogeco added the conflicts label Jun 23, 2021

jphickey merged commit 0701923 into nasa:integration-candidate Jun 23, 2021

astrogeco removed the conflicts label Jun 23, 2021

astrogeco added a commit to nasa/cFS that referenced this pull request Jun 23, 2021

IC:2021-06-22, Add cFE PR

5724303

nasa/cFE#1624, scrub all UT_Report calls

This was referenced Jun 23, 2021

cFE Integration candidate: 2021-06-22 #1632

Merged

cFS Bundle Integration candidate: 2021-06-22 nasa/cFS#270

Merged

jphickey deleted the fix-596-ut-report-macros branch August 3, 2021 15:19

skliper added this to the 7.0.0 milestone Sep 24, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix #596, scrub all UT_Report calls #1624

Fix #596, scrub all UT_Report calls #1624

jphickey commented Jun 16, 2021 •

edited

Loading

jphickey commented Jun 16, 2021

astrogeco commented Jun 16, 2021

jphickey commented Jun 22, 2021

jphickey commented Jun 22, 2021

skliper commented Jun 23, 2021

astrogeco commented Jun 23, 2021

astrogeco commented Jun 23, 2021

Fix #596, scrub all UT_Report calls #1624

Fix #596, scrub all UT_Report calls #1624

Conversation

jphickey commented Jun 16, 2021 • edited Loading

jphickey commented Jun 16, 2021

astrogeco commented Jun 16, 2021

jphickey commented Jun 22, 2021

jphickey commented Jun 22, 2021

skliper commented Jun 23, 2021

astrogeco commented Jun 23, 2021

astrogeco commented Jun 23, 2021

jphickey commented Jun 16, 2021 •

edited

Loading