Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Scrub for discrepancies between implementation return codes and documentation #1724

Closed
jphickey opened this issue Jul 28, 2021 · 3 comments
Closed

Scrub for discrepancies between implementation return codes and documentation #1724

jphickey opened this issue Jul 28, 2021 · 3 comments
Assignees
@jphickey
Labels
docs This change only affects documentation. duplicate enhancement unit-test

Comments

@jphickey
Copy link
Contributor

Is your feature request related to a problem? Please describe.
Need to perform a general scrub of API documentation for parameters, error checking, and and return codes, as was done for OSAL.

Describe the solution you'd like

  • Confirm that all return/status codes which are directly generated by an implementation appear in the documentation for that function as a @retval clause.
  • Confirm that all return codes specified by a @retval clause also have a test case that covers them
  • Confirm that parameter documentation is appropriately marked @nonnull or @nonzero where applicable
  • Confirm that there is a test case that violates above rule to test function response w/bad input
  • Confirm that coverage tests are not "overloaded" (i.e. violate only one rule at a time when confirming status code response).

Additional context
See nasa/osal#980

Requester Info
Joseph Hickey, Vantage Systems, Inc.
@skliper skliper added this to the 7.0.0 milestone Jul 28, 2021
@skliper skliper added docs This change only affects documentation. unit-test labels Jul 28, 2021
@jphickey jphickey self-assigned this Jul 29, 2021
@jphickey
Copy link
Contributor Author

jphickey commented Jul 29, 2021 •
edited
Loading

Got the script running against CFE test logs, first set of results/issues for ES (unfortunately kind of a long list to look at):

CORRECTION - original list I posted was just the coverage logs. This updated set includes both coverage and functional test - results are better, but still a bunch missing.

OVERLOADED TEST: es_UT.c:3743 (09.047)
 --> matches CFE_ES_CreateChildTask[NULL@0]=>CFE_ES_BAD_ARGUMENT
 --> matches CFE_ES_CreateChildTask[NULL@1]=>CFE_ES_BAD_ARGUMENT
RETCODE: CFE_ES_AppID_ToIndex[CFE_ES_APPID_UNDEFINED@0]=>CFE_ES_ERR_RESOURCEID_NOT_VALID
  NOT VALIDATED!!
RETCODE: CFE_ES_AppID_ToIndex[NULL@1]=>CFE_ES_BAD_ARGUMENT
  NOT VALIDATED!!
RETCODE: CFE_ES_CopyToCDS[CFE_ES_CDS_BAD_HANDLE@0]=>CFE_ES_ERR_RESOURCEID_NOT_VALID
  NOT VALIDATED!!
RETCODE: CFE_ES_CopyToCDS[NULL@1]=>CFE_ES_BAD_ARGUMENT
 --> validated by es_cds_test.c:119 (03.006)
RETCODE: CFE_ES_CounterID_ToIndex[CFE_ES_COUNTERID_UNDEFINED@0]=>CFE_ES_ERR_RESOURCEID_NOT_VALID
  NOT VALIDATED!!
RETCODE: CFE_ES_CounterID_ToIndex[NULL@1]=>CFE_ES_BAD_ARGUMENT
  NOT VALIDATED!!
RETCODE: CFE_ES_CreateChildTask[NULL@0]=>CFE_ES_BAD_ARGUMENT
 --> validated by es_UT.c:3785 (09.045)
 --> validated by es_task_test.c:83 (16.005)
RETCODE: CFE_ES_CreateChildTask[NULL@1]=>CFE_ES_BAD_ARGUMENT
 --> validated by es_UT.c:3790 (09.046)
 --> validated by es_task_test.c:85 (16.006)
RETCODE: CFE_ES_CreateChildTask[NULL@2]=>CFE_ES_BAD_ARGUMENT
 --> validated by es_UT.c:3795 (09.047)
 --> validated by es_task_test.c:87 (16.007)
RETCODE: CFE_ES_CreateChildTask[ZERO@4]=>CFE_ES_BAD_ARGUMENT
  NOT VALIDATED!!
RETCODE: CFE_ES_DeleteApp[CFE_ES_APPID_UNDEFINED@0]=>CFE_ES_ERR_RESOURCEID_NOT_VALID
  NOT VALIDATED!!
RETCODE: CFE_ES_DeleteChildTask[CFE_ES_TASKID_UNDEFINED@0]=>CFE_ES_ERR_RESOURCEID_NOT_VALID
  NOT VALIDATED!!
RETCODE: CFE_ES_DeleteGenCounter[CFE_ES_COUNTERID_UNDEFINED@0]=>CFE_ES_ERR_RESOURCEID_NOT_VALID
  NOT VALIDATED!!
RETCODE: CFE_ES_GetAppIDByName[NULL@0]=>CFE_ES_BAD_ARGUMENT
 --> validated by es_UT.c:3711 (09.032)
 --> validated by es_info_test.c:127 (04.037)
RETCODE: CFE_ES_GetAppIDByName[NULL@1]=>CFE_ES_BAD_ARGUMENT
 --> validated by es_UT.c:3716 (09.033)
RETCODE: CFE_ES_GetAppID[NULL@0]=>CFE_ES_BAD_ARGUMENT
 --> validated by es_UT.c:3707 (09.031)
 --> validated by es_info_test.c:126 (04.036)
RETCODE: CFE_ES_GetAppInfo[CFE_ES_APPID_UNDEFINED@1]=>CFE_ES_ERR_RESOURCEID_NOT_VALID
  NOT VALIDATED!!
RETCODE: CFE_ES_GetAppInfo[NULL@0]=>CFE_ES_BAD_ARGUMENT
 --> validated by es_UT.c:1393 (04.069)
 --> validated by es_info_test.c:132 (04.041)
RETCODE: CFE_ES_GetAppName[CFE_ES_APPID_UNDEFINED@1]=>CFE_ES_ERR_RESOURCEID_NOT_VALID
  NOT VALIDATED!!
RETCODE: CFE_ES_GetAppName[NULL@0]=>CFE_ES_BAD_ARGUMENT
 --> validated by es_info_test.c:130 (04.039)
RETCODE: CFE_ES_GetCDSBlockIDByName[NULL@0]=>CFE_ES_BAD_ARGUMENT
 --> validated by es_cds_test.c:92 (02.009)
RETCODE: CFE_ES_GetCDSBlockIDByName[NULL@1]=>CFE_ES_BAD_ARGUMENT
 --> validated by es_UT.c:4474 (11.071)
 --> validated by es_cds_test.c:93 (02.010)
RETCODE: CFE_ES_GetCDSBlockName[CFE_ES_CDS_BAD_HANDLE@1]=>CFE_ES_ERR_RESOURCEID_NOT_VALID
  NOT VALIDATED!!
RETCODE: CFE_ES_GetCDSBlockName[NULL@0]=>CFE_ES_BAD_ARGUMENT
 --> validated by es_UT.c:4473 (11.070)
 --> validated by es_cds_test.c:87 (02.006)
RETCODE: CFE_ES_GetGenCount[CFE_ES_COUNTERID_UNDEFINED@0]=>CFE_ES_ERR_RESOURCEID_NOT_VALID
  NOT VALIDATED!!
RETCODE: CFE_ES_GetGenCount[NULL@1]=>CFE_ES_BAD_ARGUMENT
 --> validated by es_UT.c:4142 (10.024)
RETCODE: CFE_ES_GetGenCounterIDByName[NULL@0]=>CFE_ES_BAD_ARGUMENT
 --> validated by es_UT.c:4148 (10.025)
RETCODE: CFE_ES_GetGenCounterIDByName[NULL@1]=>CFE_ES_BAD_ARGUMENT
 --> validated by es_UT.c:4162 (10.033)
RETCODE: CFE_ES_GetGenCounterName[CFE_ES_COUNTERID_UNDEFINED@1]=>CFE_ES_ERR_RESOURCEID_NOT_VALID
  NOT VALIDATED!!
RETCODE: CFE_ES_GetGenCounterName[NULL@0]=>CFE_ES_BAD_ARGUMENT
 --> validated by es_UT.c:4161 (10.032)
RETCODE: CFE_ES_GetLibIDByName[NULL@0]=>CFE_ES_BAD_ARGUMENT
 --> validated by es_info_test.c:215 (06.024)
RETCODE: CFE_ES_GetLibIDByName[NULL@1]=>CFE_ES_BAD_ARGUMENT
 --> validated by es_UT.c:1897 (05.029)
RETCODE: CFE_ES_GetLibInfo[CFE_ES_LIBID_UNDEFINED@1]=>CFE_ES_ERR_RESOURCEID_NOT_VALID
  NOT VALIDATED!!
RETCODE: CFE_ES_GetLibInfo[NULL@0]=>CFE_ES_BAD_ARGUMENT
 --> validated by es_info_test.c:216 (06.025)
RETCODE: CFE_ES_GetLibName[CFE_ES_LIBID_UNDEFINED@1]=>CFE_ES_ERR_RESOURCEID_NOT_VALID
  NOT VALIDATED!!
RETCODE: CFE_ES_GetLibName[NULL@0]=>CFE_ES_BAD_ARGUMENT
 --> validated by es_UT.c:1896 (05.028)
 --> validated by es_info_test.c:219 (06.027)
RETCODE: CFE_ES_GetMemPoolStats[CFE_ES_MEMHANDLE_UNDEFINED@1]=>CFE_ES_ERR_RESOURCEID_NOT_VALID
  NOT VALIDATED!!
RETCODE: CFE_ES_GetMemPoolStats[NULL@0]=>CFE_ES_BAD_ARGUMENT
  NOT VALIDATED!!
RETCODE: CFE_ES_GetModuleInfo[NULL@0]=>CFE_ES_BAD_ARGUMENT
 --> validated by es_info_test.c:261 (08.010)
RETCODE: CFE_ES_GetPoolBufInfo[CFE_ES_MEMHANDLE_UNDEFINED@0]=>CFE_ES_ERR_RESOURCEID_NOT_VALID
  NOT VALIDATED!!
RETCODE: CFE_ES_GetPoolBufInfo[NULL@1]=>CFE_ES_BAD_ARGUMENT
 --> validated by es_mempool_test.c:103 (11.006)
RETCODE: CFE_ES_GetPoolBuf[CFE_ES_MEMHANDLE_UNDEFINED@1]=>CFE_ES_ERR_RESOURCEID_NOT_VALID
  NOT VALIDATED!!
RETCODE: CFE_ES_GetPoolBuf[NULL@0]=>CFE_ES_BAD_ARGUMENT
 --> validated by es_mempool_test.c:74 (10.003)
RETCODE: CFE_ES_GetPoolBuf[ZERO@2]=>CFE_ES_BAD_ARGUMENT
  NOT VALIDATED!!
RETCODE: CFE_ES_GetTaskIDByName[NULL@0]=>CFE_ES_BAD_ARGUMENT
 --> validated by es_task_test.c:116 (17.006)
RETCODE: CFE_ES_GetTaskIDByName[NULL@1]=>CFE_ES_BAD_ARGUMENT
 --> validated by es_UT.c:4035 (09.084)
RETCODE: CFE_ES_GetTaskID[NULL@0]=>CFE_ES_BAD_ARGUMENT
 --> validated by es_info_test.c:162 (05.013)
RETCODE: CFE_ES_GetTaskInfo[CFE_ES_TASKID_UNDEFINED@1]=>CFE_ES_ERR_RESOURCEID_NOT_VALID
  NOT VALIDATED!!
RETCODE: CFE_ES_GetTaskInfo[NULL@0]=>CFE_ES_BAD_ARGUMENT
 --> validated by es_UT.c:3739 (09.037)
 --> validated by es_info_test.c:161 (05.012)
RETCODE: CFE_ES_GetTaskName[CFE_ES_TASKID_UNDEFINED@1]=>CFE_ES_ERR_RESOURCEID_NOT_VALID
  NOT VALIDATED!!
RETCODE: CFE_ES_GetTaskName[NULL@0]=>CFE_ES_BAD_ARGUMENT
 --> validated by es_UT.c:4026 (09.079)
 --> validated by es_task_test.c:120 (17.009)
RETCODE: CFE_ES_IncrementGenCounter[CFE_ES_COUNTERID_UNDEFINED@0]=>CFE_ES_ERR_RESOURCEID_NOT_VALID
  NOT VALIDATED!!
RETCODE: CFE_ES_LibID_ToIndex[CFE_ES_LIBID_UNDEFINED@0]=>CFE_ES_ERR_RESOURCEID_NOT_VALID
  NOT VALIDATED!!
RETCODE: CFE_ES_LibID_ToIndex[NULL@1]=>CFE_ES_BAD_ARGUMENT
  NOT VALIDATED!!
RETCODE: CFE_ES_PoolCreateEx[NULL@0]=>CFE_ES_BAD_ARGUMENT
 --> validated by es_UT.c:4707 (14.022)
 --> validated by es_mempool_test.c:54 (09.010)
RETCODE: CFE_ES_PoolCreateEx[NULL@1]=>CFE_ES_BAD_ARGUMENT
 --> validated by es_UT.c:4711 (14.023)
 --> validated by es_mempool_test.c:55 (09.011)
RETCODE: CFE_ES_PoolCreateEx[ZERO@2]=>CFE_ES_BAD_ARGUMENT
 --> validated by es_mempool_test.c:56 (09.012)
RETCODE: CFE_ES_PoolCreateNoSem[NULL@0]=>CFE_ES_BAD_ARGUMENT
 --> validated by es_mempool_test.c:44 (09.002)
RETCODE: CFE_ES_PoolCreateNoSem[NULL@1]=>CFE_ES_BAD_ARGUMENT
 --> validated by es_mempool_test.c:45 (09.003)
RETCODE: CFE_ES_PoolCreateNoSem[ZERO@2]=>CFE_ES_BAD_ARGUMENT
 --> validated by es_UT.c:4608 (14.001)
 --> validated by es_mempool_test.c:46 (09.004)
RETCODE: CFE_ES_PoolCreate[NULL@0]=>CFE_ES_BAD_ARGUMENT
 --> validated by es_mempool_test.c:49 (09.006)
RETCODE: CFE_ES_PoolCreate[NULL@1]=>CFE_ES_BAD_ARGUMENT
 --> validated by es_mempool_test.c:50 (09.007)
RETCODE: CFE_ES_PoolCreate[ZERO@2]=>CFE_ES_BAD_ARGUMENT
 --> validated by es_UT.c:4614 (14.003)
 --> validated by es_mempool_test.c:51 (09.008)
RETCODE: CFE_ES_PoolDelete[CFE_ES_MEMHANDLE_UNDEFINED@0]=>CFE_ES_ERR_RESOURCEID_NOT_VALID
  NOT VALIDATED!!
RETCODE: CFE_ES_PutPoolBuf[CFE_ES_MEMHANDLE_UNDEFINED@0]=>CFE_ES_ERR_RESOURCEID_NOT_VALID
  NOT VALIDATED!!
RETCODE: CFE_ES_PutPoolBuf[NULL@1]=>CFE_ES_BAD_ARGUMENT
 --> validated by es_mempool_test.c:123 (12.006)
RETCODE: CFE_ES_RegisterCDS[NULL@0]=>CFE_ES_BAD_ARGUMENT
 --> validated by es_UT.c:4210 (11.010)
 --> validated by es_cds_test.c:61 (01.003)
RETCODE: CFE_ES_RegisterCDS[NULL@2]=>CFE_ES_BAD_ARGUMENT
 --> validated by es_UT.c:4214 (11.011)
 --> validated by es_cds_test.c:63 (01.005)
RETCODE: CFE_ES_RegisterCDS[ZERO@1]=>CFE_ES_BAD_ARGUMENT
  NOT VALIDATED!!
RETCODE: CFE_ES_RegisterGenCounter[NULL@0]=>CFE_ES_BAD_ARGUMENT
 --> validated by es_UT.c:4121 (10.019)
RETCODE: CFE_ES_RegisterGenCounter[NULL@1]=>CFE_ES_BAD_ARGUMENT
 --> validated by es_UT.c:4126 (10.020)
RETCODE: CFE_ES_ReloadApp[CFE_ES_APPID_UNDEFINED@0]=>CFE_ES_ERR_RESOURCEID_NOT_VALID
  NOT VALIDATED!!
RETCODE: CFE_ES_ReloadApp[NULL@1]=>CFE_ES_BAD_ARGUMENT
  NOT VALIDATED!!
RETCODE: CFE_ES_RestartApp[CFE_ES_APPID_UNDEFINED@0]=>CFE_ES_ERR_RESOURCEID_NOT_VALID
  NOT VALIDATED!!
RETCODE: CFE_ES_RestoreFromCDS[CFE_ES_CDS_BAD_HANDLE@1]=>CFE_ES_ERR_RESOURCEID_NOT_VALID
  NOT VALIDATED!!
RETCODE: CFE_ES_RestoreFromCDS[NULL@0]=>CFE_ES_BAD_ARGUMENT
 --> validated by es_cds_test.c:122 (03.008)
RETCODE: CFE_ES_SetGenCount[CFE_ES_COUNTERID_UNDEFINED@0]=>CFE_ES_ERR_RESOURCEID_NOT_VALID
  NOT VALIDATED!!
RETCODE: CFE_ES_TaskID_ToIndex[CFE_ES_TASKID_UNDEFINED@0]=>CFE_ES_ERR_RESOURCEID_NOT_VALID
  NOT VALIDATED!!
RETCODE: CFE_ES_TaskID_ToIndex[NULL@1]=>CFE_ES_BAD_ARGUMENT
  NOT VALIDATED!!
RETCODE: CFE_ES_WriteToSysLog[NULL@0]=>CFE_ES_BAD_ARGUMENT
  NOT VALIDATED!!

jphickey added a commit to jphickey/cFE that referenced this issue Aug 10, 2021
@jphickey
Partial nasa#1724, update in/out status and nonnull/nonzero tags
35c502f 
First pass at API doxygen scrub, this updates the in/out markings
on parameters where they were not correct, and adds nonnull and
nonzero tags where appropriate.
@jphickey jphickey linked a pull request Aug 10, 2021 that will close this issue
Partial #1724, update in/out status and nonnull/nonzero tags #1798
Merged
@jphickey jphickey removed a link to a pull request Aug 10, 2021
Partial #1724, update in/out status and nonnull/nonzero tags #1798
Merged
@jphickey jphickey mentioned this issue Aug 10, 2021
Merged
jphickey added a commit to jphickey/cFE that referenced this issue Aug 11, 2021
@jphickey
Partial nasa#1724, update in/out status and nonnull/nonzero tags
98bc267 
First pass at API doxygen scrub, this updates the in/out markings
on parameters where they were not correct, and adds nonnull and
nonzero tags where appropriate.
jphickey added a commit to jphickey/cFE that referenced this issue Aug 11, 2021
@jphickey
Partial nasa#1724, update in/out status and nonnull/nonzero tags
052ad37 
First pass at API doxygen scrub, this updates the in/out markings
on parameters where they were not correct, and adds nonnull and
nonzero tags where appropriate.
jphickey added a commit to jphickey/cFE that referenced this issue Aug 11, 2021
@jphickey
Partial nasa#1724, update in/out status and nonnull/nonzero tags
4703123 
First pass at API doxygen scrub, this updates the in/out markings
on parameters where they were not correct, and adds nonnull and
nonzero tags where appropriate.
This was referenced Aug 13, 2021
Closed
Open
astrogeco added a commit that referenced this issue Aug 16, 2021
@astrogeco
Merge pull request #1798 from jphickey/fix-1724-api-doxygen-markup
59aa9c9 
Partial #1724, update in/out status and nonnull/nonzero tags
@astrogeco astrogeco mentioned this issue Aug 16, 2021
Merged
@astrogeco
Copy link
Contributor

#1833 is in integration-candidate now

This was referenced Aug 19, 2021
Closed
Closed
Closed
Open
Closed
Closed
Closed
jphickey added a commit to jphickey/cFE that referenced this issue Aug 23, 2021
@jphickey
Fix nasa#1812, Add SB API test cases
c24daf4 
Adds SB functional test cases to cover all missing items that
were identified as part of the scrub in issue nasa#1724.

Where a specific condition is not testable because it requires
a failure of another subsystem, it is marked as `covtest` to
indicate it is only verifiable in coverage test environment.
jphickey added a commit to jphickey/cFE that referenced this issue Aug 23, 2021
@jphickey
Fix nasa#1866, Add ES API test cases
0f569ff 
Adds ES functional test cases to cover all missing items that
were identified as part of the scrub in issue nasa#1724.

Where a specific condition is not testable because it requires
a failure of another subsystem, it is marked as `covtest` to
indicate it is only verifiable in coverage test environment.
jphickey added a commit to jphickey/cFE that referenced this issue Aug 23, 2021
@jphickey
Fix nasa#1867, Add FS API test cases
19e5f2f 
Adds FS functional test cases to cover all missing items that
were identified as part of the scrub in issue nasa#1724.

Where a specific condition is not testable because it requires
a failure of another subsystem, it is marked as `covtest` to
indicate it is only verifiable in coverage test environment.
jphickey added a commit to jphickey/cFE that referenced this issue Aug 23, 2021
@jphickey
Fix nasa#1868, Add TBL API test cases
fb7cd23 
Adds TBL functional test cases to cover all missing items that
were identified as part of the scrub in issue nasa#1724.

Where a specific condition is not testable because it requires
a failure of another subsystem, it is marked as `covtest` to
indicate it is only verifiable in coverage test environment.
This was referenced Aug 23, 2021
Merged
Merged
Merged
Merged
Open
Open
@jphickey
Copy link
Contributor Author

jphickey commented Aug 23, 2021 •
edited
Loading

After all above fixes/improvements, here are the remaining outstanding issues from the scrub

For EVS: #1817
For ES: #1874
For TBL: #1875

These remaining cases are not easily implemented with the current framework. Some need a secondary test app (see #1761), some need a state that is not under direct control of the test

@skliper skliper closed this as completed Aug 24, 2021
@skliper skliper removed this from the 7.0.0 milestone Sep 24, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@jphickey jphickey

Labels
docs This change only affects documentation. duplicate enhancement unit-test
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@jphickey @skliper @astrogeco