Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add PSP unit tests #14

Closed
skliper opened this issue Sep 25, 2019 · 49 comments
Closed

Add PSP unit tests #14

skliper opened this issue Sep 25, 2019 · 49 comments
Labels
duplicate This issue or pull request already exists

Comments

@skliper
Copy link
Contributor

skliper commented Sep 25, 2019

Currently, the JSC developed PSP unit tests are not included in the CFS release. They currently work with pc-linux and OrionSCP PSPs. Updates to so that they work with a VxWorks target PSP are in work (ut699-vxworks6.7 PSP)

These updates are being tracked in the JSC subversion repo and need to be pushed into a proper git branch and further work continued from there.
@skliper skliper self-assigned this Sep 25, 2019
@skliper
Copy link
Contributor Author

skliper commented Sep 25, 2019

Imported from trac issue 10. Created by sduran on 2015-02-26T14:12:17, last modified: 2019-08-14T14:13:14

@skliper
Copy link
Contributor Author

skliper commented Sep 25, 2019

Trac comment by sduran on 2015-05-05 16:00:01:

This branch also contains updates to the grut699 PSP - they had to be done here to get for the unit tests to work. They involve issues discovered by the unit test or parts of the PSP that were incomplete(exception handling) that had to be completed to have something to tests the unit tests against. These will have to be merged into trac-11.

@skliper skliper added the enhancement New feature or request label Sep 25, 2019
@skliper
Copy link
Contributor Author

skliper commented Sep 25, 2019

Trac comment by sduran on 2015-06-08 10:44:21:

Updates are no longer in JSC svn repo, but in bablefish branch trac-10-add-psp-unit-tests

@skliper
Copy link
Contributor Author

skliper commented Sep 25, 2019

Trac comment by sduran on 2015-06-08 10:47:51:

Not only are the updated PSP unit tests included, but VxWorks ut699 PSP updates are being made in order to pursue a Class A safety-critical classification of this software for use by Orion and potentially others. The CFS AES project performed static code analysis utilizing the Understand tool for all MISRA rules, ran static cppcheck tool for potential safety concerns, ran the existing unit tests on a vxworks/LEON3 target with WindRiver Coverage analysis enabled to determine unit test code coverage, and held a general code inspection process. Specific changes as a result will be as follows:

-Updating all ut699 PSP files to comply with "most" MISRA rules and cppcheck static analysis errors
-Updating all ut699 PSP files to remove compiler warnings
-Expanding existing unit tests to provide full API-level test coverage based on exposed uncovered code (black box testing)
-Adding a full ADDITIONAL unit test suite using UTAssert framework to provide full code and BRANCH level coverage of the ut699 PSP source code (white box testing)

Subsequent commits should address these issues.

@skliper
Copy link
Contributor Author

skliper commented Sep 25, 2019

Trac comment by gdecaruf on 2015-06-09 19:30:42:

Change to cfe_psp_start.c following Static Code analysis of cFE ES static code analysis.

  • Change to CFE_ES_Main function !StartupFileName argument casting from uint8 to char to satisfy Misra rule 6.1, where strings must be char type.
  • Corrections made to call in mac-osx, mcf5235-rtems, pc-cygwin, pc-linux

commit: [changeset:5a72a68]

@skliper
Copy link
Contributor Author

skliper commented Sep 25, 2019

Trac comment by sduran on 2015-06-10 15:15:06:

The change addressing MISRA rule 6.1 is in commit: 5a72a68

@skliper
Copy link
Contributor Author

skliper commented Sep 25, 2019

Trac comment by sduran on 2015-06-11 17:01:07:

Commit [changeset:6fd8801] addresses the MISRA rule violations.

@skliper
Copy link
Contributor Author

skliper commented Sep 25, 2019

Trac comment by dasp on 2015-08-01 15:43:48:

commit: [changeset:612f00f3] Initial white box tests for cfe_psp_memory.c

@skliper
Copy link
Contributor Author

skliper commented Sep 25, 2019 •
edited
Loading

Trac comment by abrown4 on 2015-08-07 17:04:58:

commit: [changeset:c210e81] Trac #14 Added ut-assert for white box PSP tests.
(Needed to build and run [changeset:612f00f3].)

Yes, this is a duplicate ut-assert, but it is put here for expediency. Later, ut-assert should be pulled up out of every individual app, osal, and psp that it supports, but for now it is safer and more efficient to commit it here.

@skliper
Copy link
Contributor Author

skliper commented Sep 25, 2019 •
edited
Loading

Trac comment by abrown4 on 2015-08-07 17:05:40:

commit: [changeset:4b9525c] Trac #14 added .gitignore for psp white box tests.

@skliper
Copy link
Contributor Author

skliper commented Sep 25, 2019

Trac comment by dasp on 2015-08-08 10:41:39:

commit: [changeset:ad4e7c5f] Initial white box unit tests for cfe_psp_start.c.
commit: [changeset:a7795ee1] Update to tests required to run on target

@skliper
Copy link
Contributor Author

skliper commented Sep 25, 2019

Trac comment by abrown4 on 2015-08-10 16:07:15:

commit: [changeset:a68d6f5] and [changeset:d91f334] cfe_psp_ram.c white box tests (linux and target).

@skliper
Copy link
Contributor Author

skliper commented Sep 25, 2019 •
edited
Loading

Trac comment by abrown4 on 2015-08-10 17:34:22:

commit: [changeset:c3b9703] Trac #14 Added null pointer tests for psp ram white-box test. See #34

@skliper
Copy link
Contributor Author

skliper commented Sep 25, 2019 •
edited
Loading

Trac comment by abrown4 on 2015-08-11 15:51:29:

commit: [changeset:a0a2a9b] & [changeset:05dfc40] Trac #14 Added white-box test for cfe_psp_memutils.c. See #35 #36

@skliper
Copy link
Contributor Author

skliper commented Sep 25, 2019 •
edited
Loading

Trac comment by dasp on 2015-08-11 17:19:53:

Created ticket #37 for cfe_psp_memory.c white box test violations.

@skliper
Copy link
Contributor Author

skliper commented Sep 25, 2019 •
edited
Loading

Trac comment by abrown4 on 2015-08-11 18:15:19:

commit: [changeset:ea19e38] Trac #14 Added white-box test for cfe_psp_memrange.c. See #38

@skliper
Copy link
Contributor Author

skliper commented Sep 25, 2019 •
edited
Loading

Trac comment by abrown4 on 2015-08-11 20:47:27:

commit: [changeset:091e92e] and [changeset:6ba4fcb] Trac #14 Added white box test for cfe_psp_exception.c.

@skliper
Copy link
Contributor Author

skliper commented Sep 25, 2019 •
edited
Loading

Trac comment by abrown4 on 2015-08-12 19:34:44:

comit: [changeset:2d93a56] Trac #14 Added white-box test for cfe_psp_support.c.

@skliper
Copy link
Contributor Author

skliper commented Sep 25, 2019

Trac comment by dasp on 2015-08-13 18:41:11:

commit: [changeset:998ebe4a] Initial white box unit tests for cfe_psp_watchdog.c

@skliper
Copy link
Contributor Author

skliper commented Sep 25, 2019

Trac comment by dasp on 2015-08-13 18:43:25:

commit: [changeset:eade7377] Initial white box unit tests for cfe_psp_timer.c

@skliper
Copy link
Contributor Author

skliper commented Sep 25, 2019 •
edited
Loading

Trac comment by dasp on 2015-08-13 18:46:20:

Created ticket #39 for issues identified during white box testing of cfe_psp_watchdog.c

@skliper
Copy link
Contributor Author

skliper commented Sep 25, 2019 •
edited
Loading

Trac comment by dasp on 2015-08-13 19:14:03:

Created ticket #40 for the issue identified during white box testing of cfe_psp_timer.c

@skliper
Copy link
Contributor Author

skliper commented Sep 25, 2019 •
edited
Loading

Trac comment by dasp on 2015-08-13 19:21:15:

Created ticket #41 for the issue identified during white box testing of cfe_psp_start.c

@skliper
Copy link
Contributor Author

skliper commented Sep 25, 2019 •
edited
Loading

Trac comment by abrown4 on 2015-08-17 15:42:24:

commit: [changeset:2234ae6] merged fixes for #34 and #36 into #14 branch.

@skliper
Copy link
Contributor Author

skliper commented Sep 25, 2019 •
edited
Loading

Trac comment by abrown4 on 2015-08-17 15:51:48:

commit: [changeset:1868407] Trac #38 Updated CFE_PSP_MemRangeSet() comments.

@skliper
Copy link
Contributor Author

skliper commented Sep 25, 2019 •
edited
Loading

Trac comment by abrown4 on 2015-08-17 17:29:26:

commit: [changeset:5563878] Trac #39 Added range-limited calaculation to CFE_PS_WatchdogSet().

@skliper
Copy link
Contributor Author

skliper commented Sep 25, 2019 •
edited
Loading

Trac comment by abrown4 on 2015-08-18 12:23:31:

commit: [changeset:1931394] Trac #41 Updated CFE_PSP_GetRestartType() to ignore null pointer. Updated white-box unit test for testing & added "()" for compiler warning.

@skliper
Copy link
Contributor Author

skliper commented Sep 25, 2019 •
edited
Loading

Trac comment by abrown4 on 2015-08-18 15:15:47:

commit: [changeset:319cce3] Merged changes from #35 and #37 into #14 branch.

@skliper
Copy link
Contributor Author

skliper commented Sep 25, 2019

Trac comment by jphickey on 2015-12-01 10:16:52:

I understand the genesis of this, but ultimately at the end of the day the CCB goal here is to release a source code package that can be distributed via SourceForge (or anywhere) that contains everything that a project would need to build, run, and test CFE on their target of choice.

If we don't include makefiles for the tests, then this is a problem.

In the "classic" GSFC build, the tarball/source repo has sample makefiles that are typically cloned and owned by the mission. While I don't advocate this approach (the clone/own part), at least it has all the necessary pieces of information within the source code repository such that someone can get it and build it.

We've gone over this time and time again - JSC uses their own custom makefiles, which is fine if you choose to do that, but as a CCB we cannot test against these since they are not in the CCB repository nor will they be included in a tarball when we finally release the code. We cannot expect non-JSC users to write the makefile from scratch though - we need to include something with the code.

@skliper
Copy link
Contributor Author

skliper commented Sep 25, 2019

Trac comment by sduran on 2016-02-08 15:26:33:

A build of the unit tests, with makefiles included in the PSP, has been accomplished. However, it does require several environment variables to be set(there are dependencies on cfe and osal .h files). The build also has dependencies on mission_inc, osconfig.h, and cfe_platform_cfg.h for a given psp. While the build does work, further though on where these files should be located and brought into the build is needed.

@skliper
Copy link
Contributor Author

skliper commented Sep 25, 2019

Trac comment by sduran on 2016-02-08 15:52:25:

Two specific changes in PSP trac-10 should be reviewed and considered for acceptance into PSPs. As the change is implemented, it has no impact to those that do not want to use it and is very useful to those that do. The changes of interest are in the following changes sets in psp trac-10.

commit ID: [changeset:d423ce6]

  • Adds VOLTAB macro to cfe_psp_voltab.c
  • Rather than only being able to use the default /cf physical location or having each developer make local changes to the voltab.c file, this modification allows the location of the /cf directory to be specified at build time(it could be provided on the command line or as we use the option at JSC, set in the top level makefile).
  • If the default location is acceptable, then developers do nothing new at build time.

commit ID: [changeset:44a5ba3]

  • Add += to TARGET_DEFS in compiler-opts.mak so that -DVOLTAB is included in build options

@skliper
Copy link
Contributor Author

skliper commented Sep 25, 2019

Trac comment by sduran on 2016-03-08 12:16:54:

Current compiler errors(per Susie's request) when building grut699-vxworks6 against the psp development branch - . The linux build, builds/runs with out issue. Nothing too complicate here, just have not had the time to trace it all down.

sparc-wrs-vxworks-gcc -I/home/sduran/COP_CFS_Workspace/osal/src/os/inc -I/home/sduran/COP_CFS_Workspace/cfe/fsw/cfe-core/src/inc -I/home/sduran/COP_CFS_Workspace/psp/fsw/inc -I/home/sduran/COP_CFS_Workspace/psp/fsw/grut699-vxworks6/inc -I/home/sduran/CFSTST_Workspace/CFS_TST/CFS_TST/build/mission_inc -I../../inc -Wa,-a=cfe_psp_memory.lis -D_EB -DENDIAN=EB -DSOFTWARE_BIG_BIT_ORDER -DVOLTAB_PATH='"/ram0/cf"' -D_VXWORKS_OS -D__SPARC__ -Dvxworks6 -D_EMBED_ -DTOOL_FAMILY=gnu -DTOOL=gnuv8 -D_WRS_KERNEL -DCPU=SPARC -fno-builtin -fvolatile -Wcast-align -m32 -mtune=ut699 -mv8 -ffloat-int-mode -I/gfe/vxworks/6.7/WindRiver/vxworks-6.7/target/h -I/gfe/vxworks/6.7/WindRiver/vxworks-6.7/target/h/wrn/coreip -I/gfe/vxworks/6.7/WindRiver/vxworks-6.7/target/h/drv -I/gfe/vxworks/6.7/WindRiver/vxworks-6.7/target/h/drv/os_ext -I/gfe/vxworks/6.7/WindRiver/vxworks-6.7/target/src/config -I/gfe/vxworks/6.7/WindRiver/vxworks-6.7/target/src/drv -I/gfe/vxworks/6.7/WindRiver/vxworks-6.7/target/config/comps/src -I/gfe/vxworks/6.7/WindRiver/vxworks-6.7/target/config/comps/src/dosfs2 -Wall -ansi -g -O0 -DOS_DEBUG_LEVEL=3 -c -o cfe_psp_memory.o /home/sduran/COP_CFS_Workspace/psp/fsw/grut699-vxworks6/src/cfe_psp_memory.c
In file included from /gfe/vxworks/6.7/WindRiver/vxworks-6.7/target/h/private/memPartLibP.h:55,
from /gfe/vxworks/6.7/WindRiver/vxworks-6.7/target/h/memPartLib.h:17,
from /gfe/vxworks/6.7/WindRiver/vxworks-6.7/target/h/moduleLib.h:63,
from /home/sduran/COP_CFS_Workspace/psp/fsw/grut699-vxworks6/src/cfe_psp_memory.c:32:
/gfe/vxworks/6.7/WindRiver/vxworks-6.7/target/h/vsbConfig.h:42:2: warning: #warning "VxWorks Source Build (VSB) project not specified; using default VxWorks UP configuration under $WIND_BASE/target/lib/h/config"
/home/sduran/COP_CFS_Workspace/psp/fsw/grut699-vxworks6/src/cfe_psp_memory.c:209: error: conflicting types for 'CFE_PSP_GetResetArea'
/home/sduran/COP_CFS_Workspace/psp/fsw/inc/cfe_psp.h:287: error: previous declaration of 'CFE_PSP_GetResetArea' was here
/home/sduran/COP_CFS_Workspace/psp/fsw/grut699-vxworks6/src/cfe_psp_memory.c: In function 'CFE_PSP_GetResetArea':
/home/sduran/COP_CFS_Workspace/psp/fsw/grut699-vxworks6/src/cfe_psp_memory.c:219: warning: cast increases required alignment of target type
/home/sduran/COP_CFS_Workspace/psp/fsw/grut699-vxworks6/src/cfe_psp_memory.c: At top level:
/home/sduran/COP_CFS_Workspace/psp/fsw/grut699-vxworks6/src/cfe_psp_memory.c:248: error: conflicting types for 'CFE_PSP_GetUserReservedArea'
/home/sduran/COP_CFS_Workspace/psp/fsw/inc/cfe_psp.h:294: error: previous declaration of 'CFE_PSP_GetUserReservedArea' was here
/home/sduran/COP_CFS_Workspace/psp/fsw/grut699-vxworks6/src/cfe_psp_memory.c: In function 'CFE_PSP_GetUserReservedArea':
/home/sduran/COP_CFS_Workspace/psp/fsw/grut699-vxworks6/src/cfe_psp_memory.c:258: warning: cast increases required alignment of target type
/home/sduran/COP_CFS_Workspace/psp/fsw/grut699-vxworks6/src/cfe_psp_memory.c: At top level:
/home/sduran/COP_CFS_Workspace/psp/fsw/grut699-vxworks6/src/cfe_psp_memory.c:287: error: conflicting types for 'CFE_PSP_GetVolatileDiskMem'
/home/sduran/COP_CFS_Workspace/psp/fsw/inc/cfe_psp.h:300: error: previous declaration of 'CFE_PSP_GetVolatileDiskMem' was here
/home/sduran/COP_CFS_Workspace/psp/fsw/grut699-vxworks6/src/cfe_psp_memory.c: In function 'CFE_PSP_GetVolatileDiskMem':
/home/sduran/COP_CFS_Workspace/psp/fsw/grut699-vxworks6/src/cfe_psp_memory.c:297: warning: cast increases required alignment of target type
/home/sduran/COP_CFS_Workspace/psp/fsw/grut699-vxworks6/src/cfe_psp_memory.c: At top level:
/home/sduran/COP_CFS_Workspace/psp/fsw/grut699-vxworks6/src/cfe_psp_memory.c:363: error: conflicting types for 'CFE_PSP_GetKernelTextSegmentInfo'
/home/sduran/COP_CFS_Workspace/psp/fsw/inc/cfe_psp.h:306: error: previous declaration of 'CFE_PSP_GetKernelTextSegmentInfo' was here
/home/sduran/COP_CFS_Workspace/psp/fsw/grut699-vxworks6/src/cfe_psp_memory.c:405: error: conflicting types for 'CFE_PSP_GetCFETextSegmentInfo'
/home/sduran/COP_CFS_Workspace/psp/fsw/inc/cfe_psp.h:311: error: previous declaration of 'CFE_PSP_GetCFETextSegmentInfo' was here
make[2]: *** [cfe_psp_memory.o] Error 1
make[2]: Leaving directory /home/sduran/CFSTST_Workspace/CFS_TST/CFS_TST/build/ut699-vxworks6/cfe/psp' gmake[1]: *** [all] Error 2 gmake[1]: Leaving directory /home/sduran/CFSTST_Workspace/CFS_TST/CFS_TST/build/ut699-vxworks6/cfe'
gmake: *** [one_cfs_build] Error 2
[jsc-er-gfe09:~/CFSTST_Workspace/CFS_TST/CFS_TST/build/ut699-vxworks6]$

@skliper
Copy link
Contributor Author

skliper commented Sep 25, 2019

Trac comment by jphickey on 2016-03-08 13:08:41:

This looks like a fairly straightforward error. The prototypes in cfe_psp.h were made to be type correct, so it uses the cpuaddr type instead of void* for returning the address. For instance, the prototype signature now looks like (for example):

{{{
int32 CFE_PSP_GetResetArea (cpuaddr *PtrToResetArea, uint32 *SizeOfResetArea)
}}}

(The first parameter is no longer void pointer type)

@skliper
Copy link
Contributor Author

skliper commented Sep 25, 2019

Trac comment by glimes on 2016-03-08 14:30:07:

I'm working with a GRUT-700, very similar; I will grab Steve's error messages,
move them into a new ticket, and provide an update to the PSP to resolve these
issues for these PSP APIs where parameter types were changed.

@skliper
Copy link
Contributor Author

skliper commented Sep 25, 2019 •
edited
Loading

Trac comment by glimes on 2016-03-08 14:38:22:

See also PSP ticket #56

@skliper
Copy link
Contributor Author

skliper commented Sep 25, 2019

Trac comment by jhageman on 2019-05-01 16:44:50:

Refocusing suggestion - use this ticket to integrate the PSP black box tests that would be useful for testing/certifying PSPs.

@skliper
Copy link
Contributor Author

skliper commented Sep 25, 2019

Trac comment by jphickey on 2019-05-01 16:50:56:

Just to make sure we are on the same page -- I thought the unit test being discussed was a "white box" coverage test in the form of the other GSFC unit tests.

Can we clarify what exactly a "black box" test would be? From the sounds of it, This would be a new methodology that is different than what is in CFE.

@skliper
Copy link
Contributor Author

skliper commented Sep 25, 2019

Trac comment by jhageman on 2019-05-01 17:10:48:

I think of unit tests as coverage tests, which are by necessity "white box" and tightly coupled to the written code, and I wouldn't expect unit tests for PSPs to be common.

By "black box" I'm thinking fully exercising the API to show a given PSP conforms with the (possibly as-of-yet not fully defined) specification. This I would expect to be part of a common certification framework package to show the PSP behaves per the required specs (including error responses, nominal functionality, etc). I thought JSC had developed something like this (or at least a starting point) for the class A testing which would be a huge benefit to the community.

Jumping into the middle of all the history in this ticket I could easily be way off base. If the intent of this ticket was really just integrate a custom BSP's set of unit tests (tightly coupled to code and not a general testing solution), I'd suggest it isn't all that useful to control via the CCB (just make a techdev branch and close the ticket as invalid).

@skliper
Copy link
Contributor Author

skliper commented Sep 25, 2019

Trac comment by jhageman on 2019-05-06 09:46:07:

Note JSC branch name updates for naming standards compliance:

trac-10-add-psp-unit-tests -> techdev-pc-linux-uei
ic-sgd-dev-trac-10-merge -> techdev-sp0-vxworks6.9 (also likely contains latest tests)
add-ut700-vxworks6.9_psp -> techdev-ut700-vxworks6.9
additional_ut699_changes_from_trac-10 -> techdev-ut699-vxworks

This was referenced Sep 25, 2019
Closed
Closed
@skliper skliper removed their assignment Sep 26, 2019
This was referenced Sep 26, 2019
Closed
Closed
Closed
Closed
Closed
Closed
Closed
Closed
Closed
Closed
Closed
@skliper skliper mentioned this issue Jul 24, 2020
Merged
@skliper
Copy link
Contributor Author

skliper commented Jul 29, 2020

Closing as duplicate of #168, really the thread is OBE anyways.

@skliper skliper closed this as completed Jul 29, 2020
@skliper skliper added duplicate This issue or pull request already exists and removed enhancement New feature or request labels Jul 29, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
duplicate This issue or pull request already exists
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@skliper