gcc "--wrap" argument and stubs for OS-provided functions

[CDKnightNASA]

Is your feature request related to a problem? Please describe.
I've been frustrated that we don't have unit tests for checking that OSAL properly handles error conditions when calling OS functions (like open, read, socket, accept, etc.) Apparently if we define --wrap=open, then all calls to open() will go to our open() function and the OS-provided open will be renamed as __wrap_open(). I suggest this be used to stub most/all OS calls and we can add unit tests for such things as running out of memory/storage/file descriptors and also support unit testing in sandboxed environments (don't know if Travis CI sandboxes) where such things as network calls are disabled.

Describe the solution you'd like
Stub functions for OS functions.

Describe alternatives you've considered
Been looking around for alternatives and haven't seen any.

Additional context
Add any other context about the feature request here.

Requester Info
[email protected]

[jphickey]

But we do have OSAL coverage tests, in the "unit-test-coverage" directory.

GCC wrap options have been brought up before in years past. Problem is we do not want to rely on tool-specific features to run coverage tests.

The framework that exists today AFAIK can do all we need it to, so I'm not sure about why we would need to add new dependencies or other ways of doing the same things?

[CDKnightNASA]

It's my understanding that the OSAL unit tests check that OSAL functions handle bad inputs but do they also check that OSAL handles error conditions reported by the underlying OS call? And for some calls, it may be difficult/impossible to call OSAL functions in such a way as to force the underlying OS call to fail. Sure, I can call OS_open("/bin/bash", OS_READ_WRITE, 0644) which will force the open() call to return an error...Do we do this sort of thing?

[jphickey]

Yes, this is precisely the point of the unit-test-coverage directory. It redirects the OS/C library calls to alternative/mock functions which can be controlled like any other stub functions.

Note that historically this was a separate build, but this will be changing in #362 such that the coverage will be built whenever ENABLE_UNIT_TESTS is true.

Also note that only the shared and vxworks layers are implemented for full coverage; posix and rtems are not currently implemented (never were). So the caveat with always building the coverage tests is that you'll get a coverage test for vxworks regardless of what OS you actually use.

[jphickey]

But also note - this is a separate issue from simply having a functional test for a given API. The functional test uses the real function, not a mock/stub.

[skliper]

Note the goal is eventually ENABLE_UNIT_TESTS=true would build full coverage tests for all implementations (via mocks/stubs), but short term is just to add what's currently implemented.

[CDKnightNASA]

going to close this ticket and ruminate on this topic. :D

[jphickey]

Oh yeah, the other issue with --wrap gcc/ld feature is that it generally only works well for functions which you have an existing system header/prototype for. Meaning it doesn't work so well when building the e.g. VxWorks coverage test to run on a POSIX host (which we do).

The preprocessor-based remap done in the current UT coverage build, while a little complicated, does not depending on system headers at all, so you can build coverage test for anything regardless of what you are actually running it on.