Test a function with a passed in struct that can be mocked

[Ben-PH]

We are introducing mocks into our codebase, and I'm running into a problem.

We are testing a function that takes in a struct as an argument, but we want to mock that struct. If we use the instructions (mockall_double, etc...), the compiler type-errors (Expected Foo, got MockFoo).

The solution for now, is to mock FooTrait, and impl Foo for FooTrait, then make the argument generic:

fn uses_foo<F: Foo>(some_arg: (), foo: F) {
    ....
}

...this is okay-ish, but as we start using mockall throughout our codebase, we will start getting things like

struct LotsOfComponents<F: FooComponentTrait, B: BarComponentTrait, A: AlphaWorkerTrait, B: BravoWorkerTrait, ....> {
  foo: F,
  foo_worker: FW,
  some_name: A,
  another_name: B,
  running_out_of_names: C,
  how_many_generics_do_we_need: HMGDWN,
  ...
}

For reference: massalabs/massa#3745 (comment)

[Ben-PH]

I've created an issue that hypothesizes a workable approach> #479

[asomers]

You must be doing something wrong. What you describe is exactly what #[mockall_double] is intended for. I think you're probably importing your struct in two different ways in different modules. You're probably doing use mymod::MyStruct in one place, and #[mockall_double] use mymod::MyStruct somewhere else. You must be consistent.

[Ben-PH]

so this:

#[internal_mock]
struct Foo {
   bar: u32
}

#[internal_mock]
impl Foo {
    fn uses_bar(&self) -> u32 {
        self.bar
    }
}

...couldn't expand to

struct Foo {
    mocked: MockFoo
}
impl Foo {
    fn uses_bar(&self) -> {
        self.mocked.uses_bar()
    }

???

[asomers]

Oh, it could expand to that all right. But it wouldn't solve your problem, because this Foo would be a different type than the original Foo. You would get the same compiler errors, but they would be more confusing because the two types would have the same name. You would also get some "duplicate definition for struct Foo" errors, unless you put the new Foo into a submodule. And if you did that, then they wouldn't have the same name anymore.

[Ben-PH]

how about this:

...but there is only ever one Foo. it's either the unexpanded one, or the expanded one.

[asomers]

That wouldn't solve your problem, and may well make it worse, if you arrange your code typically. Because no code, even in the same module that defines Foo, would be able to access the real Foo in test mode.

[Ben-PH]

so what if it expands to:

struct Foo {
    bar: u32,
    mocked: Option<MockFoo>,
}

If it's to be used as a real Foo, then the mocked is set to None

[Ben-PH]

I'm revisiting our codebase.

usage:

    let mut mocked1 = NetworkCommandSender::default();
    let mut mocked2 = NetworkCommandSender::default();
    mocked2
        .expect_get_bootstrap_peers()
        .times(1)
        .returning(|| Ok(get_peers()));

    mocked1.expect_clone().return_once(move || mocked2);

imported with:

#[mockall_double::double]
use massa_network_exports::network_controller::NetworkCommandSender;

mocked in the network_controller mod:

#[cfg_attr(any(test, feature = "testing"), mockall::automock)]
impl NetworkCommandSender {

I tried re-exporting with mock_double in the structs parent module, then just importing naivly everywhere else (no use of mockdouble except at the re-export) and that seems to work :)

[Ben-PH]

...but this no longer works:

    let mut mocked1: NetworkCommandSender = NetworkCommandSender::default();
    let mut mocked2: NetworkCommandSender = NetworkCommandSender::default();

error:

error[E0599]: no function or associated item named `default` found for struct `NetworkCommandSender` in the current scope
   --> massa-bootstrap/src/tests/scenarios.rs:171:67
    |
171 |     let mut mocked1: NetworkCommandSender = NetworkCommandSender::default();
    |                                                                   ^^^^^^^ function or associated item not found in `NetworkCommandSender`

error[E0599]: no function or associated item named `default` found for struct `NetworkCommandSender` in the current scope
   --> massa-bootstrap/src/tests/scenarios.rs:172:67
    |
172 |     let mut mocked2: NetworkCommandSender = NetworkCommandSender::default();

I try something like this:

#[cfg(test)]
mockall::mock! {
    pub NetworkCommandsender {}
    impl Default for NetworkCommandSender {
        fn default() -> Self;
    }
}
#[cfg_attr(test, mockall::automock)]
impl NetworkCommandSender {

I get conflicting implementation error if i try to test-combile the buffer, and default is not implemented if I try to run a test that uses the mockall_double re-export.

If I do this:

#[cfg(test)]
mockall::mock! {

    pub NetworkCommandSender {
        async fn node_ban_by_ids(&self, ids: Vec<NodeId>) -> Result<(), NetworkError>;
        async fn node_ban_by_ips(&self, ips: Vec<IpAddr>) -> Result<(), NetworkError>;
        async fn add_to_whitelist(&self, ips: Vec<IpAddr>) -> Result<(), NetworkError>;
        async fn remove_from_whitelist(&self, ips: Vec<IpAddr>) -> Result<(), NetworkError>;
        async fn node_unban_by_ids(&self, ids: Vec<NodeId>) -> Result<(), NetworkError>;
        async fn node_unban_ips(&self, ips: Vec<IpAddr>) -> Result<(), NetworkError>;
        async fn send_block_info(&self, node: NodeId, info: Vec<(BlockId, BlockInfoReply)>) -> Result<(), NetworkError>;
        async fn ask_for_block_list(&self, list: HashMap<NodeId, Vec<(BlockId, AskForBlocksInfo)>>) -> Result<(), NetworkError>;
        async fn send_block_header(&self, node: NodeId, header: SecuredHeader) -> Result<(), NetworkError>;
        async fn get_peers(&self) -> Result<Peers, NetworkError>;
        async fn get_network_stats(&self) -> Result<NetworkStats, NetworkError>;
        async fn get_bootstrap_peers(&self) -> Result<BootstrapPeers, NetworkError>;
        async fn send_operations(&self, node: NodeId, operations: Vec<SecureShareOperation>) -> Result<(), NetworkError>;
        async fn announce_operations(&self, to_node: NodeId, batch: OperationPrefixIds) -> Result<(), NetworkError>;
        async fn send_ask_for_operations(&self, to_node: NodeId, wishlist: OperationPrefixIds) -> Result<(), NetworkError>;
        async fn send_endorsements(&self, node: NodeId, endorsements: Vec<SecureShareEndorsement>) -> Result<(), NetworkError>;
        async fn node_sign_message(&self, msg: Vec<u8>) -> Result<PubkeySig, NetworkError>;
        fn default() -> Self;
    }



}

#[cfg(not(test))]
impl NetworkCommandSender {
    async fn node_ban_by_ids(&self, ids: Vec<NodeId>) -> Result<(), NetworkError> {
        self.0
            .send(NetworkCommand::NodeBanByIds(ids))
            .await
            .map_err(|_| NetworkError::ChannelError("could not send BanId command".into()))?;
        Ok(())
    }

...that doesn't help

[Ben-PH]

...I tried this in the parent module (re-export the struct if not test, otherwise make that module public)

#[cfg(test)]
pub mod network_controller;
#[cfg(not(test))]
pub use network_controller::NetworkCommandSender;
#[cfg(not(test))]
mod network_controller;

and import it like so:

#[cfg(test)]
#[cfg_attr(test, mockall_double::double)]
use massa_network_exports::network_controller::NetworkCommandSender;

and I get this:

   Compiling massa_bootstrap v0.1.0 (/home/ben/rust/massa/massa-bootstrap)
error[E0432]: unresolved import `massa_network_exports::network_controller::MockNetworkCommandSender`
  --> massa-bootstrap/src/tests/scenarios.rs:39:5
   |
39 | use massa_network_exports::network_controller::NetworkCommandSender;
   |     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^--------------------
   |     |                                          |
   |     |                                          help: a similar name exists in the module: `NetworkCommandSender`
   |     no `MockNetworkCommandSender` in `network_controller`

[asomers]

#[cfg_attr(any(test, feature = "testing"), mockall::automock)]

This is a problem . #[mockall_double] does not know about the "testing" feature. If you want to use that, then you'll have to do the doubling yourself. Fortunately, it's easy:

#[cfg(any(test, feature = "testing"))]
use massa_network_exports::network_controller::MockNetworkCommandSender as NetworkCommandSender;
#[cfg(not(any(test, feature = "testing")))]
use massa_network_exports::network_controller::NetworkCommandSender;

error[E0599]: no function or associated item named default found for struct NetworkCommandSender in the current scope

This means that you tried to call the default method on the real struct, not the mock struct. Mockall always implements Default. Probably the above error was the cause of this one.

[Ben-PH]

I'm still getting the same problems when I remove the feature attribute, and only check if it's a test combile or not

[Ben-PH]

#[cfg_attr(test, mockall::automock)]
impl NetworkCommandSender {

and the following in both in both scenarios.rs and server.rs

#[cfg(test)]
use massa_network_exports::network_controller::MockNetworkCommandSender as NetworkCommandSender;
#[cfg(not(test))]
use massa_network_exports::network_controller::NetworkCommandSender;

i get this error:

error[E0432]: unresolved import `massa_network_exports::network_controller::MockNetworkCommandSender`
  --> massa-bootstrap/src/server.rs:41:5
   |
41 | use massa_network_exports::network_controller::MockNetworkCommandSender as NetworkCommandSender;
   |     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^------------------------^^^^^^^^^^^^^^^^^^^^^^^^
   |     |                                          |
   |     |                                          help: a similar name exists in the module: `NetworkCommandSender`
   |     no `MockNetworkCommandSender` in `network_controller`

error[E0432]: unresolved import `massa_network_exports::network_controller::MockNetworkCommandSender`
  --> massa-bootstrap/src/tests/scenarios.rs:38:5
   |
38 | use massa_network_exports::network_controller::MockNetworkCommandSender as NetworkCommandSender;
   |     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^------------------------^^^^^^^^^^^^^^^^^^^^^^^^
   |     |                                          |
   |     |                                          help: a similar name exists in the module: `NetworkCommandSender`
   |     no `MockNetworkCommandSender` in `network_controller`

[asomers]

This isn't a problem in Mockall. This is a problem with your use statements. Maybe you've got a pub use in a module somewhere. Maybe you're missing pub on NetworkCommandSender. I don't know. But you need to debug this the same way that you would debug any other unresolved import error.

[asomers]

By any chance, is NetworkCommandSender defined in a different crate than server.rs ?

[Ben-PH]

Correct.

[Ben-PH]

...it's part of a multi-crate workspace.

[asomers]

Then that's your problem. Because when you build a crate in test mode, its dependencies are not build in test mode. So what you want to do is probably to create a testing feature for the massa_network_exports crate and export mock objects when that feature is set, not just when test is set. Then in your consuming create, ensure that you use the dependency's testing feature whenever you build in test mode. Or, combine the two crates into one.

[Ben-PH]

That makes sense now, and explains why I was getting the errors, and why I was getting confused by them.

I'll put together a documentation PR. Hopefully it will reduce the chances of you having to go through this back-and-forth again.

[zephod77]

Looks like I have exactly the same issue. Can you expand on your answer a bit more. I didn't understand how to export mock objects.
In my case I have a crate called common_messaging that has a lib.rs that exports 3 modules one of which is called RawMessage.
RawMessage has an associated function called read_from_skt defined like this:
pub fn read_from_skt(zmq_skt: &MockableZmqSocket) -> Result<RawMessage, zmq::Error>
MockableZmqSocket is defined like this:

#[cfg(not(test))]                                                                   
  use zmq::Socket as MockableZmqSocket;                                               
  #[cfg(test)]                                                                        
  use tests::MockSocket as MockableZmqSocket; 

and tests::MockSocket is defined:

mod tests {
...
mock! {
    pub Socket {...}
}

so how do I export MockableZmqSocket or just MockSocket?

[asomers]

Are you trying to define mock objects in one crate and use them in another? Then you can't use #[cfg(test)], because external crates are never built in test mode.

[zephod77]

Correct. I have another crate that imports my RawMessage crate and it builds and runs fine but I need to unit test it.

I have a function that calls RawMessage::read_from_socket() and I get this error message when I run cargo test on it:

  --> src/net_agent.rs:84:60
   |
84 |             let rcvd_msg = match RawMessage::read_from_skt(&rep_skt) {
   |                                  ------------------------- ^^^^^^^^ expected struct `Socket`, found struct `MockSocket`
   |                                  |
   |                                  arguments to this function are incorrect
   |
   = note: expected reference `&Socket`
              found reference `&net_agent_unit_tests::MockSocket`
note: associated function defined here
  --> /devroot/acs_src/KACommonSrc/ltv_ka/rust/common_message/src/raw_message.rs:38:12
   |
38 |     pub fn read_from_skt(zmq_skt: &MockableZmqSocket) ->

which is the same issue as described in this discussion, correct?
The way I read you answer, is that I have to export MockSocket from my common_messaging crate using features. Is that correct? So I was asking about how to do that.