Mock input arguments not matching after v1.11.0

[knightpp]

Description

After v1.11.0 some tests in my project started to fail. It can be fixed by downgrading to v1.10.0. The test setups expectations ignoring the first argument by passing mock.Anything and passing pointer to a protobuf struct as the second argument.

Step To Reproduce

Not yet

Expected behavior

It works with v1.10.0, so should still work after update. Alternatively, I'd expect Diff: No differences. to show something if the call didn't match.

Actual behavior

  [FAILED]

  mock: Unexpected Method Call
  -----------------------------

  GetTopN(*context.cancelCtx,*some.ProtoStruct)
                0: &context.cancelCtx{...}
                1: &some.ProtoStruct{state:impl.MessageState{NoUnkeyedLiterals:pragma.NoUnkeyedLiterals{}, DoNotCompare:pragma.DoNotCompare{}, DoNotCopy:pragma.DoNotCopy{}, atomicMessageInfo:(*impl.MessageInfo)(0xc0003757d8)}, unknownFields:[]uint8(nil), sizeCache:0, ...}

  The closest call I have is:

  GetTopN(string,*some.ProtoStruct)
                0: "mock.Anything"
                1: &some.ProtoStruct{state:impl.MessageState{NoUnkeyedLiterals:pragma.NoUnkeyedLiterals{}, DoNotCompare:pragma.DoNotCompare{}, DoNotCopy:pragma.DoNotCopy{}, atomicMessageInfo:(*impl.MessageInfo)(0xc0003757d8)}, unknownFields:[]uint8(nil), sizeCache:0, ...}


  Diff: No differences.
  at: [/workspace/path/mocks.go:6589 ... /usr/local/go/src/runtime/asm_amd64.s:1693]

  In [It] at: /workspace/vendor/github.com/stretchr/testify/mock/mock.go:361 @ 08/26/25 15:32:27.778

[nhatthm]

I'm encountering a similar issue in v1.11.0, and after some debugging, I've identified the root cause.

Test Code

func TestArgumentShouldNotBeModified(t *testing.T) {
	m := &dummyMock{}
	m.Mock.Test(t)

	t.Cleanup(func() { m.AssertExpectations(t) })

	m.On("SendMsg", &grpctest.Item{Id: 42})
	m.On("SendMsg", &grpctest.Item{Id: 43})

	m.SendMsg(&grpctest.Item{Id: 42})
	m.SendMsg(&grpctest.Item{Id: 43})
}

type dummyMock struct {
	mock.Mock
}

func (m *dummyMock) SendMsg(v any) {
	m.Called(v)
}

grpctest.Item is a generated proto Message, you can replace it with any proto message.

Observation

• m.SendMsg(&grpctest.Item{Id: 42}) → ✅ Passes
• m.SendMsg(&grpctest.Item{Id: 43}) → 🔴 Fails unexpectedly

Root cause

• The first call with Id: 42 matches the expectation and proceeds without issue.
• On the second call with Id: 43
  • The argument is first compared to the previous expectation Id: 42
    • Because it doesn't match, this change in v1.11.0 triggers the String() function of the proto message that eventually populates the internal message state (protoimpl.MessageState)
      

      testify/mock/mock.go

      Line 987 in f4f5960

      return fmt.Sprintf("(%[1]T=%[1]v)", actual)

  • The same argument is then compared to the second expectation Id: 43
  • Due to the modified internal state, reflect.DeepEqual returns false, even though the actual field values match.

Update: It seems that the state is also populated in v1.10.0 but it doesn't affect the comparison for some reason 🤔

Side effects

Because the input is a pointer and gets mutated during execution, it may cause unexpected behavior in other parts, even outside the mock package.

[marcind]

You're on the right track @nhatthm. Here's what happens in the mock case you provided:

1. First m.SendMsg(&grpctest.Item{Id: 42}) is called.
2. The mock runs through the list of registered expectations to find a match in

   testify/mock/mock.go

   Line 391 in f4f5960

   for i, call := range m.ExpectedCalls {

3. The first expectation is a match due to assert.ObjectsAreEqual returning true in

   testify/mock/mock.go

   Line 1072 in f4f5960

   if assert.ObjectsAreEqual(expected, Anything) || assert.ObjectsAreEqual(actual, Anything) || assert.ObjectsAreEqual(actual, expected) {

   and so in 1.11.0 neither the expected nor the actual value is formatted into a string
4. m.SendMsg(&grpctest.Item{Id: 43}) is called.
5. The mock once again runs through the list of registered expectations.
   1. The first expectation is not a match (since 42 != 43). This causes the actual argument to be formatted into a string via

      testify/mock/mock.go

      Lines 986 to 988 in f4f5960

      	actualFmt = func() string {
      	return fmt.Sprintf("(%[1]T=%[1]v)", actual)
      	}

      . This ends up calling the object's String() method, which for protobuf has the unfortunate side-effect of populating some unexported fields.
   2. The second expectation is compared. Now assert.ObjectsAreEqual returns false because the actual value has the unexported fields set while the expected argument does not. This is because ultimately reflect.DeepEqual is used, which does not play well with protobuf models..

Here's a simpler repro of the underlying cause (using google.golang.org/protobuf/types/known/durationpb).

func Test(t *testing.T) {
	a := &durationpb.Duration{Seconds: 42}
	b := &durationpb.Duration{Seconds: 42}

	assert.Equal(t, a, b) // PASSES

	require.NotEmpty(t, a.String())

	assert.Equal(t, a, b) // FAILS UNEXPECTEDLY
}

In 1.10.0 both the actual and expected arguments were unconditionally formatted into strings, which caused their String() methods to be called, which caused the internal states to be populated with the same values, which allowed reflect.DeepEqual to appear to work correctly.

[marcind]

Looks like #758 is related. Not sure how to solve the current issue without rolling back the delayed string formatting optimization.

[brackendawson]

Diagnosis

Here is a program which demonstrates this issue without protobuf:

package kata_test

import (
	"fmt"
	"strconv"
	"testing"

	"github.com/stretchr/testify/mock"
)

type MyType struct {
	N int
	s string
}

func (m *MyType) String() string {
	fmt.Println("String", m.N)
	m.s = strconv.Itoa(m.N)
	return m.s
}

type MyMock struct {
	mock.Mock
}

func (m *MyMock) Do(typ *MyType) {
	m.Called(typ)
}

func TestIfy(t *testing.T) {
	m := MyMock{}
	m.On("Do", &MyType{N: 2}).Return()
	m.On("Do", &MyType{N: 1}).Return()

	a := &MyType{N: 1}
	b := &MyType{N: 2}
	fmt.Println("Before")
	m.Do(a)
	fmt.Println("Between")
	m.Do(b)
	fmt.Println("After")
}

This passes on v1.10.0 and fails on v1.11.0. The problem is as @nhatthm diagnosed (thank you), that mock mutates the expected calls by calling their stringer via fmt.

Testify's mock has always called the stringer and mutated these objects but this change was caused by #1615 which intended to lazily render the string representation of arguments only when they did not match, rather than every time they are compared. However my example shows that this does not work; if you have more than one expected call for a method then the arguments will nearly always be rendered.

When using v1.10.0; mock is still modifying the values passed to it, I've said before that I don't think testify should ever call methods like String or Equal on the values it tests because it makes it difficult or impossible to test those methods. Mock should have always used spew to format these strings.

How to fix

I think we should revert #1615 as it doesn't actually work for its intended purpose. I will create and protect a release/1.11 branch, backport the revert into it and make a 1.11.1 release. That will resolve this issue. We should then change mock to use spew rather than fmt, and then properly make the formatting lazy.

Workarounds

It's not very clean but you can force your tests to always use values whose stringer has been called:

package kata_test

import (
	"strconv"
	"testing"

	"github.com/stretchr/testify/mock"
)

type MyType struct {
	N int
	s string
}

func (m *MyType) String() string {
	m.s = strconv.Itoa(m.N)
	return m.s
}

type MyMock struct {
	mock.Mock
}

func (m *MyMock) Do(typ *MyType) {
	typ.String() // preemptively call stringer on actual args
	m.Called(typ)
}

func TestIfy(t *testing.T) {
	m := MyMock{}
	expectA := &MyType{N: 1}
	expectB := &MyType{N: 2}
	expectA.String() // preemptively call stringer on expected args
	expectB.String()
	m.On("Do", expectA).Return()
	m.On("Do", expectB).Return()

	a := &MyType{N: 1}
	b := &MyType{N: 2}
	m.Do(a)
	m.Do(b)
}

[nhatthm]

#643 (comment) that I don't think testify should ever call methods like String or Equal on the values it tests because it makes it difficult or impossible to test those methods. Mock should have always used spew to format these strings.

I think we should revert #1615 as it doesn't actually work for its intended purpose

Totally agree!

[brackendawson]

I've opened a PR to fix this on the main branch. I've created branch release/1.11 and protected release/* with the same rules as the main branch. Once the PR is merged I intend to back-port the fix into the release/1.11 branch and create a v1.11.1 release.

[brackendawson]

v1.11.1 is released, this should now be resolved. But be aware that v1.11.1 does still call the stringer interface, just like v1.10.0 did.

[nhatthm]

Thanks @brackendawson, v1.11.0 works!

[marcind]

@brackendawson perhaps consider retracting 1.11.0 in the next release?