v1.17: gossip: process duplicate proofs for merkle root conflicts (backport of #34066)

[mergify]

This is an automatic backport of pull request #34066 done by Mergify.
Cherry-pick of ca6ab08 has failed:

On branch mergify/bp/v1.17/pr-34066
Your branch is up to date with 'origin/v1.17'.

You are currently cherry-picking commit ca6ab08555.
  (fix conflicts and run "git cherry-pick --continue")
  (use "git cherry-pick --skip" to skip this patch)
  (use "git cherry-pick --abort" to cancel the cherry-pick operation)

Changes to be committed:
	modified:   gossip/src/duplicate_shred.rs

Unmerged paths:
  (use "git add <file>..." to mark resolution)
	both modified:   gossip/src/cluster_info.rs

To fix up this pull request, you can check it out locally. See documentation: https://docs.github.com/en/github/collaborating-with-pull-requests/reviewing-changes-in-pull-requests/checking-out-pull-requests-locally

---

Mergify commands and options

More conditions and actions can be found in the documentation.

You can also trigger Mergify actions by commenting on this pull request:

• @Mergifyio refresh will re-evaluate the rules
• @Mergifyio rebase will rebase this PR on its base branch
• @Mergifyio update will merge the base branch into this PR
• @Mergifyio backport <destination> will backport this PR on <destination> branch

Additionally, on Mergify dashboard you can:

• look at your merge queues
• generate the Mergify configuration with the config editor.

Finally, you can contact us on https://mergify.com

[AshwinSekar]

need to merge #34291 first

[AshwinSekar]

necessary for the backport chain to #34270

[behzadnouri]

What was the merge conflict here?

In the future, please don't squash the commits so we can see the merge conflict and how it was resolved.

[AshwinSekar]

sorry bad habit. The merge conflict was the ABI on cluster info

[behzadnouri]

Probably I should have asked this on the original commit, but what happens if during the upgrade some % of the cluster considers 2 shreds as a duplicate proof but the rest of the cluster without this code does not recognize those as duplicates?

[AshwinSekar]

The part of the cluster without the code will be unable to deserialize the duplicate proof thinking that it is invalid. I assume this will also hurt the propagation of such proofs in gossip.

Note that we don't actually consume this proof in fork choice yet, #32963. So there shouldn't be a divergence there.

For rollout I think we should wait to consume the proof until the majority of the cluster can process it through gossip.