Should KDE performs on the two images separately?

[nnop]

Thanks for the great method which almost solved the WxBS problem.

I noticed symmetric matching is the default.

RoMa/roma/models/model_zoo/roma_models.py

Line 152 in 5052229

symmetric = True

and KDE is performed on all sampled matches from (H, 2*W) warp results:

RoMa/roma/models/matcher.py

Lines 475 to 486 in 5052229

	matches, certainty = (
	matches.reshape(-1, 4),
	certainty.reshape(-1),
	)
	expansion_factor = 4 if "balanced" in self.sample_mode else 1
	good_samples = torch.multinomial(certainty,
	num_samples = min(expansion_factor*num, len(certainty)),
	replacement=False)
	good_matches, good_certainty = matches[good_samples], certainty[good_samples]
	if "balanced" not in self.sample_mode:
	return good_matches, good_certainty
	density = kde(good_matches, std=0.1)

The positions and warping of image1 and image2 should be independent.
So, should KDE performs on the two images separately?

Another question:
Since, the sampling is from symmetric matching, will the results contain many near duplicated matches?

[Parskatt]

Kde is typically run after sparse sampling, and works on 4D coords. Symmetric shouldn't affect it.

[Parskatt]

Regarding duplicates, yes this is likely.

[nnop]

Kde is typically run after sparse sampling, and works on 4D coords. Symmetric shouldn't affect it.

Don't know if I described clear.
I mean the 4D coords in the other image shouldn't be taken into account for supporting. It not the problem of symmetric.
4D coords in the first (H, W) and the second (H, W) part are both in range [-1, 1].
Wouldn't directly computing the pairwise distance mistake the samples of the other image into account?

RoMa/roma/utils/kde.py

Line 6 in 5052229

scores = (-torch.cdist(x,x)**2/(2*std**2)).exp()

[Parskatt]

I'm not sure I understand the question unfortunately. Why would we not want to take the other coordinates into account?
We simply view the correspondences as 4D vectors. We have a bunch of those, and we want to reduce sampling in regions where we have high density. It doesn't matter where these vectors come from.

[nnop]

Suppose the red dot and the green dot have the same coordinates in Image 1 and 2. Then the matches represented by the yellow line and the blue line should be equal in the 4D match space. Therefore, does the yellow match act as an inhibition to the blue match?

[Parskatt]

No, because all corresps are from A to B even (we switch the ordering internally)

[nnop]

That clears it up for me, thank you for clarifying my misunderstanding of this representation!