question about pos_embed in tiny roma

[huang583824382]

Hi @Parskatt , thanks for your wonderful job! But I have a question regarding calculation of P_lowres. Why are the best_match indices concatenated instead of the best_match scores for calculating the softmax?

RoMa/romatch/models/tiny.py

Lines 132 to 135 in 82a3396

	best_match = cv.reshape(B,H1*W1,H0,W0).argmax(dim=1) # B, HW, H, W
	P_lowres = torch.cat((cv[:,::down,::down].reshape(B,H1*W1 // down**2,H0,W0), best_match[:,None]),dim=1).softmax(dim=1)
	pos_embeddings = torch.einsum('bchw,cd->bdhw', P_lowres[:,:-1], grid_lr)
	pos_embeddings += P_lowres[:,-1] * grid[best_match].permute(0,3,1,2)

[Parskatt]

Lol that seems bugged yeah, it should be cv[best_match] or something like that right? I made tiny roma pretty quickly so might still be bugs. Could you try the correct version?

[huang583824382]

Lol that seems bugged yeah, it should be cv[best_match] or something like that right? I made tiny roma pretty quickly so might still be bugs. Could you try the correct version?

Yeah I think so, thank you for your reply! Maybe I'll try it later.