self.pos_embedding error WHEN training on different dataset

[khawar-islam]

Please help me,

I am doing training on some different dataset using a pre-trained VIT model and trained on some different task but I got an error

model.py file

class PositionalEmbedding1D(nn.Module):
    """Adds (optionally learned) positional embeddings to the inputs."""

    def __init__(self, seq_len, dim):
        super().__init__()
        self.pos_embedding = nn.Parameter(torch.zeros(1, seq_len, dim))
    
    def forward(self, x):
        """Input has shape `(batch_size, seq_len, emb_dim)`"""
        return x + self.pos_embedding

Traceback

  result = self.forward(*input, **kwargs)
  File "/media/khawar/HDD_Khawar/n/Pretrained_ViT/pytorch_pretrained_vit/model.py", line 24, in forward
    return x + self.pos_embedding

[rwightman]

@khawar512 you aren't using the vit model from here

[khawar-islam]

I need a pre-trained VIT model. Please mention a link of pre-trained VIT code and model