[BUG] Unintended and surprising slicing of tensors with ReplayBuffer extend and sample.

[whatdhack]

Describe the bug

memory = ReplayBuffer(storage=LazyTensorStorage(max_size=10000, device=device), sampler=SamplerWithoutReplacement(), )
t1x4 = torch.Tensor([0.1, 0.2, 0.3, 0.4]).to(device)
t1x1 = torch.Tensor([0.01]).to(device)
memory.extend([t1x4, t1x1, t1x4+0.4, t1x1+0.01])
samples = memory.sample(1)
print(samples)

The output as follows s unexpected

[tensor([0.4000], device='cuda:0'), tensor([0.0100], device='cuda:0'), tensor([0.8000], device='cuda:0'), tensor([0.0200], device='cuda:0')]

To Reproduce

see above

Expected behavior

should have been [t1x4, t1x1, t1x4+0.4, t1x1+0.01]. Appears to have sliced the [4] tensors.

Screenshots

N/A

System info

Describe the characteristic of your environment:

• Describe how the library was installed (pip, source, ...)
• Python version
• Versions of any other relevant libraries

import torchrl, numpy, sys
print(torchrl.__version__, numpy.__version__, sys.version, sys.platform)

0.3.0 1.26.1 3.10.13 (main, Sep 11 2023, 13:44:35) [GCC 11.2.0] linux

Additional context

None

Reason and Possible fixes

No

Checklist

• I have checked that there is no similar issue in the repo (required)
• I have read the documentation (required)
• I have provided a minimal working example to reproduce the bug (required)

[vmoens]

Thanks for posting this issue!
A few things to note here:
IIUC you are trying to put 4 items in the buffer but their shape is different. That goes against the intended usage of TensorStorage instances, which expect every item to have the same shape and dtype. If you want to store items with differnt shapes, I would recommend using a ListStorage or padding your values.

Regarding this example, I think that what is missing is a good error message, which will be solved by #1937