[BUG] PPO with torchrl tutorial broken on Colab

[mikemykhaylov]

Describe the bug

When trying to run the tutorial on Colab, the following error is thrown when importing torchrl:

---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
[<ipython-input-4-8f198eb0e7fb>](https://localhost:8080/#) in <cell line: 9>()
      7 from torch import nn
      8 
----> 9 from torchrl.collectors import SyncDataCollector
     10 from torchrl.data.replay_buffers import ReplayBuffer
     11 from torchrl.data.replay_buffers.samplers import SamplerWithoutReplacement

8 frames
[/usr/local/lib/python3.10/dist-packages/torchrl/data/replay_buffers/samplers.py](https://localhost:8080/#) in <module>
     11 import torch
     12 
---> 13 from torchrl._torchrl import (
     14     MinSegmentTreeFp32,
     15     MinSegmentTreeFp64,

ImportError: /usr/local/lib/python3.10/dist-packages/torchrl/_torchrl.so: undefined symbol: _ZN5torch8autograd13_wrap_outputsERKSt6vectorIN2at6TensorESaIS3_EERKSt13unordered_setIPN3c1010TensorImplESt4hashISB_ESt8equal_toISB_ESaISB_EESJ_NS9_8ArrayRefINS9_8optionalIS3_EEEERKSt10shared_ptrINS0_4NodeEESt8functionIFS5_S5_S5_EESJ_

---------------------------------------------------------------------------
NOTE: If your import is failing due to a missing package, you can
manually install dependencies using either !pip or !apt.

To view examples of installing some common dependencies, click the
"Open Examples" button below.
---------------------------------------------------------------------------

To Reproduce

1. Import tutorial notebook on Colab
2. Uncomment code installing the packages
3. Run the code

Expected behavior

Error doesn't happen

Screenshots

If applicable, add screenshots to help explain your problem.

System info

Standard Google Colab En

Checklist

• I have checked that there is no similar issue in the repo (required)
• I have read the documentation (required)
• I have provided a minimal working example to reproduce the bug (required)

[vmoens]

Wild guess: Colab does not have torch 2.1. Can you confirm?

[mikemykhaylov]

Indeed, Colab seems to ship with Pytorch 2.0.1. For the time being, maybe add a pip install --upgrade torch to the Google Colab block in the tutorial?

[Sefank]

Maybe a hard requirement of Pytorch 2.1 is still too aggressive for now.

[vmoens]

I think the ideal scenario going forward would be:

• pip install torchrl upgrades PyTorch to the latest version
• pip install torchrl-nightly installs PyTorch nightlies
• python setup.py develop works with any existing PyTorch >=1.13

The first two are mandatory since it's the PyTorch version used to build the wheels (nothing else will work). But we make sure that local install work with previous versions of torch so we must ensure that there's a way to make it work!

This would also incidentally solve the notebook issue since you'd be installing the recent PyTorch with torchrl.

[skandermoalla]

Is it an option to build wheels for different versions of PyTorch and tag each wheel with the associated PyTorch version?
It doesn't seem right that TorchRL enforces a PyTorch version. How are other domain libraries doing?

[vmoens]

I don't think it is possible to tell PyPI that one version of a library has multiple sets of dependencies (at least I have never seen that in the past). Looking forward, the public wheels of v. in torchrl will always be tied to v<X+2>. in PyTorch I'm afraid 😧

[skandermoalla]

We can specify pytorch>=x in the runtime dependencies where x is the minimum PyTorch version needed, no?
And for the build dependencies, let it be controlled with an argument.

[skandermoalla]

It's also fine I guess if it is synced with a particular version of PyTorch. Users can still build a wheel with the PyTorch version they have in their environment. It would probably be more helpful to write somewhere the minimum PyTorch version needed. (E.g. torchrl 0.2 works fine with pytorch 2.0 AFAIK)

[mikemykhaylov]

Looks like Colab bumped pytorch to 2.1, tutorial works again

[vmoens]

I have made it so that torchrl depends on the latest pytorch version, hence running the tutorial with the binaries from #1642 will install the latest pytorch anyway.

[vmoens]

Torchrl now has a stronger dependency to PyTorch which means that you'll be able to install it in any Colab at any time in the future.
You can still downgrade PyTorch to a version >=1.13 if needed!