[RLlib] New API Stack: TensorFlow still default for DreamerV3

[MMXLVIII]

Description

Hey,

the page https://docs.ray.io/en/master/rllib/new-api-stack-migration-guide.html
states that:
"RLlib supports a single deep learning framework, the PyTorch framework, dropping TensorFlow support entirely."

However, when i try to run DreamerV3 normally, i still get tensorflow errors. When setting: config.framework("torch") gives an error
ValueError: The framework torch is not supported.

So I assume either the docs are wrong, or something is wrong with DreamerV3? Can i run it with pytorch somehow? Otherwise i will try to work on my tensorflow errors, but i assumed, it could be pointless if i should use pytorch anyways.

The code:

from pprint import pprint

import gymnasium as gym
import numpy as np

from ray.rllib.algorithms.ppo import PPOConfig
from ray.rllib.algorithms.dreamerv3 import DreamerV3Config
from ray import init

init(num_gpus=1)

env_name = "CartPole-v1"
# Use the vector env API.
env = gym.make_vec(env_name, num_envs=1, vectorization_mode="sync")

config = (
    DreamerV3Config()
    .environment("CartPole-v1")
    .training(
        model_size="XS",
        training_ratio=1024,
    )
    .env_runners(
        num_env_runners=1 # https://github.com/ray-project/ray/issues/47527
    )
    .resources(num_gpus=1)
)

# Make sure you always set the framework to "torch"...
# See https://docs.ray.io/en/latest/rllib/new-api-stack-migration-guide.html
config.framework("torch")

# ... and drop all tf-specific settings.
""" config.framework(
    eager_tracing=True,
    eager_max_retraces=20,
    tf_session_args={},
    local_tf_session_args={},
) """

print("TEST: " ,config.num_gpus_per_env_runner)
config.num_gpus_per_env_runner = 1

algo = config.build_algo()

for i in range(1):
    result = algo.train()
    result.pop("config")
    pprint(result)

Link

No response