[rllib] Fix AlgorithmConfig validation when using OfflinePreLearner

[johnPertoft]

Why are these changes needed?

When using an OfflinePreLearner the algorithm config validation fails.

Related issue number

#55415

Checks

• I've signed off every commit(by using the -s flag, i.e., git commit -s) in this PR.
• I've run scripts/format.sh to lint the changes in this PR.
• I've included any doc changes needed for https://docs.ray.io/en/master/.
  • I've added any new APIs to the API Reference. For example, if I added a
    method in Tune, I've added it in doc/source/tune/api/ under the
    corresponding .rst file.
• I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/
• Testing Strategy
  • Unit tests
  • Release tests
  • This PR is not tested :(

[gemini-code-assist]

Summary of Changes

Hello @johnPertoft, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

I've addressed an issue where the AlgorithmConfig validation was failing when an OfflinePreLearner was being used. The problem stemmed from an incorrect attribute being checked during the validation process, leading to erroneous validation failures.

Highlights

• AlgorithmConfig Validation Fix: I've resolved a bug where the AlgorithmConfig validation incorrectly failed when an OfflinePreLearner was configured.
• Attribute Correction: The validation logic in _validate_offline_settings was updated to correctly reference self.offline_eval_runner_class instead of the previously used self.prelearner_class when checking for OfflineEvaluationRunner inheritance.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point in your pull request via creating an issue comment (i.e. comment on the pull request page) using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in issue comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments or fill out our survey to provide feedback.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request correctly fixes a validation bug in AlgorithmConfig by checking self.offline_eval_runner_class instead of self.prelearner_class. I've added a comment pointing out a related issue where the evaluation() method seems to set a different property (self.offline_eval_runner_cls), which would make this new validation ineffective. Fixing that inconsistency will make this change fully effective.

[gemini-code-assist]

This change correctly fixes the validation logic to check self.offline_eval_runner_class. However, there appears to be a related issue in the evaluation() method. It seems to set self.offline_eval_runner_cls (with a _cls suffix) instead of self.offline_eval_runner_class when the offline_eval_runner_class parameter is provided.

Because of this, the validation here might not be effective, as self.offline_eval_runner_class may not be set correctly through the public API.

To ensure this validation works as intended, please consider correcting the property assignment in the evaluation() method (around line 2992) to use self.offline_eval_runner_class.

[theo2021]

@johnPertoft seems to me like this should also be changed to class instead of cls

[johnPertoft]

Thank you, I've updated it to include this change as well.

[johnPertoft]

Closing this as it has been fixed in #55693