python3.11 support for plugins

[Jasha10]

Follow up PR to #2418.

Add python3.11 support for plugins (except ray launcher and ax sweeper).

[skshetry]

Hi @Jasha10, sorry to push for this, but when can we expect a new release? 🙂

[Jasha10]

Coming very soon! One or two days.

[Monrer11]

#@****__

[Pedrexus]

any updates on supporting 3.11 for ray launcher and ax sweeper as well? Do you need any help with that? I use ray launcher, but had to update all dataclasses manually to make it work with 3.11

[odelalleau]

Do you need any help with that?

Most likely yes, if someone could put up a PR there's a higher chance this may happen.

[Pedrexus]

Is there any reference on how to run the tests for plugins? I can run the base hydra tests fine, but the plugins tests fail as they don't find the plugin for some reason.

[odelalleau]

Is there any reference on how to run the tests for plugins?

@Jasha10 would you be able to help with that by any chance?

[Jasha10]

Sure.

@Pedrexus I run the plugins (for python3.10) with the following command:

$ nox -s test_plugins-3.10

Does that work for you?

References:

• The Hydra developer's guide overview has some tips on environment setup: https://hydra.cc/docs/development/overview/
• The testing page of the dev guide has some tips on how to use nox: https://hydra.cc/docs/development/testing/

[Pedrexus]

@Jasha10 thank you for the instructions, I was able to run the tests now.

However, the AWS tests are being skipped with the message AWS credentials not configured correctly. Skipping AWS tests. It seems the tests spin up an EC2 instance, but I think this would potentially incur costs.

I am unsure how to proceed in this case. Please advise if possible.

[Jasha10]

Some of the ray launcher tests are integration tests. They use ray to launch jobs on an AWS compute node.

In the past, facebook sponsored the cost of compute for running those tests, paying the AWS bills. Currently, those tests are broken and would require participation of someone inside of Facebook to set up new AWS credentials.

My advice would be to ignore the failing AWS tests and focus on the other ray launcher tests.

[Pedrexus]

I have opened the respective PRs for the Ax and Ray plugins.

The Ray Launcher plugin had few changes. The AWS Ray Launcher class has a deprecation warning added to it, since the tests are being skipped.

On the other hand, the Ax plugin tests were raising multiple warnings from internal Ax dependencies, and the only solution I found was to ignore the warnings for now. Please advise if there are problems with this approach.

Some changes in the Ax experiment API also required some breaking changes to the plugin.

On a side note, I have yet to test the updates on my own projects to ensure they are working as expected. I might update this message once I do this.