Delegate wait_for_workers to cluster instances only when implemented

[consideRatio]

In #6700 the client started delegating a wait_for_workers call to a cluster instance if one was set, but that forced all cluster instances to implement such method. This PR makes it optional to implement such method.

This change is motivated by dask/dask-gateway#782 thanks to guidance from @TomAugspurger in a comment there. There a cluster instance was available, but wait_for_workers hasn't yet been implemented by the class defined by the dask/dask-gateway project.

• Tests added / passed
  I've for the moment opted to not try to develop a test for this small fix without further input.
• Passes pre-commit run --all-files

[GPUtester]

Can one of the admins verify this patch?

Admins can comment ok to test to allow this one PR to run or add to allowlist to allow all future PRs from the same author to run.

[consideRatio]

@jacobtomlinson as a reviewer of #6700 with relevant context I figure I'll give you a ping, no pressure though! Thanks for your work on this project!!!

[github-actions]

Unit Test Results

See test report for an extended history of previous test failures. This is useful for diagnosing flaky tests.

    27 files  ± 0      27 suites  ±0   10h 2m 41s ⏱️ + 18m 26s
 3 951 tests ± 0   3 841 ✅ + 3    109 💤 ±0  1 ❌  - 3 
49 696 runs  +18  47 406 ✅ +27  2 289 💤  - 6  1 ❌  - 3 

For more details on these failures, see this check.

Results for commit 1fc1897. ± Comparison against base commit 7562f9c.

♻️ This comment has been updated with latest results.

[jacobtomlinson]

This looks like a reasonable change.

It does lead me down the path of thinking that we should provide a cluster Protocol type that all cluster managers are expected to conform to though. Having implementations miss out methods and require us to check for them all over the place isn't great for maintainability.

But given this method requirement was added recently I think it's fine to add this now.

[fjetter]

It does lead me down the path of thinking that we should provide a cluster Protocol type

I would rather suggest a proper super type but that's a conversation for another ticket.

---

CI seems to be failing due to some version conflict. That's weird. @consideRatio would you mind rebasing/merging main to see if this disappears?

[consideRatio]

Thank you for spending time reviewing this @jacobtomlinson @fjetter!!!

---

CI seems to be failing due to some version conflict.

Yeah =/ I ignored the errors as they seemed unrelated.

That's weird. @consideRatio would you mind rebasing/merging main to see if this disappears?

Rebased and force-pushed on latest main branch!