Code

Cluster wait#6700

Code

Merged

jacobtomlinson merged 44 commits intodask:maindask/distributed:mainfrom idorrington92:cluster_waitidorrington92/distributed:cluster_waitCopy head branch name to clipboardApr 20, 2023

+76-1

Commits

Commits on Jul 9, 2022

• Moving wait_for_worker logic to cluster, and having client call that if it can

  idorrington92

  committed
  4f850e8

• Merge branch 'main' of https://github.com/dask/distributed into cluster_wait

  idorrington92

  committed
  ec66806

• Adding test for cluster.wait_for_workers

  idorrington92

  committed
  4ee2739

Commits on Jul 20, 2022

• use try and except to catch case where cluster is none or wait_for_workers is not implemented

  idorrington92

  committed
  f9a840b

• Merge branch 'main' into cluster_wait

  idorrington92

  authored
  5439674

• linting

  idorrington92

  committed
  ca778c7

Commits on Jul 23, 2022

• Merge remote-tracking branch 'upstream/main' into cluster_wait

  idorrington92

  committed
  8f532e9

• This test has been removed on main branch, but for some reason git merge didn't remove it from mine

  idorrington92

  committed
  78985cc

Commits on Aug 6, 2022

• Cluster has to use scheduler_info attribute instead of scheduler.identity

  idorrington92

  committed
  aad5d56

• Merge remote-tracking branch 'upstream/main' into cluster_wait

  idorrington92

  committed
  b1ad4d5

• lint

  idorrington92

  committed
  ef96549

• reverting

  idorrington92

  committed
  e682058

Commits on Aug 8, 2022

• Merge remote-tracking branch 'upstream/main' into cluster_wait

  idorrington92

  committed
  cdf2c0b

• need to use cluster.scale when client.wait_for_workers is called while using a cluster

  idorrington92

  committed
  45cf486

• need to use scheduler_info. Also, using cluster.scale to emulate behaviour of client.wait_for_workers

  idorrington92

  committed
  088e696

• using scheduler_info and dont need to call scale anymore

  idorrington92

  committed
  4861151

• lint

  idorrington92

  committed
  1d9705c

Commits on Aug 9, 2022

• Merge remote-tracking branch 'upstream/main' into cluster_wait

  idorrington92

  committed
  97f4aa4

• adding gen_test decorator

  idorrington92

  committed
  c3907b7

• Don't think we need to scale at start of wait_for_workers

  idorrington92

  committed
  ac1dbc5

Commits on Aug 14, 2022

• self.scheduler_info does not update worker status from init to running, so need to request status again

  idorrington92

  committed
  44ccdd3

• Use Status

  idorrington92

  committed
  b78772e

• Merge remote-tracking branch 'upstream/main' into cluster_wait

  idorrington92

  committed
  05eb270

• Scale was fixing the nworkers test because it forced the worker status to update. Now that worker status is checked we don't need this (and shouldn't have really included it anyway)

  idorrington92

  committed
  03d9d18

Commits on Aug 15, 2022

• Refactoring

  idorrington92

  committed
  dfa0ea0

• Fixing type information

  idorrington92

  committed
  3d8ddca

Commits on Aug 20, 2022

• Experimenting with creating new comm

  idorrington92

  committed
  625960d

• Create separate comm in _start and use that to update scheduler_info

  idorrington92

  committed
  c946540

• Close new comm

  idorrington92

  committed
  7af5fce

• initialise scheduler_info_comm

  idorrington92

  committed
  2dd7932

Commits on Sep 7, 2022

• Merge remote-tracking branch 'upstream/main' into cluster_wait

  idorrington92

  committed
  872e62a

• Don't allow n_workers to be zero for cluster wait_for_workers

  idorrington92

  committed
  05a54c7

• Adding return type

  idorrington92

  committed
  632cfa9

Commits on Jan 19, 2023

• Merge branch 'main' of https://github.com/dask/distributed into cluster_wait

  idorrington92

  committed
  5789bfd

• Change try-catch to be an explicit if-else

  idorrington92

  committed
  000febc

• Check explicitly for cluster is none, as I think it's clearer

  idorrington92

  committed
  3d573e5

• linting

  idorrington92

  committed
  2b1713a

Commits on Jan 20, 2023

• use scheduler_comm instead of opening new comm

  idorrington92

  committed
  8877a58

• remove update_scheduler_info method

  idorrington92

  committed
  005dd69

Commits on Apr 10, 2023

• Merge branch 'main' of https://github.com/dask/distributed into cluster_wait

  idorrington92

  committed
  db9dc71

• pre-commit changes

  idorrington92

  committed
  494d3fc

Commits on Apr 20, 2023

• Merge branch 'main' of https://github.com/dask/distributed into cluster_wait

  idorrington92

  committed
  21277ce

• Reduce number of works to see if it fixes github tests

  idorrington92

  committed
  d277ea6

• Changing test to make it work in python 3.8

  idorrington92

  committed
  c95aafd