Support Google Batch's DWS Flex start feature

[puerta-amarilla]

New feature

Google has added Dynamic Workload Scheduling to Cloud Batch. It's now in public preview.

Use case

When Nextflow uses Google Cloud Batch as backend, the existing implementation of VM provisioning that expects the VMs to be available right away. When the requested machine types are not available in the requested zones, it results in zone_resource_exhaustion error. Using DWS Flex Start can solve this problem.

Suggested implementation

Expose relevant flag and properties to allow DWS as VM provision mode by Google Cloud Batch.

(Highlight the main building blocks of a possible implementation and/or related components)

[pditommaso]

Is there any tech docs or reference ?

[bentsherman]

I found this blog post

We just need to figure out how to enable the flex start flag for a batch job through the Java SDK

[puerta-amarilla]

[External] User Guide_ DWS in Google Batch.pdf

Thanks folks! Here is the user guide - note that it's still in preview and the guide will be updated in a few weeks.