Skip to content

Conversation

@ejseqera
Copy link
Contributor

Add support for installing the Google Cloud Ops Agent on Google Batch instances through a new configuration option google.batch.installOpsAgent. OpsAgent on GCP helps provide telemetry and monitoring info on VM instances like CPU, memory, disk I/O metrics.

Changes

  • Add installOpsAgent config option to BatchConfig
  • Update GoogleBatchTaskHandler to include installOpsAgent in the allocation policy API request
  • Add test coverage for the new configuration option
  • Update documentation with usage notes and compatibility requirements

Test with:

process {
    executor = 'google-batch'
}

google {
    project = 'google-project'  // Replace with proj
    location = '<your_region>' // Replace with region

    batch {
        bootDiskImage = 'projects/batch-custom-image/global/images/family/batch-debian-11-official' // compatible OS image
        installOpsAgent = true
        spot = true
    }
}
nextflow run nextflow-io/hello -c batch.config

Resultant jobs will use batch-debian as the OS and the Batch job logs will show installation of the OpsAgent.

Add a new configuration option google.batch.installOpsAgent that enables
the installation of the Ops Agent on Google Batch instances for enhanced
monitoring and logging capabilities.

Changes include:
- Add installOpsAgent config field to BatchConfig
- Update GoogleBatchTaskHandler to set installOpsAgent in allocation policy
- Add test coverage for the new configuration option
- Update documentation in config.md

Signed-off-by: ejseqera <[email protected]>
@ejseqera ejseqera requested a review from a team as a code owner November 24, 2025 21:14
@netlify
Copy link

netlify bot commented Nov 24, 2025

Deploy Preview for nextflow-docs-staging canceled.

Name Link
🔨 Latest commit 8e2c0d4
🔍 Latest deploy log https://app.netlify.com/projects/nextflow-docs-staging/deploys/6924cafd70b6670008768e7c

@ejseqera ejseqera force-pushed the add-support-gcp-ops-agent branch from 7b8b468 to 8e2c0d4 Compare November 24, 2025 21:15
@pditommaso pditommaso requested a review from jorgee November 28, 2025 09:22
instancePolicyOrTemplate.setPolicy( instancePolicy )
}

if( batchConfig.installOpsAgent )
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't it check if the boot disk image is compatible with the Ops Agent?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants