Skip to content

Conversation

@michal-shalev
Copy link
Contributor

@michal-shalev michal-shalev commented Oct 25, 2025

What?

Add a new build_plugins meson option to selectively build only specific plugins instead of all available plugins.

Why?

Building all plugins increases build time and requires all plugin dependencies to be present. When developing or deploying with only specific plugins (e.g., just UCX), building unnecessary plugins wastes time and resources. This option allows users to:

  • Reduce build time significantly
  • Avoid unnecessary dependencies
  • Speed up development iteration when working on a single plugin

How?

  • Added build_plugins meson option that accepts a comma-separated list of plugin names (e.g., UCX,GPUNETIO)
  • Default value is empty string, which builds all available plugins (backward compatible)
  • Created a global should_build_plugin dictionary that is computed once in the main meson.build and used throughout
  • Applied plugin filtering to:
    • src/plugins/meson.build - only builds requested plugin libraries
    • test/unit/plugins/meson.build - only builds unit tests for enabled plugins
    • test/gtest/plugins/meson.build - skips if required plugins aren't enabled

Usage examples:

# Build only GDS plugin
meson setup build -Dbuild_plugins=GDS

# Build GDS and HF3FS plugins
meson setup build -Dbuild_plugins=GDS,HF3FS

# Build all plugins (default)
meson setup build

@github-actions
Copy link

👋 Hi michal-shalev! Thank you for contributing to ai-dynamo/nixl.

Your PR reviewers will review your contribution then trigger the CI to test your changes.

🚀

@iyastreb
Copy link
Contributor

Seems to be useful
Maybe it can be just a regex implementing both allow and deny logic, similar to GTEST_FILTER?
E.g.
-Dbuild_plugins=GDS // allow, only GDS is used
-Dbuild_plugins=^GDS //deny, all except GDS are used

option('cudapath_lib', type: 'string', value: '', description: 'Library path for CUDA')
option('cudapath_stub', type: 'string', value: '', description: 'Extra Stub path for CUDA')
option('static_plugins', type: 'string', value: '', description: 'Plugins to be built-in, comma-separated')
option('build_plugins', type: 'string', value: '', description: 'List of plugins to build, comma-separated. Default (empty) builds all available plugins')
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we name this plugins?

build_plugins_opt = get_option('build_plugins')
enabled_plugins = build_plugins_opt == '' ? [] : build_plugins_opt.split(',')

should_build_plugin = {}
Copy link
Contributor

@ovidiusm ovidiusm Nov 4, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should_build_plugin should be called enabled_plugins IMO

enabled_plugins = build_plugins_opt == '' ? [] : build_plugins_opt.split(',')

should_build_plugin = {}
foreach plugin : ['UCX', 'UCX_MO', 'LIBFABRIC', 'POSIX', 'OBJ', 'GDS', 'GDS_MT', 'MOONCAKE', 'Mooncake', 'HF3FS', 'GUSLI', 'GPUNETIO']
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's make a list all_plugins and do case insensitive comparison to avoid duplicates

@aranadive
Copy link
Contributor

@michal-shalev thanks for adding this. I think we can take a little bit better approach:
We should auto-generate a list of plugins during build by parsing the src/plugins directory for all the plugins. Then, create a map from plugin to plugin_directory. Then just iterate over this list in src/plugins/meson.build, ignoring ones in the user-provided mask. Just make sure all the dependencies for a plugin should be moved to the plugins meson.build and not handled in the src/plugins/meson.build.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants