Add link_only
option to remote file sources
#19124
Labels
Milestone
link_only
option to remote file sources
#19124
Suggesting to introduce an optional
link_only
boolean parameter infile_sources_conf.yml
(default: false). When enabled, datasets are linked rather than copied during uploads. Although the upload/fetch mechanism already supportslink_only
uploads, this functionality is not currently configurable for file sources. By enabling this feature, users can avoid duplicating large datasets when the remote file source resides on a filesystem already mounted to their compute nodes. Potentially this can also be done using the file source type attribute.Ping @jmchilton for suggestions and comments. Thanks in advance.
The text was updated successfully, but these errors were encountered: