Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add process programmatic configuration #4375

Closed
wants to merge 5 commits into from
Closed
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 20 additions & 0 deletions docs/dsl2.md
Original file line number Diff line number Diff line change
Expand Up @@ -183,6 +183,26 @@ workflow {
Optional params for a process input/output are always prefixed with a comma, except for `stdout`. Because `stdout` does not have an associated name or value like other types, the first param should not be prefixed.
:::

### Process config
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ideally would like to merge #3793 first, then this would become part of the "Workflows" page which is more prominent.


:::{versionadded} 23.10.0
:::

A process can be configured with additional directives from a workflow by using the `config` attribute on a process. For example:

```groovy
include { foo ; foo as bar } from './module.nf'

workflow {
foo()

bar.config.publishDir 'results', mode: 'copy'
bar()
}
```

Process directives can be defined using the same syntax as in a process definition, and they are applied separately to different module aliases. In the above example, the `publishDir` directive is applied to process `bar` but not to process `foo`.

## Workflow

### Workflow definition
Expand Down
bentsherman marked this conversation as resolved.
Show resolved Hide resolved
Original file line number Diff line number Diff line change
Expand Up @@ -145,7 +145,10 @@ class ProcessDef extends BindableDef implements IterableDef, ChainableDef {

String getBaseName() { baseName }

ProcessConfig getProcessConfig() { processConfig }
ProcessConfig getConfig() {
if( processConfig == null ) initialize()
processConfig
}

ChannelOut getOut() {
if( output==null )
Expand All @@ -165,7 +168,8 @@ class ProcessDef extends BindableDef implements IterableDef, ChainableDef {
@Override
Object run(Object[] args) {
// initialise process config
initialize()
if( processConfig == null )
initialize()

// get params
final params = ChannelOut.spread(args)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -98,15 +98,15 @@ class ProcessDefTest extends Specification {
def copy = proc.clone()
copy.initialize()
then:
def cfg1 = copy.processConfig.createTaskConfig()
def cfg1 = copy.config.createTaskConfig()
cfg1.getCpus()==2 // taken from the generic config
cfg1.getMemory().giga == 3 // taken from the `foo` config

when:
copy = proc.cloneWithName('flow1:bar')
copy.initialize()
then:
def cfg2 = copy.processConfig.createTaskConfig()
def cfg2 = copy.config.createTaskConfig()
cfg2.getCpus()==4 // taken from the `bar` config
cfg2.getMemory().giga == 4 // taken from the `bar` config

Expand All @@ -115,7 +115,7 @@ class ProcessDefTest extends Specification {
copy = proc.cloneWithName('flow1:flow2:flow3:bar')
copy.initialize()
then:
def cfg3 = copy.processConfig.createTaskConfig()
def cfg3 = copy.config.createTaskConfig()
cfg3.getCpus()==4 // <-- taken from `withName: foo`
cfg3.getMemory().giga == 8 // <-- taken from `withName: 'flow1:flow2:flow3:bar'`
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -137,8 +137,8 @@ class ParamsDsl2Test extends Dsl2Spec {
def process = ScriptMeta.get(script).getProcess('alpha'); process.initialize()

then:
def inputs = process.processConfig.getInputs()
def outputs = process.processConfig.getOutputs()
def inputs = process.config.getInputs()
def outputs = process.config.getOutputs()
and:
inputs.size() == 1
inputs[0] instanceof StdInParam
Expand Down Expand Up @@ -178,8 +178,8 @@ class ParamsDsl2Test extends Dsl2Spec {
def process = ScriptMeta.get(script).getProcess('beta'); process.initialize()

then:
def inputs = process.processConfig.getInputs()
def outputs = process.processConfig.getOutputs()
def inputs = process.config.getInputs()
def outputs = process.config.getOutputs()
and:
inputs.size() == 1
and:
Expand Down
Loading