Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CLI v2 #3600

Draft
wants to merge 61 commits into
base: master
Choose a base branch
from
Draft

CLI v2 #3600

Show file tree
Hide file tree
Changes from 50 commits
Commits
Show all changes
61 commits
Select commit Hold shift + click to select a range
df66de1
wip [ci skip]
bentsherman Feb 2, 2023
d0ee2d8
Update tests
bentsherman Feb 3, 2023
bb1a9b9
Remove `kuberun` command from v2 CLI
bentsherman Feb 6, 2023
d4f5cf2
minor edits [ci skip]
bentsherman Feb 6, 2023
6c23771
Add proper params parsing to CLI v2 [ci fast]
bentsherman Feb 6, 2023
e31ff64
Update params parsing in CLI v2 to match v1 [ci fast]
bentsherman Feb 7, 2023
5f62ff2
Add fallback values to CLI options, fix some error messages [ci fast]
bentsherman Feb 8, 2023
a2063eb
Minor improvements
bentsherman Feb 13, 2023
49c49a8
Fix issue with multiple positional args, update tests [ci fast]
bentsherman Feb 16, 2023
0462ac2
Revert build files to use v1 CLI, add launch-v2.sh to test v2 CLI
bentsherman Feb 16, 2023
9b34186
Minor edits
bentsherman Feb 16, 2023
fb5675d
Fix bug with plugin command
bentsherman Feb 16, 2023
e50ed67
Merge branch 'master' into 3595-nf-cli
bentsherman Feb 16, 2023
b54a217
Fix several bugs and inconsistencies
bentsherman Feb 22, 2023
f48635b
Fix failing tests
bentsherman Feb 24, 2023
8a4ed78
Merge branch 'master' into 3595-nf-cli
pditommaso Feb 27, 2023
574d853
Merge branch '3595-nf-cli' of github.com:nextflow-io/nextflow into 35…
bentsherman Feb 24, 2023
78ef208
Fix several parsing errors
bentsherman Mar 21, 2023
6363f5c
Allow subcommands to be specified as option values
bentsherman Mar 21, 2023
b21a7f3
Merge branch 'master' into 3595-nf-cli
bentsherman Mar 21, 2023
b6dea75
Merge branch 'master' into 3595-nf-cli
pditommaso Mar 28, 2023
09e95e5
Merge branch 'master' into 3595-nf-cli
bentsherman Apr 21, 2023
93dc032
Merge branch 'master' into 3595-nf-cli
bentsherman Jul 5, 2023
4bbac21
Add `nf` binary
bentsherman Jul 20, 2023
eab9d60
Fix issue with ansi logger
bentsherman Jul 20, 2023
cdbd00c
Merge branch 'master' into 3595-nf-cli
bentsherman Jul 20, 2023
fcbddf5
Cleanup
bentsherman Jul 21, 2023
fc54c4e
Put CLI v1 code back into original command classes
bentsherman Jul 21, 2023
40d3010
Cleanup
bentsherman Jul 21, 2023
3eae972
Cleanup
bentsherman Jul 22, 2023
807b848
Cleanup
bentsherman Jul 22, 2023
78509d9
Update tests
bentsherman Jul 24, 2023
d6da4e1
Cleanup
bentsherman Jul 24, 2023
78cd98b
Merge branch 'master' into 3595-nf-cli
bentsherman Sep 15, 2023
5130c33
Fix inspect command
bentsherman Sep 15, 2023
3d0b1ae
Fix failing test
bentsherman Sep 18, 2023
79c6db1
Merge branch 'master' into 3595-nf-cli
bentsherman Oct 6, 2023
c1bf2a8
Rename `--deep` to `--depth` to match git
bentsherman Oct 6, 2023
07e20e6
Remove `-dsl1` and `-dsl2` options
bentsherman Oct 6, 2023
e5dd327
Remove `-without-*` options from CLI v2
bentsherman Oct 6, 2023
33ab7be
Update docs
bentsherman Oct 6, 2023
2c9eb7f
Add nf to install procedure
bentsherman Oct 6, 2023
956170e
Merge branch 'master' into 3595-nf-cli
bentsherman Nov 24, 2023
f38fe0b
Fix build errors
bentsherman Nov 24, 2023
04e046f
Fix failing tests
bentsherman Nov 24, 2023
d6e727f
Fix bugs
bentsherman Dec 13, 2023
2377bd5
Minor edits
bentsherman Dec 13, 2023
e578b3e
FIx inspect command
bentsherman Dec 13, 2023
d0481d7
Fix help text for run options
bentsherman Dec 13, 2023
d697749
Disable clustered short options
bentsherman Dec 14, 2023
cb62507
Make v1 option warning yellow
bentsherman Dec 14, 2023
a08f6a5
Merge branch 'master' into 3595-nf-cli
bentsherman Dec 14, 2023
e24ed5b
Merge branch 'master' into 3595-nf-cli
marcodelapierre Dec 14, 2023
dc1625d
Fix merge conflicts
bentsherman Dec 14, 2023
eb10b07
Fix map options in run command
bentsherman Dec 14, 2023
0e4db16
Remove ANSI shim from params helper
bentsherman Jan 9, 2024
ec41239
Add --background to run command
bentsherman Jan 9, 2024
0d6eb3f
Minor edits
bentsherman Jan 9, 2024
07f01c8
Mergre upstream changes
bentsherman Feb 12, 2024
074556c
cleanup
bentsherman Feb 12, 2024
abe7514
Remove positional args from v2 run, inspect
bentsherman Feb 23, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 19 additions & 4 deletions docs/cli.md
Original file line number Diff line number Diff line change
Expand Up @@ -1282,14 +1282,11 @@ The `run` command is used to execute a local pipeline script or remote pipeline
: Disable process execution with Docker.

`-without-podman`
: Disable process execution in a Podman container.
: Disable process execution with Podman.

`-without-spack`
: Disable process execution with Spack.

`-without-wave`
: Disable the use of Wave containers.

`-w, -work-dir` (`work`)
: Directory where intermediate result files are stored.

Expand Down Expand Up @@ -1512,3 +1509,21 @@ When a command line parameter includes one or more glob characters, i.e. wildcar
nextflow run <my script> --files "*.fasta"
```
:::

(cli-v2)=

## CLI v2

:::{versionadded} 23.10.0
:::

Nextflow now has an alternative command-line interface called `nf`, which more closely follows conventions for CLI options. In particular, long options for `nf` have two dashes, i.e. `-resume` is now `--resume`.

Pipeline parameters can be specified alongside CLI options as before, as long as they are named differently. If for some reason you have a conflicting param, you can use a double dash `--` to separate it from the CLI options.
bentsherman marked this conversation as resolved.
Show resolved Hide resolved

The `nf` command is a near drop-in replacement, by simply using double dashes for long options. The following minor changes were also introduced:

- The `plugins` command was removed (use `plugin` instead)
- The `secrets put` command was removed (use `secrets set` instead)
- The `-deep` option was renamed to `--depth` for `clone`, `pull`, and `run`
- The `-without-*` options were removed from the `run` command
19 changes: 19 additions & 0 deletions launch-v1.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
#!/bin/bash
#
# Copyright 2013-2023, Seqera Labs
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

cwd="$(realpath "$(dirname "$0")")"

MAIN_CLASS="nextflow.cli.Launcher" "${cwd}/launch.sh" "$@"
19 changes: 19 additions & 0 deletions launch-v2.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
#!/bin/bash
#
# Copyright 2013-2023, Seqera Labs
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

cwd="$(realpath "$(dirname "$0")")"

MAIN_CLASS="nextflow.cli.v2.Launcher" "${cwd}/launch.sh" "$@"
1 change: 1 addition & 0 deletions modules/nextflow/build.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,7 @@ dependencies {
api 'jline:jline:2.9'
api 'org.pf4j:pf4j:3.10.0'
api 'dev.failsafe:failsafe:3.1.0'
api 'info.picocli:picocli-groovy:4.7.1'

testImplementation 'org.subethamail:subethasmtp:3.1.7'

Expand Down
5 changes: 1 addition & 4 deletions modules/nextflow/src/main/groovy/nextflow/Session.groovy
Original file line number Diff line number Diff line change
Expand Up @@ -432,10 +432,7 @@ class Session implements ISession {
}

/**
* Given the `run` command line options creates the required {@link TraceObserver}s
*
* @param runOpts The {@code CmdRun} object holding the run command options
* @return A list of {@link TraceObserver} objects or an empty list
* Creates the required {@link TraceObserver}s for the workflow run.
*/
@PackageScope
List<TraceObserver> createObservers() {
Expand Down
151 changes: 86 additions & 65 deletions modules/nextflow/src/main/groovy/nextflow/cli/CliOptions.groovy
Original file line number Diff line number Diff line change
Expand Up @@ -28,79 +28,100 @@ import org.fusesource.jansi.Ansi
* @author Paolo Di Tommaso <[email protected]>
*/
@Slf4j
class CliOptions {
abstract class CliOptions {

abstract Boolean getAnsiLogCli()
abstract boolean isBackground()
abstract List<String> getConfig()
abstract List<String> getDebug()
abstract boolean getIgnoreConfigIncludes()
abstract String getLogFile()
abstract boolean isQuiet()
abstract boolean getRemoteDebug()
abstract String getSyslog()
abstract List<String> getTrace()
abstract List<String> getUserConfig()
abstract boolean getVersion()
abstract void setAnsiLog(boolean value)
abstract void setBackground(boolean value)

static class V1 extends CliOptions {

/**
* The packages to debug
*/
@Parameter(hidden = true, names='-debug')
List<String> debug

@Parameter(names=['-log'], description = 'Set nextflow log file path')
String logFile

@Parameter(names=['-c','-config'], description = 'Add the specified file to configuration set')
List<String> userConfig

@Parameter(names=['-config-ignore-includes'], description = 'Disable the parsing of config includes')
boolean ignoreConfigIncludes

@Parameter(names=['-C'], description = 'Use the specified configuration file(s) overriding any defaults')
List<String> config

/**
* the packages to trace
*/
@Parameter(names='-trace', description = 'Enable trace level logging for the specified package name - multiple packages can be provided separating them with a comma e.g. \'-trace nextflow,io.seqera\'')
List<String> trace

/**
* Enable syslog appender
*/
@Parameter(names = ['-syslog'], description = 'Send logs to syslog server (eg. localhost:514)' )
String syslog

/**
* Print out the version number and exit
*/
@Parameter(names = ['-v','-version'], description = 'Print the program version')
boolean version

/**
* Print out the 'help' and exit
*/
@Parameter(names = ['-h'], description = 'Print this help', help = true)
boolean help

@Parameter(names = ['-q','-quiet'], description = 'Do not print information messages' )
boolean quiet

@Parameter(names = ['-bg'], description = 'Execute nextflow in background', arity = 0)
boolean background

@DynamicParameter(names = ['-D'], description = 'Set JVM properties' )
Map<String,String> jvmOpts = [:]

@Parameter(names = ['-self-update'], description = 'Update nextflow to the latest version', arity = 0, hidden = true)
boolean selfUpdate

@Parameter(names=['-remote-debug'], description = "Enable JVM interactive remote debugging (experimental)")
boolean remoteDebug

Boolean ansiLogCli

void setAnsiLog(boolean value) { ansiLogCli = value }

/**
* The packages to debug
*/
@Parameter(hidden = true, names='-debug')
List<String> debug

@Parameter(names=['-log'], description = 'Set nextflow log file path')
String logFile

@Parameter(names=['-c','-config'], description = 'Add the specified file to configuration set')
List<String> userConfig

@Parameter(names=['-config-ignore-includes'], description = 'Disable the parsing of config includes')
boolean ignoreConfigIncludes

@Parameter(names=['-C'], description = 'Use the specified configuration file(s) overriding any defaults')
List<String> config

/**
* the packages to trace
*/
@Parameter(names='-trace', description = 'Enable trace level logging for the specified package name - multiple packages can be provided separating them with a comma e.g. \'-trace nextflow,io.seqera\'')
List<String> trace

/**
* Enable syslog appender
*/
@Parameter(names = ['-syslog'], description = 'Send logs to syslog server (eg. localhost:514)' )
String syslog

/**
* Print out the version number and exit
*/
@Parameter(names = ['-v','-version'], description = 'Print the program version')
boolean version

/**
* Print out the 'help' and exit
*/
@Parameter(names = ['-h'], description = 'Print this help', help = true)
boolean help

@Parameter(names = ['-q','-quiet'], description = 'Do not print information messages' )
boolean quiet

@Parameter(names = ['-bg'], description = 'Execute nextflow in background', arity = 0)
boolean background

@DynamicParameter(names = ['-D'], description = 'Set JVM properties' )
Map<String,String> jvmOpts = [:]

@Parameter(names = ['-self-update'], description = 'Update nextflow to the latest version', arity = 0, hidden = true)
boolean selfUpdate

@Parameter(names=['-remote-debug'], description = "Enable JVM interactive remote debugging (experimental)")
boolean remoteDebug

Boolean ansiLog
}

boolean getAnsiLog() {
if( ansiLog && quiet )
if( ansiLogCli && quiet )
throw new AbortOperationException("Command line options `quiet` and `ansi-log` cannot be used together")

if( ansiLog != null )
return ansiLog
if( ansiLogCli != null )
return ansiLogCli

if( background )
return ansiLog = false
return ansiLogCli = false

if( quiet )
return ansiLog = false
return ansiLogCli = false

final env = System.getenv('NXF_ANSI_LOG')
if( env ) try {
Expand All @@ -113,7 +134,7 @@ class CliOptions {
}

boolean hasAnsiLogFlag() {
ansiLog==true || System.getenv('NXF_ANSI_LOG')=='true'
ansiLogCli==true || System.getenv('NXF_ANSI_LOG')=='true'
}

}
2 changes: 2 additions & 0 deletions modules/nextflow/src/main/groovy/nextflow/cli/CmdBase.groovy
Original file line number Diff line number Diff line change
Expand Up @@ -17,12 +17,14 @@
package nextflow.cli

import com.beust.jcommander.Parameter
import groovy.transform.CompileStatic

/**
* Implement command shared methods
*
* @author Paolo Di Tommaso <[email protected]>
*/
@CompileStatic
abstract class CmdBase implements Runnable {

private Launcher launcher
Expand Down
Loading
Loading