Merge pull request #33 from pepkit/master_looper_190

changes for looper 1.9.0

docs/looper/changelog.md

@@ -1,6 +1,16 @@
 # Changelog
 
 This project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html) and [Keep a Changelog](https://keepachangelog.com/en/1.0.0/) format.
+
+## [1.9.0] -- 2024-06-26
+
+### Added
+- user can now add cli modifiers to looper config instead of PEP project [#270](https://github.com/pepkit/looper/issues/270)
+- pipeline interfaces no longer must be nested under sample and project keys within looper config file [#465](https://github.com/pepkit/looper/issues/465)
+- var_templates can now be hierarchical [#334](https://github.com/pepkit/looper/issues/334)
+- looper can now gracefully halt spawned subprocesses when the user sends a keyboard interrupt [#37](https://github.com/pepkit/looper/issues/37)
+
+
 ## [1.8.1] -- 2024-06-05
 
 ### Fixed

docs/looper/defining-a-project.md

@@ -73,27 +73,3 @@ sample_modifiers:
         extend: "250"            # Default. For fixed-width peaks, extend this distance up- and down-stream.
         frip_ref_peaks: None     # Default. Use an external reference set of peaks instead of the peaks called from this run
 ```
-
-
-## 3. Customize looper
-
-You can also customize things further. Under the `looper` section, you can provide a `cli` keyword to specify any command line (CLI) options from within the project config file. The subsections within this section direct the arguments to the respective `looper` subcommands. So, to specify, e.g. sample submission limit for a `looper run` command use:
-
-```yaml
-looper:
-  output_dir: "/path/to/output_dir"
-  cli:
-    run:
-      limit: 2
-```
-
-or, to pass this argument to any subcommand:
-
-```yaml
-looper:
-  output_dir: "/path/to/output_dir"
-  all:
-    limit: 2
-```
-
-Keys in the `cli.<subcommand>` section *must* match the long argument parser option strings, so `command-extra`, `limit`, `dry-run` and so on. For more CLI options refer to the subcommands [usage](usage.md).

docs/looper/looper-config.md

@@ -34,3 +34,15 @@ one of supported ways: `namespace/name`, `namespace/name:tag`)
 
 To run pipeline, go to the directory of .looper.config and execute command in your terminal:
 `looper run --looper-config {looper_config_path}` or `looper runp --looper-config {looper_config_path}` (project-level pipeline).
+
+## Customize looper
+
+You can also customize things further. You can provide a `cli` keyword to specify any command line (CLI) options from within the looper config file. The subsections within this section direct the arguments to the respective `looper` subcommands. So, to specify, e.g. sample submission limit for a `looper run` command use:
+
+```yaml
+cli:
+    run:
+      limit: 2
+```
+
+Keys in the `cli.<subcommand>` section *must* match the long argument parser option strings, so `command-extra`, `limit`, `dry-run` and so on. For more CLI options refer to the subcommands [usage](usage.md).

docs/looper/pipeline-interface-specification.md

@@ -237,6 +237,15 @@ command_template: >
   python {looper.piface_dir}/count_lines.py {sample.file} {sample.sample_name}
 ```
 
+Note: Beginning with Looper 1.9.0, var_templates can be nested, e.g.
+```yaml
+var_templates:
+  refgenie_plugin:
+    config_path: "..."
+  custom_template_plugin:
+     config_path: "..."
+```
+
 #### pre_submit
 
 This section can consist of two subsections: `python_functions` and/or `command_templates`, which specify the pre-submission tasks to be run before the main pipeline command is submitted. Please refer to the [pre-submission hooks system](pre-submission-hooks.md) section for a detailed explanation of this feature and syntax.

docs/looper/pre-submission-hooks.md

@@ -40,7 +40,7 @@ Saves all sample metadata as a YAML file. The output file path can be customized
 pipeline_type: sample
 var_templates:
   main: "{looper.piface_dir}/pipelines/pipeline1.py"
-  sample_yaml_path: "{looper.output_dir}/custom_sample_yamls"
+  sample_yaml_path: "{looper.output_dir}/custom_sample_yamls/{sample.sample_name}.yaml"
 pre_submit:
   python_functions:
     - looper.write_sample_yaml