First release of quantmsdiann including new features previously no supported

.claude/actions_plans.md

@@ -0,0 +1,172 @@
+# quantmsdiann — Action Plans & Roadmap
+
+> Standalone DIA-NN (DIA) pipeline refactored from quantms.
+> Last updated: 2026-03-18
+
+---
+
+## Phase 1 — Cleanup & Correctness (COMPLETED)
+
+**Goal**: Remove legacy quantms artifacts that don't belong in a DIA-only pipeline.
+
+### 1.1 Remove non-DIA test profiles from CI
+
+- **Files**: `.github/workflows/ci.yml`, `.github/workflows/extended_ci.yml`
+- **Action**: Removed `test_lfq`, `test_tmt`, `test_localize`, `test_dda_id_*`, `test_tmt_corr` from CI matrices. Updated repository reference from `bigbio/quantms` to `bigbio/quantmsdiann`.
+- **Status**: [x] DONE
+
+### 1.2 Remove MSstats analysis step
+
+- **Files**: `workflows/dia.nf`, `workflows/quantmsdiann.nf`, `modules/local/msstats/`, `bin/msstats_plfq.R`, `nextflow.config`, `conf/modules/shared.config`
+- **Action**: Removed MSSTATS_LFQ module, R script, and all MSstats analysis parameters. Kept DIANN_MSSTATS conversion step (generates MSstats-compatible CSV without running MSstats analysis). Removed MSstats general and LFQ options from nextflow.config.
+- **Status**: [x] DONE
+
+### 1.3 Clean parameter schema and config
+
+- **Files**: `nextflow.config`, `nextflow_schema.json`
+- **Action**: Removed `id_only` parameter and `statistical_post_processing` schema section. Updated manifest name to `bigbio/quantmsdiann`. Removed dead publishDir rules from shared.config. Created missing nf-core lint files (logos, docs/README.md).
+- **Status**: [x] DONE
+
+### 1.4 Update documentation
+
+- **Files**: `README.md`, `docs/usage.md`, `docs/output.md`, `AGENTS.md`
+- **Action**: Rewrote README with DIA-NN workflow SVG diagram. Rewrote AGENTS.md for DIA-only. Cleaned docs. Removed 10 legacy quantms images from docs/images/, created `quantmsdiann_workflow.svg`.
+- **Status**: [x] DONE
+
+### 1.5 Remove unused modules and files
+
+- **Action**: Removed `modules/local/msstats/`, `bin/msstats_plfq.R`, `.devcontainer/`, `diann_private.yml`. Cleaned all legacy quantms images.
+- **Status**: [x] DONE
+
+---
+
+## Phase 1b — Version-Aware Testing Strategy (COMPLETED)
+
+**Goal**: Map each DIA-NN feature to the minimum version that supports it, with conditional CI.
+
+### DIA-NN Version → Feature Matrix
+
+| Version | Key Features | Container |
+| ------- | ------------ | --------- |
+| **1.8.1** (default) | Core DIA-NN workflow, library-free, .quant caching | `biocontainers/diann:v1.8.1_cv1` (public) |
+| **1.9.2** | QuantUMS quantification, Parquet libraries, redesigned NN | `ghcr.io/bigbio/diann:1.9.2` (private, needs build) |
+| **2.0** | Parquet output, proteoform confidence, decoy reporting | `ghcr.io/bigbio/diann:2.1.0` (private) |
+| **2.1.0** | Native .raw on Linux, latest improvements | `ghcr.io/bigbio/diann:2.1.0` (private) |
+| **2.2.0** | Latest release | `ghcr.io/bigbio/diann:latest` (private, needs build) |
+
+### Test Profiles Created
+
+| Profile | Feature Tested | Min DIA-NN | Container |
+| ------- | -------------- | ---------- | --------- |
+| `test_dia` | Core workflow | 1.8.1 | biocontainers (public) |
+| `test_dia_dotd` | Bruker .d format | 1.8.1 | biocontainers (public) |
+| `test_dia_quantums` | QuantUMS quantification | 1.9.2 | ghcr.io/bigbio/diann:2.1.0 |
+| `test_dia_parquet` | Parquet output + decoys | 2.0 | ghcr.io/bigbio/diann:2.1.0 |
+| `test_latest_dia` | Latest version validation | latest | ghcr.io/bigbio/diann:2.1.0 |
+| `test_full_dia` | Full-size dataset | 1.8.1 | biocontainers (public) |
+
+### CI/CD Structure
+
+**ci.yml** (every PR, fast):
+- `test_dia`, `test_dia_dotd` — public containers, no auth needed
+
+**extended_ci.yml** (5 jobs):
+1. **test-default** — always runs `test_dia`, `test_dia_dotd` (Docker, 2 NXF versions)
+2. **detect-changes** — uses `dorny/paths-filter` to detect which feature files changed
+3. **test-features** — always runs on push to dev/master, releases, manual dispatch: `test_latest_dia`, `test_dia_quantums`, `test_dia_parquet`
+4. **test-features-pr** — runs on PRs only when relevant files change (conditional per-feature)
+5. **test-singularity** — default tests only, after Docker passes
+
+### Container Build Needed
+
+- **DIA-NN 1.9.2**: Dockerfile created at `quantms-containers/diann-1.9.2/Dockerfile`. Needs to be built and pushed to `ghcr.io/bigbio/diann:1.9.2`.
+- **DIA-NN 2.2.0**: Need Dockerfile when ready to test latest.
+
+### Status: [x] DONE
+
+---
+
+## Phase 2 — DIA-NN 2.x Full Support
+
+**Goal**: Make DIA-NN 2.1.0 a first-class citizen, leverage Parquet-native output.
+
+### 2.1 Promote DIA-NN 2.1.0 to default
+
+- **Files**: All `modules/local/diann/*/main.nf`, `nextflow.config`
+- **Action**: Update default container from `diann:v1.8.1_cv1` to `diann:2.1.0`. Keep 1.8.1 available as a fallback profile.
+- **Dependencies**: Verify all 7 DIA-NN modules work with 2.1.0 CLI changes.
+- **Effort**: Medium
+- **Status**: [ ] TODO
+
+### 2.2 Parquet-native pipeline path
+
+- **Files**: `modules/local/diann/final_quantification/main.nf`, `modules/local/diann/diann_msstats/main.nf`
+- **Action**: Ensure DIANN_MSSTATS handles Parquet input end-to-end. Validated by `test_dia_parquet` CI profile.
+- **Effort**: Medium
+- **Status**: [ ] TODO
+
+### 2.3 DIA-NN version parameter
+
+- **Files**: `nextflow.config`, `nextflow_schema.json`
+- **Action**: Add a `diann_version` parameter that switches container images without needing separate profiles.
+- **Effort**: Medium
+- **Status**: [ ] TODO
+
+---
+
+## Phase 3 — Performance & Scalability
+
+**Goal**: Optimize resource usage and execution for large-scale DIA studies.
+
+### 3.1 Smarter pre-analysis file selection
+
+- **Files**: `workflows/dia.nf`
+- **Action**: Implement stratified selection (by condition/batch from SDRF) instead of purely random.
+- **Effort**: Medium
+- **Status**: [ ] TODO
+
+### 3.2 Resource profiling and tuning
+
+- **Files**: `conf/base.config`
+- **Action**: Profile resource usage across dataset sizes. Adjust labels. Consider dynamic allocation.
+- **Effort**: Medium
+- **Status**: [ ] TODO
+
+### 3.3 GPU support profile
+
+- **Files**: `conf/gpu.config` (new), `modules/local/diann/*/main.nf`
+- **Action**: Create `gpu` profile with NVIDIA runtime, `accelerator` directives, GPU container.
+- **Effort**: Medium-Large
+- **Status**: [ ] TODO
+
+### 3.4 Improved caching strategy
+
+- **Files**: DIA-NN modules
+- **Action**: Evaluate `storeDir` for expensive steps (library generation, preliminary analysis).
+- **Effort**: Small
+- **Status**: [ ] TODO
+
+---
+
+## Priority Summary
+
+| Priority        | Phase    | Items                       | Timeline             |
+| --------------- | -------- | --------------------------- | -------------------- |
+| **Done**        | Phase 1  | 1.1-1.5 (cleanup)           | Completed 2026-03-17 |
+| **Done**        | Phase 1b | Version-aware testing        | Completed 2026-03-18 |
+| **Short-term**  | Phase 2  | 2.1-2.3 (DIA-NN 2.x)        | 1-2 weeks            |
+| **Medium-term** | Phase 3  | 3.1-3.4 (performance)       | 2-4 weeks            |
+
+---
+
+## Decision Log
+
+| Date       | Decision                                         | Rationale                                                                    |
+| ---------- | ------------------------------------------------ | ---------------------------------------------------------------------------- |
+| 2026-03-17 | Created roadmap from quantms dev comparison      | Align refactoring priorities                                                 |
+| 2026-03-17 | Completed Phase 1 cleanup                        | Remove all non-DIA artifacts                                                 |
+| 2026-03-17 | Keep DIANN_MSSTATS, remove MSSTATS_LFQ           | Generate MSstats-compatible CSV but don't run MSstats analysis in-pipeline   |
+| 2026-03-17 | Removed Phases 4-6 (quantification, QC, interop) | pmultiqc already covers QC; downstream analysis/interop out of scope for now |
+| 2026-03-18 | Version-aware testing with conditional CI         | Each feature maps to min DIA-NN version; PRs only run affected feature tests |
+| 2026-03-18 | DIA-NN containers are private (license)           | Academic-only license; GHCR_USERNAME + GHCR_TOKEN secrets required           |
+| 2026-03-18 | Created DIA-NN 1.9.2 Dockerfile                  | Needed for QuantUMS feature testing at minimum supported version             |

.gitattributes

@@ -0,0 +1,4 @@
+*.config linguist-language=nextflow
+*.nf.test linguist-language=nextflow
+modules/nf-core/** linguist-generated
+subworkflows/nf-core/** linguist-generated

.github/.dockstore.yml

@@ -0,0 +1,6 @@
+# Dockstore config version, not pipeline version
+version: 1.2
+workflows:
+  - subclass: NFL
+    primaryDescriptorPath: /nextflow.config
+    publish: True

.github/CONTRIBUTING.md

@@ -0,0 +1,125 @@
+# `bigbio/quantmsdiann`: Contributing Guidelines
+
+Hi there!
+Many thanks for taking an interest in improving bigbio/quantmsdiann.
+
+We try to manage the required tasks for bigbio/quantmsdiann using GitHub issues, you probably came to this page when creating one.
+Please use the pre-filled template to save time.
+
+However, don't be put off by this template - other more general issues and suggestions are welcome!
+Contributions to the code are even more welcome ;)
+
+> [!NOTE]
+> If you need help using or modifying bigbio/quantmsdiann or bigbio/quantms then the best place to ask is on the nf-core Slack [#quantms](https://nfcore.slack.com/channels/quantms) channel ([join our Slack here](https://nf-co.re/join/slack)).
+
+## Contribution workflow
+
+If you'd like to write some code for bigbio/quantmsdiann, the standard workflow is as follows:
+
+1. Check that there isn't already an issue about your idea in the [bigbio/quantmsdiann issues](https://github.com/bigbio/quantmsdiann/issues) to avoid duplicating work. If there isn't one already, please create one so that others know you're working on this
+2. [Fork](https://help.github.com/en/github/getting-started-with-github/fork-a-repo) the [bigbio/quantmsdiann repository](https://github.com/bigbio/quantmsdiann) to your GitHub account
+3. Make the necessary changes / additions within your forked repository following [Pipeline conventions](#pipeline-contribution-conventions)
+4. Use `nf-core pipelines schema build` and add any new parameters to the pipeline JSON schema (requires [nf-core tools](https://github.com/nf-core/tools) >= 1.10).
+5. Submit a Pull Request against the `dev` branch and wait for the code to be reviewed and merged
+
+If you're not used to this workflow with git, you can start with some [docs from GitHub](https://help.github.com/en/github/collaborating-with-issues-and-pull-requests) or even their [excellent `git` resources](https://try.github.io/).
+
+## Tests
+
+You have the option to test your changes locally by running the pipeline. For receiving warnings about process selectors and other `debug` information, it is recommended to use the debug profile. Execute all the tests with the following command:
+
+```bash
+nf-test test --profile debug,test,docker --verbose
+```
+
+When you create a pull request with changes, [GitHub Actions](https://github.com/features/actions) will run automatic tests.
+Typically, pull-requests are only fully reviewed when these tests are passing, though of course we can help out before then.
+
+There are typically two types of tests that run:
+
+### Lint tests
+
+`nf-core` has a [set of guidelines](https://nf-co.re/developers/guidelines) which all pipelines must adhere to.
+To enforce these and ensure that all pipelines stay in sync, we have developed a helper tool which runs checks on the pipeline code. This is in the [nf-core/tools repository](https://github.com/nf-core/tools) and once installed can be run locally with the `nf-core pipelines lint <pipeline-directory>` command.
+
+If any failures or warnings are encountered, please follow the listed URL for more documentation.
+
+### Pipeline tests
+
+Each `nf-core` pipeline should be set up with a minimal set of test-data.
+`GitHub Actions` then runs the pipeline on this data to ensure that it exits successfully.
+If there are any failures then the automated tests fail.
+These tests are run both with the latest available version of `Nextflow` and also the minimum required version that is stated in the pipeline code.
+
+## Patch
+
+:warning: Only in the unlikely and regretful event of a release happening with a bug.
+
+- On your own fork, make a new branch `patch` based on `upstream/main`.
+- Fix the bug, and bump version (X.Y.Z+1).
+- Open a pull-request from `patch` to `main` with the changes.
+
+## Getting help
+
+For further information/help, please consult the [bigbio/quantmsdiann documentation](https://docs.quantms.org/en/latest/usage.html) and don't hesitate to get in touch on the nf-core Slack [#quantms](https://nfcore.slack.com/channels/quantms) channel ([join our Slack here](https://nf-co.re/join/slack)).
+
+## Pipeline contribution conventions
+
+To make the `bigbio/quantmsdiann` code and processing logic more understandable for new contributors and to ensure quality, we semi-standardise the way the code and other contributions are written.
+
+### Adding a new step
+
+If you wish to contribute a new step, please use the following coding standards:
+
+1. Define the corresponding input channel into your new process from the expected previous process channel.
+2. Write the process block (see below).
+3. Define the output channel if needed (see below).
+4. Add any new parameters to `nextflow.config` with a default (see below).
+5. Add any new parameters to `nextflow_schema.json` with help text (via the `nf-core pipelines schema build` tool).
+6. Add sanity checks and validation for all relevant parameters.
+7. Perform local tests to validate that the new code works as expected.
+8. If applicable, add a new test in the `tests` directory.
+9. Update MultiQC config `assets/multiqc_config.yml` so relevant suffixes, file name clean up and module plots are in the appropriate order. If applicable, add a [MultiQC](https://multiqc.info/) module.
+10. Add a description of the output files and if relevant any appropriate images from the MultiQC report to `docs/output.md`.
+
+### Default values
+
+Parameters should be initialised / defined with default values within the `params` scope in `nextflow.config`.
+
+Once there, use `nf-core pipelines schema build` to add to `nextflow_schema.json`.
+
+### Default processes resource requirements
+
+Sensible defaults for process resource requirements (CPUs / memory / time) for a process should be defined in `conf/base.config`. These should generally be specified generic with `withLabel:` selectors so they can be shared across multiple processes/steps of the pipeline. A nf-core standard set of labels that should be followed where possible can be seen in the [nf-core pipeline template](https://github.com/nf-core/tools/blob/main/nf_core/pipeline-template/conf/base.config), which has the default process as a single core-process, and then different levels of multi-core configurations for increasingly large memory requirements defined with standardised labels.
+
+The process resources can be passed on to the tool dynamically within the process with the `${task.cpus}` and `${task.memory}` variables in the `script:` block.
+
+### Naming schemes
+
+Please use the following naming schemes, to make it easy to understand what is going where.
+
+- initial process channel: `ch_output_from_<process>`
+- intermediate and terminal channels: `ch_<previousprocess>_for_<nextprocess>`
+
+### Nextflow version bumping
+
+If you are using a new feature from core Nextflow, you may bump the minimum required version of nextflow in the pipeline with: `nf-core pipelines bump-version --nextflow . [min-nf-version]`
+
+### Images and figures
+
+For overview images and other documents we follow the nf-core [style guidelines and examples](https://nf-co.re/developers/design_guidelines).
+
+## GitHub Codespaces
+
+This repo includes a devcontainer configuration which will create a GitHub Codespaces for Nextflow development! This is an online developer environment that runs in your browser, complete with VSCode and a terminal.
+
+To get started:
+
+- Open the repo in [Codespaces](https://github.com/bigbio/quantmsdiann/codespaces)
+- Tools installed
+  - nf-core
+  - Nextflow
+
+Devcontainer specs:
+
+- [DevContainer config](.devcontainer/devcontainer.json)

.github/ISSUE_TEMPLATE/bug_report.yml

@@ -0,0 +1,42 @@
+name: Bug report
+description: Report something that is broken or incorrect
+labels: bug
+body:
+  - type: textarea
+    id: description
+    attributes:
+      label: Description of the bug
+      description: A clear and concise description of what the bug is.
+    validations:
+      required: true
+
+  - type: textarea
+    id: command_used
+    attributes:
+      label: Command used and terminal output
+      description: Steps to reproduce the behaviour. Please paste the command you used to launch the pipeline and the output from your terminal.
+      render: console
+      placeholder: |
+        $ nextflow run ...
+
+        Some output where something broke
+
+  - type: textarea
+    id: files
+    attributes:
+      label: Relevant files
+      description: |
+        Please drag and drop the relevant files here. Create a `.zip` archive if the extension is not allowed.
+        Your verbose log file `.nextflow.log` is often useful _(this is a hidden file in the directory where you launched the pipeline)_ as well as custom Nextflow configuration files.
+
+  - type: textarea
+    id: system
+    attributes:
+      label: System information
+      description: |
+        * Nextflow version _(eg. 23.04.0)_
+        * Hardware _(eg. HPC, Desktop, Cloud)_
+        * Executor _(eg. slurm, local, awsbatch)_
+        * Container engine: _(e.g. Docker, Singularity, Conda, Podman, Shifter, Charliecloud, or Apptainer)_
+        * OS _(eg. CentOS Linux, macOS, Linux Mint)_
+        * Version of bigbio/quantmsdiann _(eg. 1.1, 1.5, 1.8.2)_

.github/ISSUE_TEMPLATE/config.yml

@@ -0,0 +1,7 @@
+contact_links:
+  - name: Join nf-core
+    url: https://nf-co.re/join
+    about: Please join the nf-core community here
+  - name: "Slack #quantms channel"
+    url: https://nfcore.slack.com/channels/quantms
+    about: Discussion about the bigbio/quantmsdiann pipeline

.github/ISSUE_TEMPLATE/feature_request.yml

@@ -0,0 +1,11 @@
+name: Feature request
+description: Suggest an idea for the bigbio/quantmsdiann pipeline
+labels: enhancement
+body:
+  - type: textarea
+    id: description
+    attributes:
+      label: Description of feature
+      description: Please describe your suggestion for a new feature. It might help to describe a problem or use case, plus any alternatives that you have considered.
+    validations:
+      required: true