Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 8 additions & 1 deletion .cargo/config.toml
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,14 @@ ci-tag-literals = "run --package xtask -- check-tag-literals"
ci-doctest = "test --doc --workspace --features compare,simulate,profile-adjacency --locked"
# Build the docs and fail on any rustdoc warning (e.g. broken intra-doc links).
# RUSTDOCFLAGS="-D warnings" is set by the caller (CI job / pre-push).
ci-doc = "doc --no-deps --workspace --features compare,simulate,profile-adjacency --locked"
#
# `--document-private-items` is load-bearing, not cosmetic. Without it rustdoc
# only checks links on items it renders — i.e. public ones — so a broken link in
# the doc comment of a private fn, a private field, or a `#[cfg(test)]` helper
# passes silently. That is not hypothetical: several such links accumulated
# undetected and had to be found by hand. The flag makes the docs job check every
# doc comment in the workspace, which is what the job's name implies it does.
ci-doc = "doc --no-deps --workspace --document-private-items --features compare,simulate,profile-adjacency --locked"
# Run only the #[ignore]-d integration tests (the sort-correctness suites in
# tests/integration/{test_sort_correctness,test_async_reader,test_sort_write_index}.rs).
# They require the `samtools` binary on PATH — samtools builds the BAM fixtures,
Expand Down
2 changes: 1 addition & 1 deletion crates/fgumi-bam-io/src/reorder.rs
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@ pub struct ReorderBuffer<T> {
/// Sparse buffer: index (seq - `next_seq`) maps to `Option<(T, usize)>` where
/// usize is the pre-computed heap size (0 if not tracked).
buffer: VecDeque<Option<(T, usize)>>,
/// Next sequence number to release (also the sequence number corresponding to buffer[0]).
/// Next sequence number to release (also the sequence number corresponding to `buffer[0]`).
next_seq: u64,
/// Number of items currently stored.
count: usize,
Expand Down
10 changes: 5 additions & 5 deletions crates/fgumi-pipeline-core/src/builder.rs
Original file line number Diff line number Diff line change
Expand Up @@ -1389,7 +1389,7 @@ const DEADLOCK_FATAL_MULTIPLE: u64 = 6;
/// backpressure is the internal `SortMergeSlot` slot table, not a pipeline
/// `CountBounded` edge.) [`ensure_monitor_visible_transports`] enforces this
/// invariant in **every** build (it returns
/// [`PipelineError::MonitorBlindTransport`], not a debug assertion) whenever the
/// [`crate::signal::PipelineError::MonitorBlindTransport`], not a debug assertion) whenever the
/// monitor is armed, so a future step that declares a monitor-blind transport on
/// a fail-fast pipeline fails at startup rather than silently losing the wedge
/// verdict. Extending the probe to count-based queues for full defense-in-depth
Expand Down Expand Up @@ -1455,11 +1455,11 @@ fn first_monitor_blind_transport(
/// `CountBounded`/`Unbounded` for `#[cfg(test)]` chains (which do not arm the
/// monitor), so this only fires for a real fail-fast pipeline.
///
/// Returns a [`PipelineError::MonitorBlindTransport`] (rather than panicking or
/// Returns a [`crate::signal::PipelineError::MonitorBlindTransport`] (rather than panicking or
/// being a debug-only check) so the guard runs in release builds — where the
/// blind spot actually matters — yet a misconfigured chain fails gracefully at
/// startup, consistent with the other build/run-time validations (e.g.
/// [`PipelineError::NotEnoughThreads`]) rather than crashing the process.
/// [`crate::signal::PipelineError::NotEnoughThreads`]) rather than crashing the process.
fn ensure_monitor_visible_transports(
steps: &[Box<dyn super::erased::ErasedStep>],
graph: &super::topology::ChainGraph,
Expand All @@ -1480,7 +1480,7 @@ fn ensure_monitor_visible_transports(
/// wedge from upstream starvation (mirrors legacy `check_deadlock_and_restore`):
/// - idle with nothing in flight → starvation, reset the clock, keep watching;
/// - stuck work past `warn_timeout` → `warn` snapshot, once per warn window;
/// - stuck work past `fatal_timeout` → record [`PipelineError::TimedOut`] and
/// - stuck work past `fatal_timeout` → record [`crate::signal::PipelineError::TimedOut`] and
/// `cancel()`, so workers observe `is_done()` and the run fails fast instead
/// of hanging forever.
///
Expand Down Expand Up @@ -1541,7 +1541,7 @@ struct StallMonitorState {

/// React to one classified [`StallVerdict`], mutating the rolling monitor
/// `mon_state` and performing the verdict's side effect (a warn snapshot, or
/// recording a fatal [`PipelineError::TimedOut`] + `cancel`). Extracted from
/// recording a fatal [`crate::signal::PipelineError::TimedOut`] + `cancel`). Extracted from
/// [`run_deadlock_monitor`]'s poll loop so each per-verdict action is
/// unit-testable without spawning a thread or waiting on wall-clock time —
/// mirroring the pure `classify_stall` / `in_flight_bytes` split. Returns
Expand Down
2 changes: 1 addition & 1 deletion crates/fgumi-pipeline-core/src/handles.rs
Original file line number Diff line number Diff line change
Expand Up @@ -358,7 +358,7 @@ pub(crate) struct Branch<T: Send + HeapSize + 'static> {
pub(crate) metrics: Option<Arc<crate::runtime::metrics::EdgeMetrics>>,
}

/// Mint a per-edge [`EdgeMetrics`] when instrumentation is on, else `None`.
/// Mint a per-edge [`crate::runtime::metrics::EdgeMetrics`] when instrumentation is on, else `None`.
/// Called once per branch in the `build_branch*` constructors; the same handle
/// is shared by the transport (push counts) and the input handle (pop counts).
fn edge_metrics(
Expand Down
4 changes: 2 additions & 2 deletions crates/fgumi-pipeline-core/src/topology.rs
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
//! `PipelineBuilder::build()` to assert all-outputs-wired and by the
//! runtime to construct queue topology.

/// Static branch-index display names covering every branch up to [`MAX_ARITY`].
/// Static branch-index display names covering every branch up to [`crate::outputs::MAX_ARITY`].
/// Output branch counts are bounded by `MAX_ARITY` at registration, so a valid
/// branch index always maps to a name and the build-error message never prints a
/// placeholder. Out-of-range indices fall back to `"?"` (never hit in practice,
Expand All @@ -27,7 +27,7 @@ pub struct ChainGraph {
/// Defaults to `0` for single-input consumers; multi-input
/// consumers (`Step2` / future `StepN`) record their per-input
/// slot explicitly so [`crate::runtime::contexts`]
/// can build the right [`TwoInputHandles`] wrapper.
/// can build the right [`crate::handles::TwoInputHandles`] wrapper.
consumer_input_slots: Vec<Option<usize>>,
/// `branch_count[step]` — number of output branches for that step.
branch_counts: Vec<usize>,
Expand Down
14 changes: 7 additions & 7 deletions crates/fgumi-sort/src/external.rs
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,7 @@ use tempfile::TempDir;
/// Maximum number of records held in one in-memory chunk.
///
/// Each key carries its ingest position within the chunk (see
/// [`RawSortKey::set_position`](crate::keys::RawSortKey::set_position)) so that
/// [`RawSortKey::set_position`]) so that
/// the key is a *total* order and the chunk sort can be unstable. That position
/// is a `u32`, so a chunk that grew past `u32::MAX` records would have to stamp
/// two records with the same position — exact name/flag ties would stop being
Expand Down Expand Up @@ -1389,8 +1389,8 @@ impl<K: RawSortKey + 'static> MainThreadChunkConsumer<K> {

/// Parse the next record from a source's byte stream.
///
/// Handles the format: for `EMBEDDED_IN_RECORD` keys, reads [len(4)][record(len)].
/// For keyed format, reads [key][len(4)][record(len)].
/// Handles the format: for `EMBEDDED_IN_RECORD` keys, reads `[len(4)][record(len)]`.
/// For keyed format, reads `[key][len(4)][record(len)]`.
fn parse_next_record(&mut self, source_id: usize, buf: &mut Vec<u8>) -> Result<Option<K>> {
let mut len_buf = [0u8; 4];

Expand Down Expand Up @@ -3474,7 +3474,7 @@ impl RawExternalSorter {
/// Disk chunks become `PoolDisk` sources: the shared worker pool reads and
/// decompresses them in the background while the main thread parses records,
/// so no per-source threads are spawned. Both the plain
/// ([`merge_chunks_generic`]) and indexed ([`merge_chunks_with_index`])
/// ([`Self::merge_chunks_generic`]) and indexed ([`Self::merge_chunks_with_index`])
/// merges go through this pool-integrated path.
fn build_chunk_sources<K: RawSortKey + Default + 'static>(
chunk_files: &[PathBuf],
Expand Down Expand Up @@ -3503,8 +3503,8 @@ impl RawExternalSorter {

/// Build the pooled merge sources and activate Phase 2.
///
/// Shared by both the plain ([`merge_chunks_generic`]) and indexed
/// ([`merge_chunks_with_index`]) merges so the Phase-2 lifecycle is
/// Shared by both the plain ([`Self::merge_chunks_generic`]) and indexed
/// ([`Self::merge_chunks_with_index`]) merges so the Phase-2 lifecycle is
/// single-sourced and the two paths cannot drift. Returns the sources plus
/// an RAII [`Phase2Guard`] (borrowing `pool`); callers finish through
/// [`Phase2Guard::finish_output`], and `Drop` resets Phase 2 on any path
Expand Down Expand Up @@ -3694,7 +3694,7 @@ impl RawExternalSorter {
/// Identical input/output pipeline to `merge_chunks_generic` — the shared
/// worker pool decompresses the input runs (`PoolDisk` sources) and
/// compresses the output blocks — but the output goes through
/// [`PooledBamWriter::new_indexing`], which tracks each record's virtual
/// [`crate::pooled_bam_writer::PooledBamWriter::new_indexing`], which tracks each record's virtual
/// file offset and returns the generated BAI index. A single pool of
/// workers therefore serves both decompression and compression; there is no
/// separate writer thread pool and no serialized single-reader input.
Expand Down
2 changes: 1 addition & 1 deletion crates/fgumi-sort/src/memory_probe.rs
Original file line number Diff line number Diff line change
Expand Up @@ -488,7 +488,7 @@ impl MergeProbe {
/// threshold.
///
/// `pool_depths` is the `(raw_input, decompressed_input, buffer_pool)`
/// triple from [`SortWorkerPool::phase1_queue_depths`].
/// triple from [`crate::worker_pool::SortWorkerPool::phase1_queue_depths`].
pub fn log_mid_with_depths(
&mut self,
pool_depths: (usize, usize, usize),
Expand Down
4 changes: 2 additions & 2 deletions crates/fgumi-sort/src/read_ahead.rs
Original file line number Diff line number Diff line change
Expand Up @@ -399,7 +399,7 @@ impl PooledInputStream {
/// Read the next raw BAM record, borrowing its bytes from the current
/// decompressed block when possible.
///
/// This is the borrow-in-place counterpart to [`read_raw_record`]: it removes
/// This is the borrow-in-place counterpart to [`fgumi_raw_bam::read_raw_record`]: it removes
/// the per-record `read_exact` copy into a `RawRecord` on the common path
/// where the record body lies wholly within the current decompressed block.
///
Expand All @@ -414,7 +414,7 @@ impl PooledInputStream {
/// The returned slice borrows `self`; it is invalidated by the next call to
/// any method on this stream. Returns `Ok(None)` at clean EOF.
///
/// A `block_size` of 0 is treated as EOF, mirroring [`read_raw_record`].
/// A `block_size` of 0 is treated as EOF, mirroring [`fgumi_raw_bam::read_raw_record`].
///
/// # Errors
///
Expand Down
2 changes: 1 addition & 1 deletion src/lib/commands/clip.rs
Original file line number Diff line number Diff line change
Expand Up @@ -238,7 +238,7 @@ impl ClipParams {
/// the template — including secondary/supplementary alignments — before the primary pair is
/// clipped, matching fgbio `ClipBam`'s `template.allReads.foreach(upgradeAllClipping)`
/// (`ClipBam.scala:123`). Doing this template-wide pre-pass here (rather than per-primary
/// inside [`clip_pair`]/[`clip_fragment`]) is what lets supplementary reads' clipping be
/// inside `clip_pair`/`clip_fragment`) is what lets supplementary reads' clipping be
/// upgraded too, and keeps both threading paths in lockstep since both go through this method.
///
/// This is the single shared implementation used by both the single-threaded and
Expand Down
10 changes: 5 additions & 5 deletions src/lib/commands/compare/bams.rs
Original file line number Diff line number Diff line change
Expand Up @@ -152,7 +152,7 @@ impl CommandPreset {
/// `clip` rewrites CIGAR/SEQ/QUAL in the clipped span and, as a consequence, regenerates
/// the alignment tags (`NM`/`UQ`/`MD`) and repairs mate-pair metadata — but neither ever
/// recomputes the depth-tag values. All five compare positionally under
/// [`ContentPredicate::Exact`](super::engines::content::ContentPredicate::Exact):
/// [`ContentPredicate::Exact`]:
/// because fgumi clamps the depth tags to fgbio's `Short` ceiling at the source, the
/// tags are bit-identical and an exact comparison is both sound and complete — no
/// consensus-specific predicate is needed. Since `Exact` holds every core SAM field
Expand All @@ -168,7 +168,7 @@ impl CommandPreset {
/// ([`super::engines::molecule_join::molecule_join_compare`]): molecules are matched by
/// an MI-invariant canonical id (no re-sort — both inputs must already be grouped),
/// and each matched pair is checked for record membership, content under
/// [`ContentPredicate::ExactMinusMi`](super::engines::content::ContentPredicate::ExactMinusMi)
/// [`ContentPredicate::ExactMinusMi`]
/// (everything except the MI tag), and duplex `/A`/`/B` strand-partition equivalence —
/// the predicate excludes MI precisely because the canonical-id matching, not the
/// content predicate, is what verifies MI equivalence.
Expand Down Expand Up @@ -197,7 +197,7 @@ impl CommandPreset {
/// The [`ContentPredicate`] to use for this preset's content comparison. See
/// [`Self::resolve`] for the full resolution table and rationale. Note that for `Group`
/// (resolved mode `CompareMode::Grouping`), this value is *not* consulted by the
/// molecule-join engine — [`engines::molecule_join::molecule_join_compare`] hardcodes
/// molecule-join engine — [`super::engines::molecule_join::molecule_join_compare`] hardcodes
/// [`ContentPredicate::ExactMinusMi`] internally and is not configurable via
/// `--command`/`--mode`. `Group` never reaches `execute_content`, but this method is
/// still live for it (its resolved value is exercised only by the
Expand Down Expand Up @@ -960,7 +960,7 @@ impl CompareBams {
///
/// This is the sole `CompareMode::Grouping` path (and what `--command group` uses). Both
/// inputs are cut into per-molecule runs and matched across the two files by an
/// MI-invariant canonical id (see [`molecule_join_compare`]) — no re-sort, so both inputs
/// MI-invariant canonical id (see [`super::engines::molecule_join::molecule_join_compare`]) — no re-sort, so both inputs
/// must already be grouped (same-MI reads consecutive, as `fgumi group`/`fgbio group`
/// output is). This makes the comparison inherently order-independent at the molecule
/// level — `--ignore-order` is therefore a no-op under `--mode grouping`. Each matched
Expand All @@ -972,7 +972,7 @@ impl CompareBams {
///
/// # Errors
///
/// Returns an error if either input cannot be read (see [`molecule_join_compare`]), or
/// Returns an error if either input cannot be read (see [`super::engines::molecule_join::molecule_join_compare`]), or
/// [`super::CompareMismatch`] if the two BAMs are found to differ (non-zero exit via the
/// `Command` trait).
fn execute_grouping(&self, input1: OpenedInput, input2: OpenedInput) -> Result<u64> {
Expand Down
2 changes: 1 addition & 1 deletion src/lib/commands/compare/engines/mod.rs
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
//! Comparison engines for `fgumi compare bams`.
//!
//! An "engine" pairs records from the two input streams (see
//! [`positional`](self::positional), added alongside the positional-alignment work) and
//! [`positional`], added alongside the positional-alignment work) and
//! decides whether each pair is content-equal via a [`content::ContentPredicate`].
//! Splitting pairing (positional) from equality (content) keeps the pairing logic honest:
//! it can never quietly resync past a mismatch just because two *unrelated* records
Expand Down
8 changes: 4 additions & 4 deletions src/lib/commands/compare/engines/positional.rs
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,8 @@
//! coincidence, not evidence of parity, so it must never be allowed to mask
//! the desync.
//! - If the keys agree, the pair is compared under a
//! [`ContentPredicate`](super::content::ContentPredicate) via
//! [`content_diffs`](super::content::content_diffs).
//! [`ContentPredicate`] via
//! [`content_diffs`].
//!
//! A record-count mismatch (one file longer than the other) is reported as a
//! presence difference, independent of and in addition to any key mismatch.
Expand Down Expand Up @@ -53,7 +53,7 @@ pub struct PositionalOutcome {
/// pairing stopped at the first desync rather than attempting to resync.
pub key_mismatch_at: Option<u64>,
/// `true` if the two inputs' `@HD`/`@SQ`/`@RG` headers disagreed on a field
/// [`compare_headers`](super::header::compare_headers) considers significant (`@PG`/`@CO`
/// [`compare_headers`] considers significant (`@PG`/`@CO`
/// are normalized and never contribute here).
pub header_mismatch: bool,
/// Human-readable diff strings (header, key mismatch, content diffs, and/or the
Expand Down Expand Up @@ -85,7 +85,7 @@ impl PositionalOutcome {
/// always reflect the true totals even after pairing has stopped.
///
/// Also compares the two inputs' headers via
/// [`compare_headers`](super::header::compare_headers) (`@HD`/`@SQ`/`@RG`, normalizing away
/// [`compare_headers`] (`@HD`/`@SQ`/`@RG`, normalizing away
/// `@PG`/`@CO`); a significant divergence is folded into
/// `PositionalOutcome::header_mismatch`/`PositionalOutcome::is_match` alongside the
/// record-level findings.
Expand Down
4 changes: 2 additions & 2 deletions src/lib/commands/compare/engines/sort_verify.rs
Original file line number Diff line number Diff line change
Expand Up @@ -88,7 +88,7 @@ pub struct SortVerifyOutcome {
/// compared (no resync), though both streams are still drained for accurate counts.
pub presence_mismatch: bool,
/// `true` if the two inputs' `@HD`/`@SQ`/`@RG` headers disagreed on a field
/// [`compare_headers`](super::header::compare_headers) considers significant (`@PG`/`@CO`
/// [`compare_headers`] considers significant (`@PG`/`@CO`
/// are normalized and never contribute here). Note that `@HD` `SO`/`GO`/`SS` agreement is
/// already implied by `detect_sort_order` succeeding on both inputs with a matching
/// [`SortOrder`] (checked before this field is ever populated) *only when both writers use
Expand Down Expand Up @@ -822,7 +822,7 @@ where
/// by maximal equal-core-sort-key run (see the module docs).
///
/// Never re-sorts either input. Also compares the two inputs' headers via
/// [`compare_headers`](super::header::compare_headers) (`@HD`/`@SQ`/`@RG`, normalizing away
/// [`compare_headers`] (`@HD`/`@SQ`/`@RG`, normalizing away
/// `@PG`/`@CO`); a significant divergence is folded into
/// [`SortVerifyOutcome::header_mismatch`]/[`SortVerifyOutcome::is_match`] alongside the
/// run-comparison findings. `max_diffs` caps the number of entries collected in
Expand Down
2 changes: 1 addition & 1 deletion src/lib/commands/extract.rs
Original file line number Diff line number Diff line change
Expand Up @@ -815,7 +815,7 @@ impl Extract {
/// <https://support.illumina.com/help/BaseSpace_OLH_009008/Content/Source/Informatics/BS/FileFormat_FASTQ-files_swBS.htm>.
///
/// An old-style Casava (<1.8) `/1` / `/2` read-number suffix is stripped from
/// the returned name (see [`strip_read_suffix`](crate::fastq_parse::strip_read_suffix))
/// the returned name (see [`strip_read_suffix`])
/// so both mates of a pair share an identical QNAME, matching fgbio's `FastqSource`.
/// Stripping happens before UMI extraction so a read-number digit never leaks into the UMI.
///
Expand Down
2 changes: 1 addition & 1 deletion src/lib/commands/review.rs
Original file line number Diff line number Diff line change
Expand Up @@ -722,7 +722,7 @@ impl Review {
None
}

/// Extracts the leading numeric value from a typed VCF sample [`Value`] (used to
/// Extracts the leading numeric value from a typed VCF sample `Value` (used to
/// read `AF`). Scalars convert directly; arrays take the first present element;
/// strings parse their first comma-separated token (fgbio does `AF.toDouble`).
fn value_as_f64(
Expand Down
2 changes: 1 addition & 1 deletion src/lib/commands/simulate/common.rs
Original file line number Diff line number Diff line change
Expand Up @@ -434,7 +434,7 @@ impl ReferenceGenome {
Some(subseq.to_vec())
}

/// Build a BAM [`Header`] with `@SQ` lines for every loaded contig.
/// Build a BAM `Header` with `@SQ` lines for every loaded contig.
pub(super) fn build_bam_header(&self) -> noodles::sam::header::Header {
use bstr::BString;
use noodles::sam::header::Header;
Expand Down
2 changes: 1 addition & 1 deletion src/lib/unified_pipeline/fastq.rs
Original file line number Diff line number Diff line change
Expand Up @@ -3177,7 +3177,7 @@ fn fastq_try_step_parse<R: BufRead + Send, P: Send + MemoryEstimate>(
/// Create templates by zipping records from synchronized FASTQ streams.
///
/// For single-end: each record becomes its own template.
/// For paired-end: R1[i] and R2[i] are zipped into a single template.
/// For paired-end: `R1[i]` and `R2[i]` are zipped into a single template.
fn create_templates_from_streams(
mut streams: Vec<FastqParsedStream>,
) -> io::Result<Vec<FastqTemplate>> {
Expand Down
Loading