Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 11 additions & 7 deletions crates/fgumi-metrics/src/dedup.rs
Original file line number Diff line number Diff line change
Expand Up @@ -353,9 +353,14 @@ impl crate::ProcessingMetrics for DeduplicationMetrics {
}

/// Serializable row for the `--duplication-ladder` sampled duplication
/// saturation curve: "after `templates_seen` templates processed (in
/// coordinate order) for this library, what cumulative fraction were
/// duplicates".
/// ladder: "after `templates_seen` templates processed (in coordinate order)
/// for this library, what cumulative fraction were duplicates".
///
/// This is not a saturation curve. A template's duplicate status is decided
/// by the other templates at its own position, so the fraction after N
/// templates is the duplicate rate of the genome covered so far, not the rate
/// a library sequenced to N templates would show. Changes along the ladder
/// reflect regions with different duplicate rates.
///
/// One row per (library, snapshot) — a library gets a row each time its
/// cumulative `templates_seen` crosses a multiple of `--ladder-interval`,
Expand All @@ -376,10 +381,9 @@ pub struct DuplicationLadderMetrics {
/// `templates_seen` on the first snapshot.
pub window_templates: u64,
/// Marginal duplicate fraction over just this window's templates
/// (`window_duplicate_templates / window_templates`). Often the more legible
/// view of the saturation curve than the cumulative `duplicate_fraction`,
/// since it isolates each depth band instead of averaging over all prior
/// ones. Mirrors dupblaster's per-window complexity columns.
/// (`window_duplicate_templates / window_templates`). Often more legible than
/// the cumulative `duplicate_fraction`, since it isolates each window
/// instead of averaging over all prior ones. Mirrors dupblaster's per-window complexity columns.
#[serde(with = "crate::float")]
pub window_duplicate_fraction: f64,
}
Expand Down
18 changes: 10 additions & 8 deletions src/lib/commands/dedup.rs
Original file line number Diff line number Diff line change
Expand Up @@ -256,16 +256,16 @@ pub(crate) struct CollectedDedupCounts {
}

//////////////////////////////////////////////////////////////////////////////
// Duplication saturation ladder (--duplication-ladder)
// Duplication ladder (--duplication-ladder)
//////////////////////////////////////////////////////////////////////////////

/// Per-library cumulative counters and emitted snapshot rows backing
/// `--duplication-ladder`.
///
/// # Ordering
///
/// A saturation curve plots "after N templates processed, in coordinate
/// order, what cumulative fraction were duplicates" — so [`Self::record`]
/// The ladder records "after N templates processed, in coordinate order, what
/// cumulative fraction were duplicates" — so [`Self::record`]
/// MUST be called in strict serial/coordinate order, one call per position
/// group. It is wired into the chain's `MiAssignDedup` step (see
/// `pipeline::chains::commands::dedup::build_mi_assign_step`), which the
Expand All @@ -291,7 +291,7 @@ pub(crate) struct DuplicationLadderRecorder {
rows: Vec<(u16, u64, u64, u64, u64)>,
}

/// Running cumulative state for one library's saturation ladder.
/// Running cumulative state for one library's duplication ladder.
#[derive(Default)]
struct LadderLibraryState {
/// Cumulative templates seen so far for this library.
Expand Down Expand Up @@ -1221,10 +1221,12 @@ pub struct MarkDuplicates {
#[arg(short = 'H', long = "family-size-histogram")]
pub family_size_histogram: Option<PathBuf>,

/// Path to write the sampled duplication saturation ladder: per-library
/// cumulative duplicate fraction vs. templates seen (in coordinate
/// order), snapshotted every `--ladder-interval` templates. Off by
/// default (no recorder is built, so no added work).
/// Path to write the sampled duplication ladder: per-library cumulative
/// duplicate fraction vs. templates seen (in coordinate order),
/// snapshotted every `--ladder-interval` templates. Because templates are
/// counted in coordinate order, this shows how the duplicate rate varies
/// along the genome; it is not a saturation curve. Off by default (no
/// recorder is built, so no added work).
#[arg(long = "duplication-ladder")]
pub duplication_ladder: Option<PathBuf>,

Expand Down
2 changes: 1 addition & 1 deletion src/lib/pipeline/chains/builder.rs
Original file line number Diff line number Diff line change
Expand Up @@ -6027,7 +6027,7 @@ impl<'a> ChainBuilder<'a> {
dedup.include_unmapped,
accumulators_for_process,
);
// Duplication-saturation ladder recorder (`--duplication-ladder`).
// Duplication ladder recorder (`--duplication-ladder`).
// `Some` only when the flag is set; shared (via `Arc`) between the serial
// MI-assign step, which accumulates it in coordinate order, and the
// finalize hook, which writes it. `None` means the MI-assign step does
Expand Down
4 changes: 2 additions & 2 deletions src/lib/pipeline/chains/commands/dedup.rs
Original file line number Diff line number Diff line change
Expand Up @@ -148,7 +148,7 @@ impl FinalizeHook for DedupFinalizeHook {
write_family_size_histogram(&final_family_sizes, path)?;
}

// Write the duplication-saturation ladder if requested. Accessed through
// Write the duplication ladder if requested. Accessed through
// the lock (not `Arc::try_unwrap`): the serial MI-assign step holds a
// clone of this `Arc` that may still be alive here.
if let Some((path, recorder)) = duplication_ladder {
Expand Down Expand Up @@ -268,7 +268,7 @@ pub(crate) fn build_process_step(
/// monotonically increasing MI offsets to each batch.
///
/// When `ladder_recorder` is `Some`, this step also records the
/// `--duplication-ladder` saturation curve — per position group, in this
/// `--duplication-ladder` — per position group, in this
/// serial/coordinate-order seam, **not** in the parallel serialize step.
/// `MiAssign` is `Serial` + `ByItemOrdinal`, so batches reach this closure in
/// input-record order and the groups within a batch are in coordinate order —
Expand Down
Loading