Repository navigation
fix: abort pipeline on deadlock detection when recovery is disabled - #252
Conversation
When deadlock detection fires and --deadlock-recover is not enabled, the pipeline now sets an error and exits instead of logging a warning and continuing to hang indefinitely. Workers observe has_error() on their next iteration and exit gracefully. Both monitor paths are fixed: - run_monitor_loop (base.rs) — used by FASTQ pipeline - inline monitor thread (bam.rs) — used by BAM pipeline
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #252 +/- ##
==========================================
- Coverage 89.28% 89.27% -0.02%
==========================================
Files 119 119
Lines 57770 57784 +14
==========================================
+ Hits 51582 51584 +2
- Misses 6188 6200 +12 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (2)
📝 WalkthroughWalkthroughThe pull request modifies deadlock detection handling in the unified pipeline's monitor loop across two files. Previously, 🚥 Pre-merge checks | ✅ 3✅ Passed checks (3 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Summary
--deadlock-recoveris not enabled, the pipeline now aborts with a clear error instead of logging a warning and hanging indefinitelyhas_error()on their next iteration and exit gracefully — no panics, proper output cleanupTimedOutkind with an actionable message suggesting--deadlock-recoverrun_monitor_loop(FASTQ pipeline) and the inline BAM monitor threadBefore
Deadlock detected → warning logged → progress timer reset → pipeline continues hanging → repeated warnings every 10s → user must Ctrl-C
After
Deadlock detected → warning logged with diagnostics →
set_error()called → all workers exit on nexthas_error()check → pipeline exits with non-zero status and clear messageTest plan
cargo ci-fmtcleancargo ci-lintclean--deadlock-timeout 5(no--deadlock-recover) on a workload that deadlocks — should exit with error after 5s instead of hanging