Skip to content

Conversation

@pencil
Copy link
Collaborator

@pencil pencil commented Oct 23, 2025

This pull request refactors the AtMostOnceBarrier logic to improve reliability when scheduling workflow completion callbacks. The core change is to separate the confirmation of barrier release from the initial barrier wait, ensuring that expensive workflow deserialization succeeds before marking the barrier as released. This decreases the risk of missed callback executions if a worker crashes during deserialization.

@pencil pencil requested a review from dan-woz October 23, 2025 22:51
@macroscopeapp
Copy link

macroscopeapp bot commented Oct 23, 2025

Rework barrier release to a two-step process by adding AtMostOnceBarrier.confirm_release in dramatiq_workflow/_barrier.py and updating WorkflowMiddleware._process_completion_callbacks in dramatiq_workflow/_middleware.py

Introduce a two-step barrier release: record completion via AtMostOnceBarrier.confirm_release and run callbacks after confirmation, with middleware deferring confirmation until after workflow unserialization.

📍Where to Start

Start with the WorkflowMiddleware._process_completion_callbacks handler in dramatiq_workflow/_middleware.py, then review AtMostOnceBarrier.confirm_release in dramatiq_workflow/_barrier.py.


Macroscope summarized 2c529a5.

Copy link
Collaborator

@dan-woz dan-woz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gonna chuckle if this is the problem. Deserialization: free until it's not 😔

@pencil pencil marked this pull request as draft November 18, 2025 18:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants