BUG: When restarting a failed task, the task completes but state of job is stuck in running.

[bwilliams42]

i also think the job should continue from where it left off. After the failed task is re-ran, it should move onto the next task(s)

[thieman]

Can you provide a minimal test case here? The job export functionality can be useful for that. Does this involve any remote tasks?

[bwilliams42]

@thieman. Sure. If you create a job that has say 5 tasks. 1 > 2 > 3 > 4 > 5. You purposely make number 3 fail. The execution stops there. 4 and 5 are not executed. You then fix 3 and hit the re-run failed tasks button, 3 will complete successfully. The state of the overall job remains as running. So I can't restart the the job using the "start job from beginning" button. This is definitely a bug. I need to restart the server to force it back to a normal state.

There are no remote tasks. How would the export help?

-B

[thieman]

Sorry, was just suggesting export as a way of passing along a test job. Will take a look.

[bwilliams42]

Ah i see. I can whip one up and send it over a little later. It will be later on tonight/tomorrow. Is that OK?

[thieman]

Sure. Sounds simple enough that it may not be necessary, but will be helpful if I encounter issues.

[nnfuzzy]

I experienced the same. Here are simple templates.
https://github.com/nnfuzzy/dagobah/tree/master/tests

[rclough]

I believe the issue arrived with one of my more recent PRs, where we now are operating on a copy of the graph. When a task fails, the DAG snapshot gets deleted. When you retry, the snapshot is not reconstructed, and thus after the task finishes, downstream() doesnt operate as expected because snapshot is None

My initial inclination is just to rebuild the snapshot within the retry() method. I'm trying to think if there are any edge cases in which we would want to preserve the snapshot on failure (especially since fixing a failure may mean changing the the DAG).

[thieman]

Should be corrected now, going to push out a release shortly.

[thieman]

v0.3.1 is live on PyPi with this fix