Skip to content

[Transform] Reset failure counter after 1st success after a failure #76074

@hendrikmuhs

Description

@hendrikmuhs

Transform retries per default 10 times for non-permanent errors and goes into failed state if it fails 10 times in a row.

However, inspecting the code, it turns out the counter is reset per checkpoint. This does not take into account the amount of data processed for a checkpoint, neither does a success reset the counter to 0.

If a checkpoint looks like this with s==success and f== failure:

ssfffsfssfffsssssssssfssssssssssssfsssssssssf

the transform fails, although the failures were not in a row and there could be many s in between.

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions