[Task]: RunInference - send failures to dead letter queue #24209
Labels
done & done
Issue has been reviewed after it was closed for verification, followups, etc.
ml
P2
python
run-inference
task
Milestone
What needs to happen?
Right now, if RunInference fails a batch inference, it fails the whole transform. For batch pipelines, this means failing the pipeline on non-retryable failures (which represent most inference failures), for streaming it means infinite retries and a stuck pipeline.
We should handle failures by passing them to the next step as part of the
PredictionResult
object instead so that users can perform custom error handling. We should also document this behavior in the PyDoc and on our website.Issue Priority
Priority: 2
Issue Component
Component: run-inference
The text was updated successfully, but these errors were encountered: