Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature Request]: Support Streaming model updates for the RunInference transform #24042

Closed
AnandInguva opened this issue Nov 8, 2022 · 3 comments
Assignees
Labels
done & done Issue has been reviewed after it was closed for verification, followups, etc. ml new feature P2 python run-inference

Comments

@AnandInguva
Copy link
Contributor

What would you like to happen?

For streaming pipeline in Python, add support for updating the models.

  • Allow streaming pipelines to update the model(s) in use without requiring the use of pipeline lifecycle events ( Update / Drain ) .

Issue Priority

Priority: 2

Issue Component

Component: run-inference

@AnandInguva
Copy link
Contributor Author

.take-issue

@AnandInguva
Copy link
Contributor Author

AnandInguva commented Jan 27, 2023

Part 1: Add API that accepts Singleton side input PCollection.
Part 2: Use case of the API. Example: WatchFilePattern transform with an example.
Part 3: DLQ to catch errors.

Also, update beam website documentation with

## Side Inputs to Update Models
From Beam 2.45.0, the RunInference PTransform will accept a side input of `ModelMetadata`, which is a `NamedTuple` containing the `model_id` and `model_name`.
  * `model_id`: The model_id is used to load the models. It could be an URI or path to the model.
  * `model_name`: Unique identifier used to append the metrics. This should be short relative to the model_id so that it can be attached to the metrics to identify which model was used to calculate the metrics.

**Note**: The side input PCollection must be compatible with `AsSingleton` view or the pipeline will result in error.

**Note**: If the main PCollection emits inputs and side input has yet to receive inputs, the main PCollection will get buffered until there is
            an update to the side input. This could happen with Global windowed side inputs with data driven triggers such as `AfterCount`, `AfterProcessingTime`. So until there is an update to the side input, emit the default/initial model id that is used to pass the respective `ModelHandler` as side input..

Some other ideas as extension of this feature:

  1. Add Read/Write lock for updating and runInference part.
  2. An idea from Danny : https://github.com/apache/beam/pull/25200/files#r1089107034
    image

@AnandInguva AnandInguva mentioned this issue Feb 9, 2023
3 tasks
@AnandInguva
Copy link
Contributor Author

.close-issue

@github-actions github-actions bot added this to the 2.47.0 Release milestone Apr 1, 2023
@damccorm damccorm added the done & done Issue has been reviewed after it was closed for verification, followups, etc. label Apr 4, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
done & done Issue has been reviewed after it was closed for verification, followups, etc. ml new feature P2 python run-inference
Projects
None yet
Development

No branches or pull requests

2 participants