You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
In BEAM-13203SnsIO.writeAsync was deprecated to address the risk of data loss in that transform. Unfortunately, it has considerable issues in design and implementation with respect to the beam programming model as well as general async programming:
Output for a bundle may be emitted even after the bundle is finished (lack of join / waiting for pending futures), additionally output is emitted from multiple threads risking concurrent modification issues on underlying data structures.
Exceptions disappear into the void (respective tests are implemented incorrectly as they don't throw asynchronously and could make one think this works).
Parallelism is unlimited causing the writer to instantly accumulate everything into memory.
With BEAM-13563 it will be possible to change SnsIO.write into a more performant async implementation without any breaking changes to the public user facing API making SnsIO.writeAsync obsolete.
Imported from Jira BEAM-13824. Original Jira may contain additional context.
Reported by: mosche.
The text was updated successfully, but these errors were encountered:
In BEAM-13203
SnsIO.writeAsync
was deprecated to address the risk of data loss in that transform. Unfortunately, it has considerable issues in design and implementation with respect to the beam programming model as well as general async programming:With BEAM-13563 it will be possible to change
SnsIO.write
into a more performant async implementation without any breaking changes to the public user facing API makingSnsIO.writeAsync
obsolete.Imported from Jira BEAM-13824. Original Jira may contain additional context.
Reported by: mosche.
The text was updated successfully, but these errors were encountered: