[Feature Request] Inquiry About ONNX Runtime Support for Dynamic Decoding Correction in Machine Translation on Android

[tigflanker]

Describe the feature request

Dear ONNX Runtime Maintainers,

Merry Christmas and Happy New Year to you!

I am planning to develop a machine translation product based on the T5 model, which will be deployed on Android devices. However, there is a specific feature that I am unsure if ONNX Runtime currently supports, and I would greatly appreciate your guidance on this matter.

During machine translation, due to the limitations of the model size, users sometimes find that certain tokens in the target language are not translated correctly. In such cases, users may input the correct target token themselves. At this point, I would like the model to re-decode the subsequent part of the sequence starting from where the user made the correction (assuming that all content before this token position has been translated correctly).

This requirement calls for a rather flexible functionality, and I am uncertain whether the current version of ONNX Runtime can support such an operation. If it does, could you please provide me with an API or, even better, an example? Your assistance would be immensely appreciated.

Thank you very much for your time and help.

Describe scenario use case

Dynamic Decoding Correction in Machine Translation on Android.