Support streaming for text generation task

[alonsosilvaallende]

I'm trying to add streaming to a text generation task. I saw that in transformers.js it's handled with a callback_function:
huggingface/transformers.js#641 (comment)

So I was trying to do the same. This is the code I have:

from transformers_js_py import import_transformers_js

transformers = await import_transformers_js()
pipeline = transformers.pipeline

# Define the callback function that will handle streaming
def callback_function(beams):
    decoded_text = pipeline.tokenizer.decode(beams[0]["output_token_ids"], {
        "skip_special_tokens": True
    })
    print(decoded_text, end="", flush=True)  # Print streaming output

pipe = await pipeline('text-generation', 'onnx-community/Qwen2.5-0.5B-Instruct')

message = [{"role": "user", "content": "Hi!"}]
out = await pipe(
    message,
    callback_function=callback_function  # Pass the callback function
)

print(out[0]['generated_text'])

But it is not working (see here.

Any hint would be appreciated.
Thank you.