[Frontend] Write run-batch responses incrementally instead of at the end

examples/features/openai_batch/README.md

@@ -13,7 +13,10 @@ The OpenAI batch file format consists of a series of json objects on new lines.
 Each line represents a separate request. See the [OpenAI package reference](https://platform.openai.com/docs/api-reference/batch/requestInput) for more details.
 
 ```{note}
-We currently support `/v1/chat/completions`, `/v1/embeddings`, and `/v1/score` endpoints (completions coming soon).
+We currently support `/v1/chat/completions`, `/v1/embeddings`, `/score`, `/rerank`,
+`/v1/audio/transcriptions`, and `/v1/audio/translations` endpoints (`/v1/completions`
+coming soon). Score and rerank are also served under a version prefix, such as
+`/v1/score`.
 ```
 
 ## Pre-requisites
@@ -67,6 +70,10 @@ vllm run-batch \
 
 You should now have your results at `results.jsonl`. You can check your results by running `cat results.jsonl`
 
+Responses are written as each request finishes, so their order does not follow
+the input file. Match a response to its request with `custom_id` rather than by
+line number.
+
 ```bash
 cat results.jsonl
 {"id":"vllm-383d1c59835645aeb2e07d004d62a826","custom_id":"request-1","response":{"id":"cmpl-61c020e54b964d5a98fa7527bfcdd378","object":"chat.completion","created":1715633336,"model":"meta-llama/Meta-Llama-3-8B-Instruct","choices":[{"index":0,"message":{"role":"assistant","content":"Hello! It's great to meet you! I'm here to help with any questions or tasks you may have. What's on your mind today?"},"logprobs":null,"finish_reason":"stop","stop_reason":null}],"usage":{"prompt_tokens":25,"total_tokens":56,"completion_tokens":31}},"error":null}