[Spec Decode][Benchmark] Add Blitzedit dataset

[ekagra-ranjan]

I have been looking for datasets where Ngram is better than Eagle for exploring the idea of combining Ngram and EAGLE #18633. InstructCoder being an editing task was the go to dataset in vLLM for Ngram until I found that fixing the prompt made EAGLE quite strong and better than Ngram on InstructCoder dataset. An ideal dataset would be the one where the overlap bw input and output are high. Blazedit dataset is a promising one since it can allow observing AL of Ngram over different input-output overlap.

• This PR add Blazedit dataset.
• Compared to InstructCode dataset which we used for Ngram, it is a longer dataset and has associated each data with the normalized Levenshtein distance [0.0, 1.0] which can help with observing the gain of Ngram wrt to the overlap between input and output
• Source
  • Blog: https://huggingface.co/blog/ganler/blazedit
  • HF dataset: https://huggingface.co/vdaita
  • available in 2 versions: 5k char (vdaita/edit_5k_char) and 10k char (vdaita/edit_10k_char)
• This needs model which can support >3k seq len for the 5k char variant of dataset. llama 3.1 8b only supports 2048 so couldn't run this dataset completely but will be useful for models which have longer seq len.

Sample Cmd:
time VLLM_USE_V1=1 python3 examples/offline_inference/spec_decode.py --method eagle --num_spec_tokens 3 --tp 1 --dataset-name hf --dataset-path vdaita/edit_5k_char --num-prompts 90 --hf-output-len 2048 --blazedit-min-distance 0.01 --blazedit-max-distance 0.99 --print-output

[gemini-code-assist]

Code Review

This PR adds support for the Blitzedit dataset for benchmarking. The changes correctly add command-line arguments, integrate the new dataset class into the factory function, and implement the dataset loading and sampling logic. My review focuses on cleaning up some leftover debugging code and unused variables in the new BlazeditDataset implementation to improve code quality and ensure the dataset is not unnecessarily filtered.

[mergify]

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @ekagra-ranjan.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

[LiuXiaoxuanPKU]

Can you share some numbers of InstructCoder if any?

[ekagra-ranjan]

I have some numbers here: #18971