[Benchmark] Add --use_random_tokens flag to C benchmark

[Vishal Jain (VishalX)]

Summary

Adds a --use_random_tokens flag to the C benchmark that feeds randomly.

• Generated token IDs (uniform in [0, 99]) directly into the prompt sequences, bypassing tokenization entirely.
• This is useful for isolating model/runtime performance from tokenizer overhead and for benchmarking without requiring
  a valid text prompt for the target model.
• Similar to benchmark_e2e.py

  parser.add_argument(
        "--use_random_tokens", action="store_true", help="Use random tokens instead of generating a prompt"
    )

Usage

model_benchmark -i <model-dir> --use_random_tokens -l 512 -g 128 -r 2 -w 2

Constraints

• Requires -l/--prompt_length (token count must be known upfront)
• Mutually exclusive with --prompt and --prompt_file

[Copilot]

Pull request overview

Adds a --use_random_tokens mode to the C++ model_benchmark tool so it can benchmark model/runtime performance without tokenizing a real prompt (by directly feeding synthetic token IDs).

Changes:

• Introduces a --use_random_tokens CLI flag and corresponding Options::use_random_tokens setting.
• Adds validation and help text describing constraints for --use_random_tokens.
• Updates the benchmark to bypass tokenization and append randomly generated token IDs into OgaSequences.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.

File	Description
benchmark/c/options.h	Adds use_random_tokens to the benchmark options struct.
benchmark/c/options.cpp	Documents/parses --use_random_tokens and adds basic option validation.
benchmark/c/main.cpp	Generates random token sequences and bypasses tokenizer encoding when enabled.
.github/copilot-instructions.md	Fixes a documentation path (src/runtime_settings.cpp).

Comments suppressed due to low confidence (1)

benchmark/c/options.cpp:105

• --use_random_tokens validation is order-dependent because prompt_num_tokens_or_content is overwritten during parsing. E.g., --prompt "hi" -l 16 --use_random_tokens will pass verification and silently ignore --prompt, even though the help/PR description say --use_random_tokens cannot be used with --prompt/--prompt_file. Track which prompt source flags were provided (or count prompt options) and fail if more than one is specified, regardless of argument order.

  if (opts.use_random_tokens && !std::holds_alternative<size_t>(opts.prompt_num_tokens_or_content)) {
    throw std::runtime_error(
        "--use_random_tokens requires -l/--prompt_length and cannot be used with --prompt or --prompt_file.");
  }

[Vishal Jain (VishalX)]

kunal-vaishnavi / Baiju Meswani (@baijumeswani) pls review this, added a new option in c/benchmark similar to python variant.

[Copilot]

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.

[Vishal Jain (VishalX)]

pls consider this for 0.14.0 if possible.