Gather static input buffers for cuda graph

[cctry]

Motivation

To combat with data race between scheduler and model runner, the effective approach iis to fully decouple the two components by staging all dynamic model inputs in pre-allocated static buffers. This transforms the interaction between scheduler and model runner into SPSC pattern.

As the first step, this PR is to organize the static buffers needed for forward.
Note that this is not the complete list. Some tensors such as req_to_token are static and captured in graph.

Modifications

Accuracy Tests

Benchmarking and Profiling

Checklist

• Format your code according to the Format code with pre-commit.
• Add unit tests according to the Run and add unit tests.
• Update documentation according to Write documentations.
• Provide accuracy and speed benchmark results according to Test the accuracy and Benchmark the speed.
• Follow the SGLang code style guidance.
• Work with maintainers to merge your PR. See the PR Merge Process