fix: Add support for single element arrays for chat and completions prompts

[zhongdaor-nv]

Overview:

1. Remove batch_token_ids — unused across all engines and currently non-functional.
2. Handle single element prompt arrays by passing them directly to token_ids.
3. Added tests of different prompt type for /completions

Details:

Where should the reviewer start?

Related Issues: (use one of the action keywords Closes / Fixes / Resolves / Relates to)

• closes GitHub issue: #xxx

Summary by CodeRabbit

• Refactor
  • Simplified request preprocessing by removing unused batch token storage and streamlining single-input tokenization.
• Bug Fixes
  • Multi-prompt (batch) completion requests now return a clear error instead of attempting unsupported processing, improving consistency and transparency.
• Tests
  • Added end-to-end tests for completion behavior with single-string, single-element array, and multi-element array prompts, asserting success for supported cases and error for unsupported multi-prompt inputs.

[zhongdaor-nv]

make engine works when prompt is a array of string

[zhongdaor-nv]

This is just a simple fix. #1648
is still open, but no engine is currently using batch_token_ids