Add unit tests for batched guided and non-guided requests

[sarckk]

Purpose

Context: A bug was introduced in #21862 that causes non-guided requests to output incorrect values when batched together with guided requests. See #22896 for technical details of this bug. This bug is fixed by either #22896 or #22963.

This PR adds a unit test that would've caught this bug.

Test Plan

pytest tests/v1/entrypoints/llm/test_struct_output_generate.py::test_structured_output_batched_with_non_guided_requests

Test Result

Trunk, test passes:

Prompt:
"Give an example JSON for an employee profile that fits this schema. Make the response as short as possible. Schema: {'type': 'object', 'properties': {'name': {'type': 'string'}, 'age': {'type': 'integer'}, 'skills': {'type': 'array', 'items': {'type': 'string'}}, 'grade': {'type': 'string', 'pattern': '^[A-D]$'}, 'email': {'type': 'string', 'pattern': '^[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\\\\.[a-zA-Z]{2,}$'}, 'work_history': {'type': 'array', 'items': {'type': 'object', 'properties': {'company': {'type': 'string'}, 'duration': {'type': 'number', 'minimum': 0.0, 'maximum': 100.0}, 'position': {'type': 'string'}}, 'required': ['company', 'duration', 'position'], 'additionalProperties': False}, 'minItems': 0, 'maxItems': 3}}, 'required': ['name', 'age', 'skills', 'grade', 'email', 'work_history'], 'additionalProperties': False}"
Generated text:
'{"name": "John", "age": 30, "skills": ["python", "javascript", "react"], "grade": "A", "email": "john@example.com", "work_history": [{"company": "ABC", "duration": 5, "position": "Software Engineer"}, {"company": "XYZ", "duration": 3, "position": "DevOps Engineer"}, {"company": "PQR", "duration": 2, "position": "SDE-1"}]}'
Prompt:
'The diameter of the Earth in kilometers is '
Generated text:
'12,742. The radius of the Earth is half of the diameter. What'

After undoing fixes in both #22896 and #22963 (outlines backend seems fine).

FAILED tests/v1/entrypoints/llm/test_struct_output_generate.py::test_structured_output_batched_with_non_guided_requests[guidance] - AssertionError: assert '12,742' in '12!!!!!!!!!!!!!!!'
FAILED tests/v1/entrypoints/llm/test_struct_output_generate.py::test_structured_output_batched_with_non_guided_requests[xgrammar] - AssertionError: assert '12,742' in '12!!!!!!!!!!!!!!!'

(Optional) Documentation Update

---

Essential Elements of an Effective PR Description Checklist

• The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
• The test plan, such as providing test command.
• The test results, such as pasting the results comparison before and after, or e2e results
• (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.

[gemini-code-assist]

Code Review

This pull request adds a valuable unit test to catch a bug related to batching guided and non-guided requests. The test case is well-structured and directly addresses the scenario described. My main feedback is a critical improvement to make the test more robust by ensuring proper resource cleanup, which will prevent potential test flakiness in the future.

[sarckk]

@mgoin @russellb @JartX follow up to #22896

[mgoin]

Looks reasonable to me, let's see if CI green. Thanks!

[sarckk]

@mgoin the newly added test passed in ci/pr/v1-test. A single unrelated test failed with error:

huggingface_hub.errors.HfHubHTTPError: 504 Server Error: Gateway Timeout for url: https://huggingface.co/api/models/google/gemma-2-2b-it/tree/main?recursive=False&expand=False (Request ID: Root=1-68a7c260-6f4f44cc3173e43632f8da4e;8e4182ef-b4a0-4bbd-87a2-2457f179fb11)