[Structural Output] Get prepared for the upcoming support for XGrammar's latest structural tag format#10497
[Structural Output] Get prepared for the upcoming support for XGrammar's latest structural tag format#10497aaronyeeio wants to merge 2 commits intosgl-project:mainfrom
Conversation
…tag format in documentation and backend. Added new code examples in the Jupyter notebook and updated the backend to handle both new and deprecated structural tag formats. Introduced a new response format class for compatibility with the updated API.
aaronyeeio
requested review from
CatherineSue,
hnyls2002,
ispobock,
merrymercy and
slin1237
as code owners
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Summary of Changes
Hello @aaronyeeio, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!
This pull request prepares the system for the upcoming changes in XGrammar's structural tag format. It ensures that the backend can process both the new V1 format and the older deprecated format, maintaining backward compatibility while embracing the latest standards. Additionally, the documentation has been updated with practical examples to guide users on integrating the new format into their applications.
Highlights
- XGrammar Structural Tag Format Support: Introduced comprehensive support for XGrammar's latest structural tag format, aligning with upcoming changes in the XGrammar library.
- Backward Compatibility: The backend now intelligently handles both the new V1 structural tag format and the previously deprecated format, ensuring seamless transitions for existing implementations.
- Documentation Updates: New code examples have been added to the Jupyter notebook documentation, illustrating how to effectively use the updated structural tag format across different API entry points.
- API Protocol Enhancement: A new Pydantic model,
StructuralTagResponseFormatV1, has been added to the API protocol to formally define and support the new structural tag format.
Using Gemini Code Assist
The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.
Invoking Gemini
You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.
| Feature | Command | Description |
|---|---|---|
| Code Review | /gemini review |
Performs a code review for the current pull request in its current state. |
| Pull Request Summary | /gemini summary |
Provides a summary of the current pull request in its current state. |
| Comment | @gemini-code-assist | Responds in comments when explicitly tagged, both in pull request comments and review comments. |
| Help | /gemini help |
Displays a list of available commands. |
Customization
To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.
Limitations & Feedback
Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.
You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.
Footnotes
-
Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩
There was a problem hiding this comment.
Code Review
This pull request adds support for the latest XGrammar structural tag format. The changes include updating the documentation with new examples and modifying the backend to handle both the new and deprecated formats for backward compatibility. The logic in xgrammar_backend.py correctly distinguishes between the two formats, and the Pydantic models in protocol.py are appropriately updated. My main feedback is on the documentation notebook, where the new structural tag format dictionary is duplicated, which could be refactored for better maintainability. Overall, the changes are well-implemented and achieve the stated goal.
| " \"format\": {\n", | ||
| " \"type\": \"triggered_tags\",\n", | ||
| " \"triggers\": [\"<function=\"],\n", | ||
| " \"tags\": [\n", | ||
| " {\n", | ||
| " \"begin\": \"<function=get_current_weather>\",\n", | ||
| " \"content\": schema_get_current_weather,\n", | ||
| " \"end\": \"</function>\",\n", | ||
| " },\n", | ||
| " {\n", | ||
| " \"begin\": \"<function=get_current_date>\",\n", | ||
| " \"content\": schema_get_current_date,\n", | ||
| " \"end\": \"</function>\",\n", | ||
| " },\n", | ||
| " ],\n", | ||
| " \"at_least_one\": False,\n", | ||
| " \"stop_after_first\": False,\n", | ||
| " }\n", |
There was a problem hiding this comment.
The dictionary for the format key is duplicated in three different cells in this notebook. To improve the maintainability of this documentation, consider defining this dictionary as a variable in a preceding cell and reusing it in all three examples. This would make future updates to the format much easier to manage.
|
|
||
| class StructuralTagResponseFormatV1(BaseModel): | ||
| type: Literal["structural_tag"] | ||
| format: Dict[str, Any] |
There was a problem hiding this comment.
You can just import StructuralTag from xgrammar, that's also a pydantic BaseModel and all its fields are type checked.
Co-authored-by: Aaron Yee <110548922+aaronyeeio@users.noreply.github.com>
merrymercy
requested review from
DarkSharpness and
JustinTong0323
as code owners
|
This PR has been partly combined into #10691 and merged into main. Thanks for the contribution! Feel free to reopen if anything is missing ) |
3 participants
Motivation
Enhance support for XGrammar's latest structural tag format in documentation and backend. Added new code examples in the Jupyter notebook and updated the backend to handle both new and deprecated structural tag formats. Introduced a new response format class for compatibility with the updated API.
See: mlc-ai/xgrammar#420
Modifications
Modified:
protocol.pyandxgrammar_backend.pyfor compatibility through all entrypoints.Docs also updated. Check docs for the usage.
Accuracy Tests
Currently XGrammar has not published a new version with this PR yet and I believe they soon will. So please test it with the latest build of XGrammar.
Benchmarking and Profiling
Checklist