fix: broaden llama.cpp grammar error classifier to match newer builds - #51527
Closed
gonzalofrancoceballos wants to merge 2 commits into
Closed
Conversation
The error message varies across llama.cpp versions: - Older: "error parsing grammar: unknown escape at \\d" - Newer: "Failed to initialize samplers: failed to parse grammar" Only the older phrasing was recognized, causing Hermes to fall back to cloud providers instead of stripping pattern/format keywords and retrying locally. Added "failed to parse grammar" to the match list and updated the classifier comment to document the newer variant. Test: added test_llama_cpp_failed_to_parse_grammar covering the new phrase.
gonzalofrancoceballos
force-pushed
the
fix/llama-cpp-grammar-classifier-broader-match
branch
from
June 30, 2026 08:31
035bce3 to
930d0f6
Compare
…-classifier-broader-match
gonzalofrancoceballos
deleted the
fix/llama-cpp-grammar-classifier-broader-match
branch
July 1, 2026 13:31
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The llama.cpp grammar error classifier in
agent/error_classifier.pyonly recognizes older error phrasing ("error parsing grammar"), but newer llama.cpp builds return"Failed to initialize samplers: failed to parse grammar". When the classifier fails to match, Hermes skips the grammar recovery path (strippingpattern/formatkeywords from tool schemas and retrying locally) and falls directly to a cloud fallback provider.This is especially impactful for users running local inference via custom providers (llama.cpp) with MCP servers whose tool schemas include regex patterns like
"\\d{4}-\\d{2}-\\d{2}".Reproduction
In Hermes, this error passes through the classifier unrecognized, triggering cloud fallback instead of the schema-stripping recovery.
Fix
Added
"failed to parse grammar"to the match list in the classifier condition, alongside the existing"error parsing grammar"and"unable to generate parser"checks.Changes
agent/error_classifier.py: Added"failed to parse grammar" in error_msgconditiontests/agent/test_error_classifier.py: Addedtest_llama_cpp_failed_to_parse_grammartest caseTesting