Conversation
zjin1126
force-pushed
the
feat/provider-prompt-cache-policy
branch
from
August 5, 2026 19:16
9d51af3 to
0cbc2af
Compare
This was referenced Aug 5, 2026
Author
|
Concrete external consumer Draft: oliver-mee/hermes-alibaba-token-plan#18 It implements the proposed |
zjin1126
added a commit
to zjin1126/hermes-alibaba-token-plan
that referenced
this pull request
Aug 5, 2026
Use the optional ProviderProfile prompt-cache hook proposed in NousResearch/hermes-agent#79621. Scope the OpenAI-wire cache marker declaration to qwen3.8-max-preview on Chat Completions and leave every other model and API mode on Hermes' existing fallback. The QwenCloud context-cache guide lists qwen3.8-max-preview (Token Plan only) as explicit-cache capable and documents cache_control markers for the OpenAI-compatible Chat API. Standalone and supported-Hermes compatibility suites pass without network access. AI-assisted: Hermes Agent (GPT-5.6 Sol); the author reviewed the implementation, evidence, diff, and test results.
13 tasks
This was referenced Aug 13, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Adds a narrow
ProviderProfile.prompt_cache_policy()hook so model-provider plugins can declare request-scoped explicit prompt-cache support without adding plugin IDs or endpoint allowlists to Hermes core.The hook returns
(enabled, native_layout)orNone.None, invalid values, and exceptions preserve the existing core detection path. An explicit operator cache disable still wins, and MoA still resolves the acting aggregator before consulting its provider profile.This extends the existing provider-profile abstraction rather than adding a second strategy manager or any core model-tool surface. The system prompt and conversation cache prefix are unchanged.
Prior art checked:
ProviderProfileextension point.Related Issue
Fixes #79602
Type of Change
Changes Made
providers/base.py: add the optional, fail-safeprompt_cache_policy()provider hook and document its contract.agent/agent_runtime_helpers.py: resolve plugin policy after operator-disable and MoA-provider resolution, before legacy core rules.tests/providers/test_provider_profiles.py: add the default contract and a fresh-process$HERMES_HOMEprovider-plugin discovery test.tests/run_agent/test_anthropic_prompt_cache_policy.py: cover request context, legacy fallback, malformed returns, exceptions, and operator-disable precedence.website/docs/developer-guide/model-provider-plugin.md: document the new hook for external provider authors.How to Test
Local results:
git diff --check: passed.[all,dev]environment: 25,618 passed, 69 failed in 27 untouched optional-dependency/platform-specific files. Representative failures require non-[all]lazy dependencies such as the Anthropic SDK, Parallel SDK, wake-word runtime, and audio backends. The changed policy/discovery tests passed in that run; CI remains authoritative for the repository-wide matrix.Checklist
Code
fix(scope):,feat(scope):, etc.)pytest tests/ -qand all tests passDocumentation & Housekeeping
docs/, docstrings) —ProviderProfiledocstring and model-provider authoring guidecli-config.yaml.exampleif I added/changed config keys — N/A, no config keysCONTRIBUTING.mdorAGENTS.mdif I changed architecture or workflows — N/A, this extends the existingProviderProfilehook surfaceScreenshots / Logs
No UI changes.
AI assistance was used to help draft and review the implementation and tests; all commands and validation reported above were run in the contributor checkout.