fix: deterministic MCP tool and property ordering for prompt caching

[ivanetchart]

Summary

MCP tools backed by Go maps have non-deterministic iteration order, causing the tool list and property keys to shuffle across requests. This breaks prefix-based prompt caching (Anthropic, OpenAI) since tool definitions form the first level of the cache prefix.

Changes

• Sort GetAvailableTools() result by function name (core/mcp/toolmanager.go)
• Sort MCP tool property keys via OrderedMapFromMap default sorting (core/mcp/utils.go)
• Make OrderedMapFromMap sort keys lexicographically by default, with an optional sortFn override for custom ordering (core/schemas/orderedmap.go)

Type of change

• Bug fix
• Feature
• Refactor
• Documentation
• Chore/CI

Affected areas

• Core (Go)
• Transports (HTTP)
• Providers/Integrations
• Plugins
• UI (Next.js)
• Docs

How to test

cd core
go test ./schemas/... ./mcp/... -count=1 -v

Key tests:

• TestOrderedMapFromMap_DefaultSortsKeys — verifies default lexicographic sorting
• TestOrderedMapFromMap_CustomSortFn — verifies custom sort override
• Existing TestOrderedMap* tests verify no regressions in insertion-order preservation

Screenshots/Recordings

N/A

Breaking changes

• Yes
• No

OrderedMapFromMap now sorts keys by default. Existing callers that relied on non-deterministic (random) ordering will now get sorted output, which is strictly more deterministic. The variadic sortFn parameter is backward-compatible — no call sites need changes.

Related issues

Closes #2347

Related upstream context:

• [Bug]: MergeExtraParams produces non-deterministic JSON key ordering, breaking Anthropic prompt caching #1742 — MergeExtraParams produces non-deterministic JSON key ordering
• fix(anthropic): deterministic tool schema serialization for prompt caching #2082 — deterministic tool schema serialization for prompt caching
• fix: sort tools alphabetically while generating hash #522 — sort tools alphabetically while generating hash (still open)

Security considerations

None.

Checklist

• I read docs/contributing/README.md and followed the guidelines
• I added/updated tests where appropriate
• I updated documentation where needed
• I verified builds succeed (Go and UI)
• I verified the CI pipeline passes locally if applicable

[CLAassistant]

All committers have signed the CLA.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: bc4d3303-a3a6-44c8-8720-739d4f435b6d

📥 Commits

Reviewing files that changed from the base of the PR and between 2774218 and e397828.

📒 Files selected for processing (5)

• core/changelog.md
• core/mcp/toolmanager.go
• core/mcp/utils.go
• core/schemas/orderedmap.go
• core/schemas/orderedmap_test.go

---

📝 Walkthrough

Summary by CodeRabbit

• Bug Fixes

  • Ensured deterministic ordering of MCP tool output and properties for consistent behavior.

• Tests

  • Added test coverage for ordered data structure key sorting to validate consistency.

Walkthrough

The changes implement deterministic ordering of MCP tools and their properties to support prefix-based prompt caching. ToolsManager.GetAvailableTools() now sorts tools by function name, OrderedMapFromMap() applies lexicographic sorting of keys by default with optional custom comparators, and convertMCPToolToBifrostSchema() uses this functionality for stable property serialization.

Changes

Cohort / File(s)	Summary
Tool Manager Sorting core/mcp/toolmanager.go	Added chatToolName() helper function to safely extract tool names and implemented deterministic sorting of availableTools slice before return, ordering by function name.
OrderedMap Deterministic Ordering core/schemas/orderedmap.go, core/schemas/orderedmap_test.go	Enhanced OrderedMapFromMap() signature to accept optional variadic sortFn parameter. Function now sorts keys lexicographically by default (or via custom comparator if provided). Added two test cases validating lexicographic and custom sort ordering during JSON marshaling.
Schema Conversion Updates core/mcp/utils.go	Refactored convertMCPToolToBifrostSchema() to use OrderedMapFromMap() directly instead of manual map iteration with capacity reservation. Updated function comment to document lexicographic sorting of properties for stable JSON serialization.
Changelog core/changelog.md	Added changelog entry documenting the deterministic ordering fix for MCP tool output and properties to support prompt caching.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Poem

🐰 hops excitedly
Tools once danced in random ways,
Breaking caches day by day,
Now they sort in proper line—
Lexicographic, how divine!
Cache hits bloom where chaos died,
Determinism is our guide! ✨

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Title check	✅ Passed	The PR title clearly and concisely summarizes the main change: deterministic ordering of MCP tools and properties to enable prompt caching.
Description check	✅ Passed	The PR description follows the template structure with all major sections completed: Summary, Changes, Type of change, Affected areas, How to test, Breaking changes, Related issues, Security considerations, and Checklist.
Linked Issues check	✅ Passed	All code changes directly implement the objectives from issue #2347: sorting GetAvailableTools by function name, sorting MCP tool properties via OrderedMapFromMap, and implementing lexicographic key sorting with optional override.
Out of Scope Changes check	✅ Passed	All changes are scoped to the objectives of issue #2347: toolmanager.go sorts tools, utils.go applies OrderedMapFromMap sorting, orderedmap.go implements deterministic sorting, and orderedmap_test.go adds validation tests.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[greptile-apps]

Confidence Score: 5/5

Safe to merge — the fix is correct, backward-compatible, and well-tested; all remaining findings are minor style nits.

No P0 or P1 issues found. The core logic (sorting in OrderedMapFromMap and GetAvailableTools) is straightforward and correct. The variadic API is backward-compatible and the new tests cover both code paths. The only observations are a redundant double-sort in pre-existing SortKeys/SortedCopy paths (harmless) and a defensive-comment suggestion for chatToolName. Neither affects correctness or production reliability.

No files require special attention.

Important Files Changed

Filename	Overview
core/schemas/orderedmap.go	Adds variadic sortFn to OrderedMapFromMap; defaults to sort.Strings for deterministic key order. Backward-compatible and correct. Minor side effect: existing callers that pass the result directly to SortKeys()/SortedCopy() now sort twice (once lexicographically, once by schema priority), which is harmless but slightly wasteful.
core/mcp/toolmanager.go	Adds sort.Slice on availableTools keyed by function name and a helper chatToolName. Cleanly resolves non-deterministic tool ordering from Go map iteration. Logic is correct; empty-name tools sort to the front (stable).
core/mcp/utils.go	Replaces manual map-copy loop with OrderedMapFromMap, which now provides lexicographic sorting by default. FixArraySchemas still runs first. Functionally equivalent to old code but now guarantees stable property ordering.
core/schemas/orderedmap_test.go	Adds TestOrderedMapFromMap_DefaultSortsKeys and TestOrderedMapFromMap_CustomSortFn to verify both default and custom sort paths. Tests are clear and sufficient.
core/changelog.md	Changelog entry added for this fix. No issues.

Comments Outside Diff (1)

1. core/schemas/orderedmap.go, line 68-76 (link)

   Redundant sort in SortKeys/SortedCopy call-sites

   OrderedMapFromMap now always sorts keys lexicographically, but every existing caller inside this file (SortKeys, SortedCopy, SortedCopyPreservingProperties, preserveKeysOrderedCopyWithAwareness) immediately re-sorts the result with the JSON Schema priority ordering. For example:

   // SortKeys – line ~353
   case map[string]interface{}:
       converted := OrderedMapFromMap(nested)  // ① sorts lexicographically
       converted.SortKeys()                     // ② sorts again with priority order
       om.values[k] = converted

   The final output is correct (priority ordering wins), but the first sort.Strings pass is discarded work. For large tool schemas this is hit once per nested map[string]interface{} value. Consider passing a no-op sort function at those call sites, or exposing a private orderedMapFromMapUnsorted helper, so the schema-priority sort is the only pass. This is a minor performance nit with no correctness impact.

_{Reviews (1): Last reviewed commit: "[fix]: deterministic MCP tool and proper..." | Re-trigger Greptile}

[greptile-apps]

Tools with nil Function sort to the front together

chatToolName returns "" when Function is nil, so any such tools will cluster at the front of the sorted slice in an undefined relative order. In practice the inner loop already guards against nil / empty-name tools before appending, so availableTools should never contain them by the time the sort runs. This is safe today, but a brief comment (or an assert in tests) noting that invariant would make the code easier to audit in future.

This is a documentation/hardening nit — no change to logic needed.

[bilby91]

Would be nice to get this merged. We really need it :)

[akshaydeo]

This is already fixed so closing the PR