Skip to content

UPSTREAM PR #17947: scripts: add script to compare logits of llama.cpp against other frameworks - #529

Open
loci-dev wants to merge 8 commits into
mainfrom
upstream-PR17947-branch_ngxson-xsn/compare_logits
Open

UPSTREAM PR #17947: scripts: add script to compare logits of llama.cpp against other frameworks#529
loci-dev wants to merge 8 commits into
mainfrom
upstream-PR17947-branch_ngxson-xsn/compare_logits

Conversation

@loci-dev

Copy link
Copy Markdown

Mirrored from ggml-org/llama.cpp#17947

Compare logits between llama.cpp and another inference engine.

Example usage:
    Step 1: Dump logits from two different endpoints
        python scripts/compare-logits.py dump logits_llama.log http://localhost:8080/completions
        python scripts/compare-logits.py dump logits_other.log http://other-engine:8000/completions

        (optionally, you can add --api-key <key> if the endpoint requires authentication)

    Step 2: Compare the dumped logits
        python scripts/compare-logits.py compare logits_llama.log logits_other.log report.md

@loci-review

loci-review Bot commented Dec 12, 2025

Copy link
Copy Markdown

Explore the complete analysis inside the Version Insights

@loci-dev
loci-dev force-pushed the main branch 19 times, most recently from c05b224 to e70bc15 Compare December 14, 2025 08:10
@loci-dev
loci-dev force-pushed the main branch 30 times, most recently from 9f1f66d to ec69147 Compare December 19, 2025 12:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants