Repository navigation
Conversation
semif-score --backend sglang scores the direct, serial, shared, and reranker modes through the /v1/score endpoint of a running SGLang server that contains sgl-project/sglang#40826. SemIf still renders every prompt, checks the answer slots, and validates the input on the pinned reference tokenizer, so prompt_sha256, input_tokens, prompt_version, option order, and answer_token_ids equal the Torch values. The server only scores SemIf's token ids, with each row's answer slots as that item's candidates and return_token_logprobs, and the client uses the standard library, so there is no new dependency. Before the first row, the backend refuses an unreachable server, a server without per-item candidate scoring, another model or revision, a Hub model whose server reports no revision, a non-generation model, --load-format dummy, --enable-mis, --allow-auto-truncate, any --preferred-sampling-params, a server tokenizer that disagrees with the reference tokenizer, and a --max-tokens at or above the server's max_req_input_len. Every response must carry one finite log-probability per candidate and a usage.prompt_tokens equal to the token ids sent. Requests go straight to the server with no proxy, redirect, or retry, and SEMIF_SGLANG_API_KEY is sent only as a bearer header and never written to a row. option_logits hold the candidates' full-vocabulary log-probabilities, one constant per row away from raw logits, as the readout string says. cli.py now binds a reranker scorer per backend, so reranker mode reaches the SGLang scorer, and --sglang-url and --sglang-timeout require --backend sglang. docs/SGLANG.md covers the server launch and the refused settings, and the real-server tests read SEMIF_SGLANG_URL and SEMIF_SGLANG_RERANKER_URL.
tvpavan
added a commit
to tvpavan/SemIf-OpenJev
that referenced
this pull request
Oct 2, 2026
Merge picks: upstream PRs TheoLeeCJ#34 TheoLeeCJ#41 TheoLeeCJ#42 TheoLeeCJ#47 TheoLeeCJ#55 + lazy option permutations
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
sgl-project/sglang#40826 added per-item candidate scoring to SGLang's
/v1/score. This adds an SGLang backend so SemIf can score rows on an SGLang server.Modifications
--backend sglangfor the direct, serial, shared, and reranker modes insrc/semif_phase1/sglang_backend.py. SemIf still renders and validates prompts, and the server only scores SemIf's token ids.--sglang-url,--sglang-timeout,docs/SGLANG.md, and tests.Related Issues
Requires an SGLang build with sgl-project/sglang#40826, which is SGLang main or a nightly from
0.5.21.dev20260925on.Accuracy Test