Skip to content

Simplify dialects and reuse SDK inference transport - #2675

Closed
xeophon wants to merge 1 commit into
mainfrom
codex/sdk-backed-dialects
Closed

xeophon wants to merge 1 commit into
mainfrom
codex/sdk-backed-dialects

Conversation

@xeophon

@xeophon xeophon commented Sep 24, 2026 •

Copy link
Copy Markdown
Member

Overview

Use the provider SDKs for inference transport and stream assembly, with dialects projecting native requests and responses into Verifiers traces. Keep evaluation and training clients separate and retain the interception server's rollout, cancellation, and replay ownership.

Details

  • Share sampling overrides and Chat stream accumulation, while preserving each provider's authentication, terminal events, usage extensions, error statuses, and reasoning state.
  • Preserve native content and provider metadata through task-hook edits using per-message setters and per-part provenance.
  • Keep network capability mediation separate from wire-format translation, with conservative handling of provider domain filters.

Note

Rework dialects around native dict responses, setters, and SDK transport

Simplifies the Chat, Responses, and Anthropic dialects by removing per-dialect SSE parsers, Pydantic response models, and local capability mediation. Responses are now passed around as native dictionaries, streamed completions are assembled through shared SDK readers (read_chat_completion, AsyncMessageStream), and request edits go through setter callbacks returned by parse_request.

  • The eval client now sends requests through OpenAI and Anthropic SDK clients instead of a hand-rolled HTTPX relay; provider auth comes from the selected SDK, and error status mapping covers Anthropic, httpx, and connection-reset failures in errors.py.
  • Adds NativeContentPart so unrecognized provider content blocks are preserved instead of dropped, and exports it from verifiers.v1 in types.py.
  • Network-policy request mediation moves from dialect methods to a reusable RequestFilter base in policy.py.
  • Risk: parse_request now returns a (Request, setters) tuple instead of just a Request, so all callers (e.g. train.py, taskset.py, server.py) must unpack it; StreamParser and the removed dialect filter classes are no longer importable from verifiers.v1.dialects.

Macroscope summarized 7022952.

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Sep 24, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-22T22:43:54.431930Z 7022952 New commits
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

Comment thread verifiers/v1/policy.py Resolved

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 97fbff7ea6

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread verifiers/v1/clients/eval.py Resolved

@cursor cursor Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale Bugbot comment from a previous run.

Comment thread verifiers/v1/clients/eval.py Resolved
@macroscopeapp

macroscopeapp Bot commented Sep 24, 2026 •

Copy link
Copy Markdown
Contributor

Approvability

Verdict: Needs Human Approval

Macroscope's review found this PR requires human approval — This PR replaces the inference relay with SDK-backed OpenAI/Anthropic transports and rewires streaming, request rewriting, authentication filtering, and error handling. It also introduces a substantial network-policy mediation module affecting provider tools and remote content, creating broad production and security-sensitive impact.

You can add or adjust custom eligibility rules. Learn more.

@macroscopeapp

macroscopeapp Bot commented Sep 24, 2026 •

Copy link
Copy Markdown
Contributor

Macroscope has since reviewed this pull request. An earlier review was skipped by a cost limit; a review has now completed, so that notice no longer applies.

@xeophon xeophon changed the title refactor(dialects): reuse provider SDKs for evaluation Simplify dialects and reuse SDK inference transport Sep 24, 2026
@xeophon
xeophon changed the base branch from main to refactor/interception-server September 24, 2026 11:41
Comment thread verifiers/v1/dialects/chat.py Resolved
Comment thread verifiers/v1/dialects/base.py Resolved
Comment thread verifiers/v1/dialects/chat.py Resolved
@xeophon
xeophon changed the base branch from refactor/interception-server to main September 24, 2026 11:47
@xeophon
xeophon force-pushed the codex/sdk-backed-dialects branch from adf6626 to 60c20f9 Compare September 24, 2026 11:47
@xeophon
xeophon changed the base branch from main to refactor/interception-server September 24, 2026 11:48

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 60c20f9074

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread verifiers/v1/dialects/responses.py Resolved
Comment thread verifiers/v1/dialects/anthropic.py Resolved
@xeophon
xeophon force-pushed the codex/sdk-backed-dialects branch from 60c20f9 to 99b2ebe Compare September 24, 2026 13:20
Comment thread verifiers/v1/dialects/anthropic.py Resolved
Comment thread verifiers/v1/session.py Resolved

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 99b2ebee17

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread verifiers/v1/dialects/chat.py Resolved

@cursor cursor Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale Bugbot comment from a previous run.

Comment thread verifiers/v1/clients/eval.py Resolved
Comment thread verifiers/v1/dialects/responses.py Resolved
@xeophon
xeophon force-pushed the codex/sdk-backed-dialects branch from 99b2ebe to 5d754d2 Compare September 28, 2026 07:43
@xeophon
xeophon force-pushed the refactor/interception-server branch from 0239711 to 2b6af86 Compare September 28, 2026 07:43

@cursor cursor Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale Bugbot comment from a previous run.

Comment thread verifiers/v1/dialects/chat.py Resolved
@xeophon
xeophon force-pushed the refactor/interception-server branch from fe4157c to 575cf75 Compare September 29, 2026 08:19
@xeophon
xeophon force-pushed the codex/sdk-backed-dialects branch from 5d754d2 to 899cbf2 Compare September 29, 2026 08:19

@cursor cursor Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale Bugbot comment from a previous run.

Comment thread verifiers/v1/types.py Resolved
@xeophon
xeophon force-pushed the refactor/interception-server branch from 575cf75 to 3bac8cd Compare September 30, 2026 15:42
@xeophon
xeophon force-pushed the codex/sdk-backed-dialects branch from 899cbf2 to 9ef4500 Compare September 30, 2026 15:46
@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, add credits to your account and enable them for code reviews in your settings.

@cursor cursor Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale Bugbot comment from a previous run.

Comment thread verifiers/v1/utils/chat.py Resolved
Comment thread verifiers/v1/dialects/base.py Resolved
@xeophon
xeophon added this pull request to stack #2730 October 1, 2026 01:33
@xeophon
xeophon force-pushed the codex/sdk-backed-dialects branch from 9ef4500 to c777d36 Compare October 2, 2026 16:04

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: c777d364fa

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread verifiers/v1/clients/eval.py Resolved
@xeophon
xeophon force-pushed the codex/sdk-backed-dialects branch from c777d36 to 177030d Compare October 2, 2026 16:16
@xeophon
xeophon force-pushed the refactor/interception-server branch 2 times, most recently from 27e28e3 to c42001c Compare October 4, 2026 00:07
@xeophon
xeophon force-pushed the codex/sdk-backed-dialects branch from 177030d to 2704d26 Compare October 4, 2026 00:07

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 2704d26f8b

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread verifiers/v1/clients/eval.py Resolved
@xeophon
xeophon force-pushed the codex/sdk-backed-dialects branch from 2704d26 to d9d71f5 Compare October 8, 2026 18:05
@xeophon
xeophon force-pushed the refactor/interception-server branch from c42001c to 1558d27 Compare October 8, 2026 18:05

@cursor cursor Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale Bugbot comment from a previous run.

Comment thread verifiers/v1/utils/chat.py Resolved
@xeophon
xeophon force-pushed the refactor/interception-server branch from 1558d27 to edd9f34 Compare October 15, 2026 02:20
@xeophon
xeophon force-pushed the codex/sdk-backed-dialects branch from d9d71f5 to 242fac6 Compare October 15, 2026 02:20
Comment thread verifiers/v1/dialects/policy.py Resolved

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 242fac6055

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread verifiers/v1/dialects/chat.py Resolved
Comment thread verifiers/v1/dialects/responses.py Resolved
Comment thread verifiers/v1/interception/server.py Resolved

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 242fac6. Configure here.

Comment thread verifiers/v1/clients/eval.py Resolved
@xeophon
xeophon force-pushed the codex/sdk-backed-dialects branch from 242fac6 to fa72b35 Compare October 15, 2026 20:56
@xeophon
xeophon force-pushed the refactor/interception-server branch from edd9f34 to b0a3837 Compare October 15, 2026 20:56
@xeophon
xeophon force-pushed the codex/sdk-backed-dialects branch from fa72b35 to 7022952 Compare October 22, 2026 22:39
@xeophon
xeophon removed this pull request from stack #2730 October 22, 2026 22:41
@xeophon
xeophon changed the base branch from refactor/interception-server to main October 22, 2026 22:41
@chatgpt-codex-connector

Copy link
Copy Markdown

💡 Codex Review

native["content"] = patch_content(
message.content, _content_to_wire(message.content)
)

P2 Badge Pass the original content when patching scalar rewrites

When a request interceptor replaces a one-part Chat content list with a scalar string, this call omits the original native content passed as patch_content's third argument. Consequently the scalar branch that preserves a single text block's native metadata never runs, so fields such as prompt-cache controls are dropped and the upstream request changes from a content-part list to plain text. The Responses setters at responses.py:104-110 have the same issue; pass each setter's original native content into patch_content.


declared = {str(PurePosixPath(workdir) / a.source): a for a in artifacts}

P2 Badge Resolve saved sidecar artifacts with their own workdirs

For a Harbor Compose artifact whose service is a sidecar and whose source is relative, collection resolves the source against that sidecar's inspected WorkingDir, but this lookup resolves every declaration against the main agent workdir supplied by write_episode. If the workdirs differ, the collected absolute source misses declared, so save_artifacts falls back to a generic artifact and silently loses the configured destination and service in the saved tree and manifest. Preserve the resolved declaration per collected source or resolve each service's artifacts with that service's workdir.

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant