Skip to content

Add Pi coding-agent loop-owning GRPO examples - #6600

Closed
sergiopaniego wants to merge 1 commit into
mainfrom
add-pi-openenv-example
Closed

Add Pi coding-agent loop-owning GRPO examples#6600
sergiopaniego wants to merge 1 commit into
mainfrom
add-pi-openenv-example

Conversation

@sergiopaniego

Copy link
Copy Markdown
Member

What does this PR do?

Adds a loop-owning GRPO training example for the real Pi coding agent, mirroring the opencode example. Two scripts under examples/scripts/openenv/:

  • pi.py: local subprocess sandbox (one node, no tunnel)
  • pi_hf_sandbox.py: one remote HF sandbox per rollout

Pi speaks OpenAI natively, so the in-sandbox interception proxy captures each turn's token ids and logprobs directly (no shim). AsyncGRPOTrainer + HarnessRolloutWorker (harness_adapter=None) train on the trace, with a held-out DeepCoder verifier as the reward.

Depends on the pi_env environment (huggingface/OpenEnv#999). Draft until that merges, then the uv headers point at the released package.

Before submitting

  • This PR improves the examples.

AI writing disclosure

  • AI-assisted

@sergiopaniego

Copy link
Copy Markdown
Member Author

Parking this as draft. The per-agent env approach here is superseded by the Harbor integration (huggingface/OpenEnv#1036), which covers these harnesses generically. Keeping it open rather than closing until Harbor actually lands, then it can be closed.

@sergiopaniego

Copy link
Copy Markdown
Member Author

Closing in favor of the generic harness infrastructure coming with Harbor (huggingface/OpenEnv#1036), which supersedes per-harness example wiring. Happy to revisit as a Harbor-based example.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant