feat(agent): add simple agent with context compaction

[aroshanghias-nvd]

Summary

This PR adds a new opt-in simple_agent_with_compaction Responses API agent for multi-turn semantic context compaction.

Following reviewer feedback, the implementation is owned entirely by the new agent:

• responses_api_agents/simple_agent is unchanged from the base branch;
• shared schemas in nemo_gym are unchanged;
• compaction history, policies, materialization, scheduling, and response contracts live under responses_api_agents/simple_agent_with_compaction/compaction;
• compaction-specific tests live alongside the new agent.

Users who select the existing simple_agent see no code-path or configuration change.

Motivation

Multi-turn multimodal rollouts can repeatedly accumulate expensive context, particularly image observations and model reasoning blocks.

Dropping old request content is straightforward, but RL training also needs exact evidence about what each generation consumed and produced:

• prompt token IDs;
• sampled token IDs and corresponding logprobs;
• ordered media occurrences;
• enough information to determine whether a call appends to the previous physical trace or begins a rewritten trace.

The new agent keeps semantic compaction separate from generation evidence. NeMo-RL can infer physical trace boundaries from exact prompt-token prefix relationships without retokenizing Gym output.

Design

Agent-owned semantic history

Each rollout owns an append-only semantic history containing model outputs and environment observations. Images are stored once in a content-addressed media arena and referenced by stable IDs. Compaction policies derive model-facing views without mutating the complete logical history.

Built-in policies:

• identity: retains all semantic content;
• recency: independently supports image-group and reasoning-block retention.

Policies return a view plan, which is materialized into a Responses API request. Omitted image runs may be replaced with explicit text markers.

Scheduling and rewrite boundaries

Two schedules are supported:

• rolling recency;
• turn-chunked recency, which freezes one compacted base for a configured number of actions and appends a growing tail.

The controller compares each materialized request with the previously acknowledged request. A non-append-compatible view records a rewrite boundary and advances the context segment. Pending boundaries are retry-stable.

Exact generation evidence

The compaction session consumes generation-observed prompt IDs, sampled IDs, and logprobs, and carries exact-prefix requirements across append-compatible calls.

The vLLM adapter provides two explicit compaction capabilities:

• /tokenize for optional context-guard preflight;
• /v1/responses/context-compaction for generation with an exact required token prefix.

The prefix field is defined locally by the new agent and vLLM adapter rather than added to Gym's shared OpenAI schemas. Preflight tokenization is diagnostic only; generation evidence still comes from the generation operation.

Context guards

Optional guards can enforce limits on:

• total prompt plus reserved generation tokens;
• active image count;
• projected vision tokens.

A turn-chunked schedule may close a nonempty chunk early and recompact before rejecting a request.

Response contract

The new agent returns versioned evidence records containing per-call token evidence, media occurrence order, policy-decision identity, lineage deltas, chunk records, rewrite-boundary records, and guard outcomes.

After resource verification, validation-only duplication is projected into a bounded transport envelope. When verification is explicitly skipped, the complete validation envelope is retained.

Example configuration

simple_agent_with_compaction:
  responses_api_agents:
    simple_agent_with_compaction:
      entrypoint: app.py
      resources_server:
        type: resources_servers
        name: <resources-server>
      model_server:
        type: responses_api_models
        name: policy_model
      context_history:
        enabled: true
        policy:
          type: recency
          config:
            images:
              enabled: true
              protect_initial_context: true
              keep_last_groups: 3
            reasoning:
              enabled: true
              keep_first_block: false
              keep_last_blocks: 3
        schedule:
          type: turn_chunked_recency
          actions_per_chunk: 2

Compatibility

• The existing simple_agent implementation and tests match the base branch.
• Shared nemo_gym request schemas match the base branch.
• Context compaction is selected explicitly through simple_agent_with_compaction.
• Logical rollout output remains complete.
• Other agents do not automatically acquire compaction behavior.
• Rollout-level generation provenance remains explicitly incomplete when the generation server does not report exact tokenizer, template, or multimodal-processor identities.

Validation

Focused validation ran on DFW with one GPU using the rl-gym.63363213.sqsh container:

• all 63 focused compaction-agent, original simple_agent, and vLLM adapter cases passed;
• coverage includes identity behavior, image-only recency, reasoning-only recency, combined recency, chunk boundaries, guards, media ordering, lineage, retries, skipped verification, malformed generation evidence, and the local exact-prefix transport;
• git diff --check passes;
• all relocated Python sources parse successfully.

Supplemental downstream validation performed with the same compaction contract:

• 7 Gym-to-NeMo-RL bridge tests passed;
• no-compaction, image-only, reasoning-only, and combined policies were exercised;
• physical-trace reconstruction, token/logprob alignment, media ordering, rollout isolation, and inconsistent-evidence rejection were verified.

The deterministic environment and mocked-generation harness used for downstream validation are maintained outside this PR.

Non-goals

This PR does not:

• modify the existing simple_agent;
• add compaction to every Gym agent;
• promote context compaction into shared Gym core infrastructure;
• implement LLM-generated summaries;
• modify NeMo-RL training logic;
• include the external deterministic environment or fake reasoning generator used for downstream testing.

Suggested review order

1. responses_api_agents/simple_agent_with_compaction/app.py
2. responses_api_agents/simple_agent_with_compaction/compaction/history.py
3. compaction/policies.py and compaction/materialization.py
4. compaction/controller.py
5. compaction/session.py
6. responses_api_models/vllm_model/app.py
7. tests

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[cmunley1]

/claude review

[claude]

SHIP WITH CARE

Large, self-contained new agent (simple_agent_with_compaction) plus a vLLM /tokenize + /v1/responses/context-compaction extension. The scoring path is a faithful copy of simple_agent.run() — it proxies to the resources server's /verify and /aggregate_metrics, honors skip_verification/skip_verification_reward, and only decorates the response envelope with compaction evidence. No verifier or reward-aggregation logic is reimplemented, so the correctness-critical scoring surface is low-risk. Async is clean: all HTTP goes through self.server_client (aiohttp), no httpx/ray.get, and every awaited call is awaited. Test coverage is genuinely substantial (recency policies, chunk boundaries, guard admission, evidence capture, 100-turn boundedness).

One material finding, inline: the outbound body always carries required_prefix_token_ids via model_copy(update=...), which pydantic marks as set — so with context_history.enabled=false (the field default) the standard extra="forbid" /v1/responses endpoint rejects it (422). The shipped YAML sets enabled: true, so the default config works and the mocked tests pass, but any compaction-disabled run against a real model server fails. Graded RISK rather than BLOCKER since the exemplar avoids it; worth fixing before someone flips the flag off.

Notes (author's call):

• configs/simple_agent_with_compaction.yaml sets context_history.enabled: true inline; per CLAUDE.md, YAML is the right place for defaults, so this is fine — just keep the code default (ContextHistoryConfig.enabled=False) and the exemplar in sync with the README's "opt-in" framing.
• The compaction subsystem is a lot of provenance/lineage machinery for one agent; no correctness objection, but it's a large surface for future maintainers.

[claude]

RISK — extra field breaks the compaction-disabled path.

WHAT BREAKS: required_prefix_token_ids is placed in the model_copy(update=...) dict unconditionally, even when context_session is None (compaction disabled). In pydantic v2, model_copy(update={...}) adds every update key to __pydantic_fields_set__, so ServerClient.request() — which serializes with model_dump(exclude_unset=True) (nemo_gym/server_utils.py:337) — keeps required_prefix_token_ids: null. When context_session is None the POST targets the standard /v1/responses handler (line 280-283), which validates into NeMoGymResponseCreateParamsNonStreaming (extra="forbid", openai_utils.py:499) that has no such field → 422 extra_forbidden on every model call, and raise_for_status aborts the rollout.

BLAST RADIUS: the shipped exemplar YAML sets context_history.enabled: true, so the default path (which routes to /v1/responses/context-compaction, a model that does declare the field) works. But the README advertises compaction as opt-in, and any config that leaves enabled at its False default hits the standard endpoint and hard-fails against a real vLLM server. It's not caught in tests because ServerClient.post is mocked and pydantic __eq__ ignores fields_set.

FIX: only inject the key when compaction is active, e.g. build the update dict conditionally so required_prefix_token_ids is absent when prepared_call is None. That keeps it out of fields_set and the standard endpoint accepts the body.