chore: added RedTeamExperiment round-trips

[poshinchen]

Description

RedTeamExperiment now round-trips through to_file / from_file (and to_dict / from_dict).
Currently Live targets (agent) are not JSON-serializable, so we'll need load-then-plug-in:

exp.to_file("rt.json")
loaded = RedTeamExperiment.from_file("rt.json")
loaded.agent = my_agent
report = loaded.run_evaluations()

What's persisted: cases (validated as RedTeamCase so the typed config survives reload), evaluators, attack strategies, and the experiment-level model id. What's deliberately omitted: the live agent (Agent / MultiAgentBase / TargetSession) and per-run state (_run_meta).

Changes

• AttackStrategy.to_dict / from_dict (base): emits strategy_type + label; from_dict resolves against a built-in registry (CrescendoStrategy, PromptStrategy) plus a custom_strategies=[...] escape hatch.
• CrescendoStrategy.to_dict: persists max_turns, max_backtracks, success_threshold, and model (when set, as a string id).
• PromptStrategy.to_dict: persists strategy_name, system_prompt_template, max_turns.
• RedTeamExperiment:
  • agent property + setter (and attack_strategies read-only view).
  • to_dict override adds attack_strategies and model; never persists agent.
  • from_dict / from_file overrides accept custom_evaluators and custom_strategies, auto-register AttackSuccessEvaluator, and validate cases as RedTeamCase.
• _serialize_model helper in strategies/base.py: coerces Model | str | None to a JSON-safe id; preserves None (so a strategy's "defer to experiment-level model" semantic survives a round-trip).

Why setter, not constructor injection at load time

A loaded experiment is a complete config object containing case set, evaluators, strategies, model, and is valid even without an agent. Only _default_task reads the agent, so deferring attachment until just before run_evaluations mirrors how cases and evaluators are already exposed as settable properties on the base Experiment. Calling run_evaluations() with no agent and no explicit task= still raises the existing message.

Related Issues

Type of Change

New feature

Testing

5 new tests in tests/strands_evals/experimental/redteam/test_experiment.py:

• test_agent_setter_round_trip — setter accepts a target after construction.

• test_to_dict_persists_strategies_and_model — exact serialized shape; verifies agent is omitted.

• test_from_dict_round_trip_runs_after_setting_agent — full file round-trip, raise-without-agent, then run after attaching.

• test_from_dict_accepts_custom_strategies — custom strategy subclass round-trips via the registry hook.

• test_from_dict_unknown_strategy_raises — clear error when a custom subclass isn't registered.

• I ran hatch run prepare (mypy + ruff clean; full redteam suite, 121 tests, passes).

Checklist

• I have read the CONTRIBUTING document
• I have reviewed and understand every line of code in this PR, including any generated by AI tools, and I can explain why it works
• My change is focused and reasonably small; I have split unrelated work into separate PRs
• I have added any necessary tests that prove my fix is effective or my feature works
• I have updated the documentation accordingly
• I have added an appropriate example to the documentation to outline the feature, or no new docs are needed
• My changes generate no new warnings
• Any dependent changes have been merged and published

---

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

[github-actions]

Assessment: Comment (approve-leaning)

Solid, well-scoped addition of to_file/from_file round-trips for RedTeamExperiment. The "load-then-plug-in" design (deferring the live agent until just before run_evaluations rather than persisting it) is the right call and is clearly documented in both docstrings and the PR description. Tests pass (16), ruff is clean, and I verified locally that the to_dict → from_dict → to_dict round-trip is symmetric. Nothing here is blocking.

Review Categories

• Serialization robustness: The _serialize_model helper silently drops a Model it can't coerce by returning None, which collides with the legitimate "defer to experiment model" sentinel — worth surfacing rather than swallowing.
• Test coverage: Round-trip and serialization tests only exercise string model ids; the Model-instance branch (the riskiest path) is untested. Suggested a symmetric to_dict equality assertion to harden the round-trip test against field drift.
• Module structure: A private _serialize_model is imported across three packages; promoting it to a shared/public helper would make the coupling intentional. Minor, given experimental/ status.

Nice work — the docstrings and inline rationale (esp. around the max_workers=1 case-swap and the model None semantics) made this easy to review.

[github-actions]

Assessment: Approve

Re-reviewed after the latest push — all four threads from the previous round are addressed in code, not just marked resolved. Verified locally: 18 tests pass, ruff clean.

Verification of resolved threads

• Serialization robustness ✅ — _serialize_model (now in new utils.py) logs a logger.warning in the STYLE_GUIDE type=<...> format when a non-None Model can't be coerced, and the docstring explains why it diverges from Evaluator._get_model_id. The silent-drop concern is gone.
• Test coverage ✅ — test_to_dict_serializes_model_instance exercises the Model-instance → model_id path via _StubModel, and test_to_dict_drops_non_coercible_model_with_warning covers the config=None branch and asserts on the warning. The riskiest path is now locked in.
• Module structure ✅ — _serialize_model was relocated to a shared redteam/utils.py and imported consistently across experiment.py, crescendo, and prompt_strategy. The cross-module dependency is now intentional rather than reaching into another module's private symbol.
• Round-trip assertion ✅ — line 223 adds the symmetric RedTeamExperiment.from_dict(exp.to_dict()).to_dict() == exp.to_dict() check while keeping the targeted per-field asserts for the behaviors equality can't express (agent is None, raise-without-agent).

Clean execution on the follow-up — thanks for the thorough turnaround.

[github-actions]

Assessment: Approve

Re-reviewed after commit 2b917bc. The author addressed @jjbuck's get_config() point — _serialize_model now uses the Model ABC contract method (model.get_config()) instead of the BedrockModel-specific config attribute, with the isinstance(..., dict) guard preserved. Verified against the SDK source that get_config() is the abstract interface and config is a provider implementation detail, so this is the correct forward-compatible fix. 18 tests pass, ruff clean. Nothing outstanding — good to merge.