fix(consolidation): stop emitting unsupported maxItems that breaks all Bedrock consolidation (#2500)

[r266-tech]

Fixes #2500.

Problem

Setting max_observations_per_scope > 0 on a Bedrock-backed bank (any Claude model via Bedrock Converse) causes 100% of consolidation batches to fail — every batch fails all 3 retries and is silently skipped, so memory quality decays permanently:

litellm.BadRequestError: BedrockException - {"message":"The model returned the following errors:
output_config.format.schema: For 'array' type, property 'maxItems' is not supported"}

Root cause

_build_response_model() dynamically subclasses the consolidation response model when an observation cap is active and puts a Pydantic max_length on creates. Pydantic v2 serializes list max_length to the JSON-schema maxItems keyword, and Bedrock Converse rejects maxItems on array types outright — so a single maxItems in the structured-output schema breaks the whole call.

Fix

Drop the schema-level constraint: _build_response_model() now always returns the base _ConsolidationBatchResponse, whose creates field carries no length constraint, so the emitted schema is provider-portable.

This is safe because the observation cap does not depend on the schema hint — it is enforced by two other layers that are unchanged:

1. Prompt capacity note — whenever the cap binds (remaining_observation_slots <= len(memories), or == 0), the consolidation prompt already tells the model "This scope has {k} observation slot(s) remaining … Prefer UPDATE over CREATE" / "OBSERVATION LIMIT REACHED … Do NOT create new ones". Every provider (Bedrock or not) still receives the cap.
2. Authoritative truncation — the batch-call path unconditionally does creates = creates[:remaining_observation_slots], with your own comment "some LLM providers may not enforce JSON schema max_length". This was already the guaranteed enforcement path.

Known tradeoff (documented deliberately)

Providers that did honor maxItems lose that redundant schema-level ceiling and now rely on the prompt note + keep-first-N truncation. But keep-first-N was already your designated fallback for non-honoring providers, and consolidation creates are not priority-ordered, so this matches existing semantics. If you'd prefer to preserve the hint for compatible providers, a provider-gated variant (emit maxItems only when the provider supports it) is a straightforward follow-up — I kept this PR to the minimal, provably-safe change that unblocks Bedrock.

Tests

TestBuildResponseModel is rewritten to the new contract:

• test_returns_base_model_regardless_of_cap — factory always returns the base model.
• test_schema_omits_max_items — max_observations_per_scope > 0 breaks ALL consolidation on Bedrock: emits unsupported "maxItems" JSON Schema keyword #2500 regression: emitted schema must not contain maxItems on creates.
• test_model_does_not_reject_creates_over_cap — over-cap responses are accepted (truncated downstream), not rejected wholesale.

End-to-end cap enforcement remains covered by the existing test_max_observations_per_scope_* integration tests (which exercise the truncation path via a mock LLM, independent of schema validation).

[nicoloboschi]

@r266-tech can you make this change only true if a new config about the llm "supports max items" is set to false (default = true). the maxItems is actually very important here and removing from any llm is quite concerning to me

[r266-tech]

Reworked this around an explicit capability flag as requested.

HINDSIGHT_API_LLM_SUPPORTS_MAX_ITEMS defaults to true, so existing schema-level maxItems enforcement remains unchanged. Operators on Bedrock Converse (or another backend that rejects the keyword) can set it to false; that choice is read from the operation's config, and the existing runtime truncation remains authoritative.

I also added zero-slot and positive-slot tests that make the LLM overproduce with schema enforcement disabled and assert the returned creates are still capped, plus environment/default and schema-shape coverage. The focused tests pass (10 tests), changed-source Ruff checks pass, and the reworked diff passed Codex adversarial review.

[nicoloboschi]

lgtm

[nicoloboschi]

@r266-tech please rebase

[r266-tech]

Rebased onto a23187a4 as requested.

During the post-rebase adversarial review I also tightened the new capability flag: it now defaults to true for legacy direct HindsightConfig construction, and ambiguous environment values fail fast instead of silently disabling maxItems. The focused schema/config/runtime-cap tests pass (15 tests), along with changed-source Ruff and ty checks. CI is running on ea9a6d77.