Skip to content
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 3 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,8 @@ A tested platform you can run, inspect, and build on:
- **Dual-backend persistence**: SQLite (single-node default) and PostgreSQL (multi-instance), conformance-tested for parity, with in-process yoyo schema migrations and ISO 4217 currency stamping on every cost-bearing row.
- **Provider layer**: any LLM via LiteLLM with built-in retry and rate-limit handling; local model management for Ollama and LM Studio.
- **Configuration and templates**: define a company in YAML; importable/shareable agent, department, and company templates with personality presets.
- **Subsystem libraries, tested as components**: the engine, memory, budget, security, coordination, and intake modules exist as importable, unit-tested code. Note honestly: these are exercised by their test suites, not yet by a running agent (see below).
- **Subsystem libraries, tested as components**: the engine, memory, budget, security, and coordination modules exist as importable, unit-tested code. Note honestly: these are exercised by their test suites, not yet by a running agent (see below).
- **Client-simulation intake runtime**: the intake engine is wired into boot via the client-simulation runtime and driven end to end by a deterministic simulation harness (synthetic clients, scripted provider, zero LLM spend), which is also the acceptance substrate for the runtime work in flight.
Comment thread
coderabbitai[bot] marked this conversation as resolved.
Outdated
- **Operations**: structured logging with redaction and correlation, Prometheus metrics and OTLP, HttpOnly-cookie multi-user sessions with CSRF protection, Chainguard distroless images with Trivy + Grype scanning, cosign signatures, and SLSA L3 provenance.
- **Distributed dispatch plumbing**: NATS JetStream queue and a worker pool. The dispatch path exists; the task-execute endpoint currently advances task state and does not yet invoke an agent.

Expand All @@ -44,7 +45,7 @@ These are the capabilities that make SynthOrg an autonomous studio. They are des
- **Best-in-class operate tier**: a golden-company benchmark, mission control with run replay, a cost forecast/kill-switch dial, a measurable learning curve, deterministic replay, run narratives, and an adversarial red-team.
- **Agent capability layer**: a knowledge and provenance retrieval substrate, research mode, continual improvement, governed external API access, headless-browser and virtual-desktop testing, and more.

Until the runtime lands, multi-agent coordination, coordination metrics, the intake engine, autonomy/trust enforcement on a live run, and the self-improvement loop are designed and unit-tested but not exercised end to end. The design for each lives in the [Design Specification](https://synthorg.io/docs/design/).
Until the runtime lands, multi-agent coordination, coordination metrics, autonomy/trust enforcement on a live run, and the self-improvement loop are designed and unit-tested but not exercised end to end. The design for each lives in the [Design Specification](https://synthorg.io/docs/design/).
Comment thread
coderabbitai[bot] marked this conversation as resolved.
Outdated

## Quick Start

Expand Down
22 changes: 18 additions & 4 deletions docs/design/client-simulation.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,10 +5,6 @@ description: Synthetic client framework for generating task requirements, review

# Client Simulation

!!! warning "Designed behaviour; runtime in active development"

This page is the source of truth for the **designed** behaviour of this subsystem. The intake engine and simulation runtime are not wired into the shipped product yet; this is in active development (see the [Roadmap](../roadmap/index.md)). The code described here is built and unit-tested as components but not yet run end to end.

The client simulation subsystem generates synthetic workloads that exercise the full
task lifecycle end-to-end. Simulated clients (AI-driven, human, or hybrid) submit task
requirements through an intake pipeline and review completed deliverables via a
Expand Down Expand Up @@ -195,6 +191,23 @@ through `TASK_CREATED`. It routes requests to a configured `IntakeStrategy`:
- **AgentIntake**: routes to an intake agent (PM/Account Manager) for
triage, scoping, and approval before task creation.

### Boot wiring

`synthorg.client.runtime_builder.build_client_simulation_runtime`
constructs the `IntakeEngine` (plus a single-stage `ReviewPipeline`
of `InternalReviewStage`) during app construction whenever a
`TaskEngine` is present, and `create_app` attaches the resulting
`ClientSimulationState` so `has_simulation_runtime` is true and the
`/simulations` + `/requests` controllers register. The strategy is
selected from the `simulations` settings namespace
(`intake_strategy` ∈ {`direct`, `agent`}, `intake_model`) via the
bootstrap resolver (env > registered default); the choice is baked
in at startup (`read_only_post_init`). The default `direct` strategy
makes no LLM calls, so the runtime comes online for an empty company.
A selected `agent` strategy that cannot be satisfied (no provider or
no model) degrades to `direct` with a warning rather than failing
boot.

---

## Task Source Tracking
Expand Down Expand Up @@ -272,6 +285,7 @@ discriminator rather than silently falling back to a default.
| `ReportConfig.strategy` | `build_report_strategy()` | `summary` → `SummaryReport`, `detailed` → `DetailedReport`, `json_export` → `JsonExportReport`, `metrics_only` → `MetricsOnlyReport` |
| `ClientPoolConfig.selection_strategy` | `build_client_pool_strategy()` | `round_robin` → `RoundRobinStrategy`, `weighted_random` → `WeightedRandomStrategy`, `domain_matched` → `DomainMatchedStrategy` |
| `adapter` arg (intake entry point) | `build_entry_point_strategy(adapter, *, project_id=None)` | `direct` → `DirectAdapter`, `project` → `ProjectAdapter`, `intake` → `IntakeAdapter` |
| `IntakeConfig.strategy` | `build_intake_strategy(config, *, task_engine, provider=None, cost_tracker=None)` | `direct` → `DirectIntake`, `agent` → `AgentIntake` |

The factories follow the project-wide pluggable-subsystems pattern
(protocol + strategy + factory + config discriminator). No silent
Expand Down
2 changes: 1 addition & 1 deletion scripts/_ghost_wiring_manifest.txt
Original file line number Diff line number Diff line change
Expand Up @@ -27,4 +27,4 @@ ENFORCED AgentEngine #1956 -- runtime root; construct at boot behind the provide
PENDING build_coordinator #1958 -- call at boot to populate app_state.coordinator
PENDING BaselineStore #1959 -- construct at boot (window from budget.baseline_window_size)
PENDING CoordinationMetricsCollector #1959 -- construct at boot, thread into execution
PENDING IntakeEngine #1961 -- wire via the client-simulation runtime
ENFORCED IntakeEngine #1961 -- wired at boot via client/runtime_builder.build_client_simulation_runtime
33 changes: 24 additions & 9 deletions src/synthorg/api/app.py
Original file line number Diff line number Diff line change
Expand Up @@ -886,17 +886,32 @@ def create_app( # noqa: C901, PLR0912, PLR0913, PLR0915
service="a2a_gateway",
)

# Default to a fresh ``ClientSimulationState()`` so the
# always-registered ``ClientController`` can serve an empty
# ``/clients`` list instead of 503ing on every dashboard poll.
# Callers wanting the full intake / review pipeline pass a
# configured state via the kwarg.
# Client-simulation runtime. An explicit kwarg always wins (test
# doubles / bespoke wiring). Otherwise, when a TaskEngine is
# present, build the live runtime (real IntakeEngine + review
# pipeline) so ``has_simulation_runtime`` is true and the
# ``/simulations`` + ``/requests`` controllers register; the
# default ``direct`` intake strategy makes no LLM calls and works
# for an empty company. With no TaskEngine the intake engine has
# nothing to create tasks against, so fall back to a fresh empty
# ``ClientSimulationState()`` -- the always-registered
# ``ClientController`` still serves an empty ``/clients`` list
# instead of 503ing on every dashboard poll. This mirrors the
# ``review_gate_service`` "build it whenever task_engine exists"
# gate above.
if client_simulation_state is None:
from synthorg.client.simulation_state import ( # noqa: PLC0415
ClientSimulationState as _ClientSimulationState,
)
if task_engine is not None:
from synthorg.client.runtime_builder import ( # noqa: PLC0415
build_client_simulation_runtime,
)

client_simulation_state = build_client_simulation_runtime(app_state)
else:
from synthorg.client.simulation_state import ( # noqa: PLC0415
ClientSimulationState as _ClientSimulationState,
)

client_simulation_state = _ClientSimulationState()
client_simulation_state = _ClientSimulationState()
app_state.set_client_simulation_state(client_simulation_state)

# Optional controllers gated on their primary collaborator service.
Expand Down
4 changes: 4 additions & 0 deletions src/synthorg/client/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@
ClientSimulationConfig,
ContinuousModeConfig,
FeedbackConfig,
IntakeConfig,
ReportConfig,
RequirementGeneratorConfig,
SimulationRunnerConfig,
Expand All @@ -21,6 +22,7 @@
build_client_pool_strategy,
build_entry_point_strategy,
build_feedback_strategy,
build_intake_strategy,
build_report_strategy,
build_requirement_generator,
)
Expand Down Expand Up @@ -79,6 +81,7 @@
"HybridClient",
"HybridRouter",
"InMemoryHumanInputQueue",
"IntakeConfig",
"PendingRequirement",
"PendingReview",
"PoolConstraints",
Expand All @@ -97,6 +100,7 @@
"build_client_pool_strategy",
"build_entry_point_strategy",
"build_feedback_strategy",
"build_intake_strategy",
"build_report_strategy",
"build_requirement_generator",
"default_router",
Expand Down
42 changes: 38 additions & 4 deletions src/synthorg/client/config.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,12 +5,18 @@
pools, requirement generators, and feedback strategies.
"""

from typing import Self
from typing import Final, Self

from pydantic import BaseModel, ConfigDict, Field, model_validator

from synthorg.core.types import NotBlankStr # noqa: TC001

# Acceptance band for the AI/human/hybrid pool-ratio sum: floating
# point makes an exact == 1.0 check brittle, so a +/-1% tolerance is
# allowed around the unit sum.
_RATIO_SUM_TOLERANCE_LOW: Final[float] = 0.99
_RATIO_SUM_TOLERANCE_HIGH: Final[float] = 1.01


class RequirementGeneratorConfig(BaseModel):
"""Configuration for requirement generation strategy.
Expand Down Expand Up @@ -123,9 +129,7 @@ class ClientPoolConfig(BaseModel):
def _validate_ratio_sum(self) -> Self:
"""Ensure ratios sum to approximately 1.0."""
total = self.ai_ratio + self.human_ratio + self.hybrid_ratio
_tolerance_low = 0.99
_tolerance_high = 1.01
if not (_tolerance_low <= total <= _tolerance_high):
if not (_RATIO_SUM_TOLERANCE_LOW <= total <= _RATIO_SUM_TOLERANCE_HIGH):
msg = (
f"Ratios must sum to approximately 1.0, got "
f"{self.ai_ratio} + {self.human_ratio} + "
Expand Down Expand Up @@ -207,6 +211,31 @@ class ReportConfig(BaseModel):
)


class IntakeConfig(BaseModel):
"""Configuration for the intake strategy.

Attributes:
strategy: Strategy identifier dispatched by
``build_intake_strategy``: ``direct`` (no LLM, creates a
task per accepted request) or ``agent`` (LLM-driven triage
via a completion provider).
model: Model identifier passed to the agent intake strategy.
Only consulted when ``strategy == "agent"``; ignored by
the ``direct`` strategy.
"""

model_config = ConfigDict(frozen=True, allow_inf_nan=False, extra="forbid")

strategy: NotBlankStr = Field(
default="direct",
description="Intake strategy identifier (direct or agent)",
)
model: NotBlankStr | None = Field(
default=None,
description="Model id for the agent intake strategy",
)


class ClientSimulationConfig(BaseModel):
"""Top-level client simulation configuration.

Expand All @@ -219,6 +248,7 @@ class ClientSimulationConfig(BaseModel):
report: Report format configuration.
runner: Simulation runner configuration.
continuous: Continuous mode configuration.
intake: Intake strategy configuration.
"""

model_config = ConfigDict(frozen=True, allow_inf_nan=False, extra="forbid")
Expand Down Expand Up @@ -247,3 +277,7 @@ class ClientSimulationConfig(BaseModel):
default_factory=ContinuousModeConfig,
description="Continuous mode configuration",
)
intake: IntakeConfig = Field(
default_factory=IntakeConfig,
description="Intake strategy configuration",
)
Loading
Loading