From 8188963062e71645d7d3ebe70eca136ef01dc07d Mon Sep 17 00:00:00 2001 From: arewm Date: Mon, 9 Mar 2026 11:48:20 -0400 Subject: [PATCH] docs: Add agent-compatible code problem document No existing problem document addresses what properties of code make it suitable for autonomous agent operation. This is distinct from repo-readiness (which is diagnostic) and architectural-invariants (which derives from the architecture repo). The new document defines criteria-based language selection (static type checking, error locality, refactoring safety, tooling, deployment simplicity) without prescribing specific languages. It distinguishes between in-house code (which should meet the criteria) and consumed dependencies (where the requirement shifts to typed integration boundaries), using hermeto/cachi2 as a concrete example. Assisted-by: Claude Code (Opus 4.6) Signed-off-by: arewm --- README.md | 1 + docs/problems/agent-compatible-code.md | 60 ++++++++++++++++++++++++++ docs/problems/repo-readiness.md | 3 +- 3 files changed, 63 insertions(+), 1 deletion(-) create mode 100644 docs/problems/agent-compatible-code.md diff --git a/README.md b/README.md index 679985412e..a3754d8d96 100644 --- a/README.md +++ b/README.md @@ -20,6 +20,7 @@ This is not a product spec. It's an evolving exploration of a hard problem space - [Repo Readiness](docs/problems/repo-readiness.md) — Test coverage, CI/CD maturity, what's needed before agents can be trusted - [Code Review](docs/problems/code-review.md) — How agents review code, including security-focused sub-agents - [Architectural Invariants](docs/problems/architectural-invariants.md) — Enforcing things that must always be true, grounded in the existing architecture repo + - [Agent-Compatible Code](docs/problems/agent-compatible-code.md) — Language properties that affect agent effectiveness - [Codebase Context](docs/problems/codebase-context.md) — How agents acquire codebase understanding and how to structure org-level context - **[docs/landscape.md](docs/landscape.md)** — Survey of existing AI code review tools and how they relate to our goals (time-sensitive — check the date) - **[docs/experiments/](docs/experiments/)** — Logs and results from trying things in practice diff --git a/docs/problems/agent-compatible-code.md b/docs/problems/agent-compatible-code.md new file mode 100644 index 0000000000..781b7fdbf3 --- /dev/null +++ b/docs/problems/agent-compatible-code.md @@ -0,0 +1,60 @@ +# Agent-Compatible Code + +What properties of code affect how effectively agents can generate, review, and refactor it? + +## Language selection criteria + +Not all languages are equally suited for agentic development. The properties that matter: + +**Static type checking** catches type mismatches at compile time rather than at runtime, constraining the solution space so agents get immediate feedback on errors. + +**Error locality** means errors point to the source, not somewhere downstream. In dynamically typed languages a type error may not surface until deep in the call stack; in statically typed languages the compiler identifies the exact mismatch. + +**Type-propagated refactoring safety** is critical for autonomous changes. When an agent changes a function signature, the compiler surfaces every caller that needs updating — no heuristic search, no reliance on test coverage to catch what was missed. + +**Tooling ecosystem** — LSP servers, formatters, and linters provide semantic information and deterministic quality checks that agents can invoke programmatically. Mature tooling reduces guesswork about code structure. + +**Deployment simplicity** — single-artifact builds are easier to reason about than systems with runtime dependency graphs. Fewer moving parts mean fewer ways for an agent to introduce subtle breakage. + +## In-house vs consumed code + +There are two different standards, because the org doesn't control what languages upstream projects choose. + +### In-house code + +Code owned by konflux-ci — services, controllers, tooling — should meet the language criteria directly. When new tooling is created or experiments graduate to production, prefer languages that satisfy static typing, error locality, refactoring safety, mature tooling, and simple deployment. + +This doesn't mean rewriting existing code. It means that when choosing a language for new work, agents benefit from typed languages with strong tooling ecosystems. + +### Consumed dependencies + +Upstream libraries and services (hermeto/cachi2, PatternFly, Tekton, third-party libraries) can't be required to change languages. The requirement shifts to the integration boundary: + +**Schema-validated inputs and outputs** — if the org's code calls an external service or library, the contract should be explicit and validated. API schemas, protobuf definitions, OpenAPI specs — anything that makes the boundary machine-checkable. + +**Pinned versions and digests** — consumed dependencies should be pinned to specific versions or content hashes. This prevents unexpected changes from propagating into the system without review. + +**Integration tests** — the boundary between the org's code and the external dependency needs tests that verify the contract holds. If the upstream library changes its behavior in a way that breaks the integration, the test catches it. + +**Narrow contact surface** — minimize the API surface the org's code touches. Don't import an entire library when you only need one function. Wrap external dependencies in adapters that expose only the needed functionality. + +Example: hermeto (the build dependency solver) is written in Python. It runs as a container image invoked from Tekton tasks. The integration boundary is already container-isolated — hermeto's internal implementation language doesn't leak into the rest of the system. The risks are config schema drift and unvalidated outputs. The mitigation isn't rewriting hermeto; it's schema-validating the inputs, pinning the image digest, and testing the contract. + +## Relationship to other problem areas + +**Agent drift** — typed code constrains the solution space, reducing the ways an agent can drift over time. Type errors surface immediately rather than accumulating as latent bugs. + +**Code review** — a type checker handles entire categories of review concerns (type safety, null pointer dereference in nullable-aware type systems, resource lifetimes in Rust). Review agents can focus on semantic correctness, architectural compliance, and security implications rather than hunting for type errors. + +**Security** — typed languages prevent certain vulnerability classes (type confusion, buffer overflows in memory-safe languages). Boundary typing for consumed dependencies reduces supply chain risk by making the contract explicit and testable. + +**Repo readiness** — type safety could be a readiness criterion alongside test coverage and CI maturity. A repo with strong static typing and comprehensive integration tests is safer for agent autonomy than one relying entirely on runtime validation. + +## Open questions + +- Should type safety be a hard requirement for new services, or a strong preference with room for exceptions? +- How do we handle repos that are polyglot (e.g., Go controllers with shell scripts, Python utilities alongside React frontends)? Is type safety repo-level or component-level? +- Can gradual typing systems (TypeScript, Python with mypy) provide sufficient safety for agentic development, or do they retain too much of the dynamic language risk? +- What's the migration path for existing services written in dynamically typed languages? Is there a threshold where rewriting becomes worthwhile? +- How do we measure "agent compatibility" empirically? Can we compare agent error rates or refactoring success rates across languages? +- Should consumed dependencies have a "compatibility tier" based on how well-defined their boundaries are, affecting the autonomy level for changes that touch them? diff --git a/docs/problems/repo-readiness.md b/docs/problems/repo-readiness.md index 3352000993..26dd850c35 100644 --- a/docs/problems/repo-readiness.md +++ b/docs/problems/repo-readiness.md @@ -53,8 +53,9 @@ Test coverage is necessary but not sufficient. For agents to merge autonomously, - **Integration and e2e tests** — unit tests catch local bugs; integration tests catch system-level regressions - **Linting and formatting in CI** — prevents agent style drift - **Clear CI signals** — tests must be reliable (no flaky tests that train agents to ignore failures) -- **CLAUDE.md or equivalent** — agents need codebase context to work effectively +- **CLAUDE.md or equivalent** — agents need codebase context to work effectively (see [codebase-context.md](codebase-context.md) for what makes context files effective) - **CODEOWNERS** — defines the human-required approval paths +- **Language properties** — type safety, tooling ecosystem, and deployment simplicity affect how effectively agents can operate (see [agent-compatible-code.md](agent-compatible-code.md)) ## Diagnostic tooling