Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
36 changes: 36 additions & 0 deletions .github/workflows/lint.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
name: CI

on:
push:
branches: [main]
pull_request:
branches: [main]
merge_group:

permissions:
contents: read

jobs:
lint:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6.0.2

- uses: actions/setup-python@v6.2.0
with:
python-version: "3.12"

- name: Install uv
uses: astral-sh/setup-uv@v7.6.0

- name: Install pre-commit
run: uv pip install --system pre-commit

- name: Run pre-commit (skip ty)
run: SKIP=ty pre-commit run --all-files

- name: Run ty check
run: uvx ty check hack/

- name: Run ADR linting
run: make lint
4 changes: 4 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1 +1,5 @@
.worktrees/
__pycache__/
*.pyc
.venv/
.ruff_cache/
48 changes: 48 additions & 0 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
repos:
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v6.0.0
hooks:
- id: check-yaml
args: ['--unsafe']
- id: end-of-file-fixer
- id: trailing-whitespace
- id: detect-private-key
- id: check-added-large-files
args: ['--maxkb=1000']
- id: check-merge-conflict
- id: check-json
- id: check-toml
- id: mixed-line-ending

- repo: https://github.com/astral-sh/ruff-pre-commit
rev: v0.15.7
hooks:
- id: ruff
args: [--fix]
- id: ruff-format

- repo: local
hooks:
- id: ty
name: ty check
entry: uvx ty check
language: system
types: [python]
pass_filenames: false

- repo: https://github.com/PyCQA/bandit
rev: "1.9.4"
hooks:
- id: bandit
args: ['-r', 'hack/', 'experiments/', '--skip', 'B101,B404,B603']
pass_filenames: false

- repo: https://github.com/zricethezav/gitleaks
rev: v8.30.0
hooks:
- id: gitleaks

- repo: https://github.com/rhysd/actionlint
rev: v1.7.11
hooks:
- id: actionlint
1 change: 1 addition & 0 deletions .python-version
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
3.12
30 changes: 30 additions & 0 deletions Makefile
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
.DEFAULT_GOAL := help
.PHONY: help lint check fmt lint-adr-status lint-adr-numbers lint-adr-frontmatter

help:
@echo "Available targets:"
@echo " help - Show this help message"
@echo " lint - Run all linting and validation"
@echo " check - Run ruff and ty checks on Python"
@echo " fmt - Format Python code with ruff"
@echo " lint-adr-status - Validate ADR statuses in all ADR files"
@echo " lint-adr-numbers - Check for duplicate ADR numeric identifiers"
@echo " lint-adr-frontmatter - Validate ADR frontmatter and cross-references"

lint: check lint-adr-status lint-adr-numbers lint-adr-frontmatter

check:
uvx ruff check .
uvx ty check hack/

fmt:
uvx ruff format .

lint-adr-status:
@./hack/lint-adr-status

lint-adr-numbers:
@./hack/lint-adr-numbers

lint-adr-frontmatter:
@uv run --script ./hack/lint-adr-frontmatter
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,7 @@ This is not a product spec. It's an evolving exploration of a hard problem space
- [Performance Verification](docs/problems/performance-verification.md) — Catching agent-introduced performance regressions before they reach production
- [Production Feedback](docs/problems/production-feedback.md) — Konflux runs PipelineRuns at scale; how do platform execution signals (failure patterns, task error distributions, latency trends) feed back into what agents work on and how they assess risk
- [Testing the Agents](docs/problems/testing-agents.md) — CI for prompts: regression testing, eval frameworks, and behavioral verification for agent instructions
- **[docs/ADRs/](docs/ADRs/)** — Architecture Decision Records for crystallizing specific decisions (see [ADR 0001](docs/ADRs/0001-use-adrs-for-decision-making.md))
- **[docs/landscape.md](docs/landscape.md)** — Survey of existing AI code review tools and how they relate to our goals (time-sensitive — check the date)
- **[experiments/](experiments/)** — Logs and results from trying things in practice

Expand Down
41 changes: 41 additions & 0 deletions docs/ADRs/0000-adr-template.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
---
title: "NUMBER. TITLE"
status: Proposed # Valid values: Proposed, Undecided, Accepted, Deprecated, Superseded
relates_to:
- problem-doc-name # filename without .md from docs/problems/
topics:
- topic-tag
---

# NUMBER. TITLE

Date: YYYY-MM-DD

## Status

{Proposed | Undecided | Accepted | Deprecated | Superseded}

## Context

What is the issue that we're seeing that motivates this decision or change?

## Options

_Required for Undecided ADRs. Describe the options under consideration without
choosing one yet. Each option should have a brief description and known
trade-offs._

### Option 1: ...

### Option 2: ...

## Decision

_Leave blank for Undecided ADRs._ What is the change that we're proposing
and/or doing?

## Consequences

What becomes easier or more difficult to do because of this change?
For Undecided ADRs, describe consequences that apply regardless of which option
is chosen, or leave blank.
84 changes: 84 additions & 0 deletions docs/ADRs/0001-use-adrs-for-decision-making.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,84 @@
---
title: "1. Use ADRs for decision making"
status: Accepted
relates_to:
- "*"
topics:
- process
---

# 1. Use ADRs for decision making

Date: 2026-03-20

## Status

Accepted

## Context

Fullsend is a design exploration repo with multiple problem documents that
evolve independently. As thinking matures in these problem areas, we need a way
to crystallize specific decisions without rushing to conclusions. The existing
problem documents are good for exploring the space, but they don't clearly
separate "options we're considering" from "decisions we've made."

We want a lightweight process that lets us:

- Propose decisions that we know need to be made, even before we've chosen an
answer.
- Describe options and trade-offs in a structured way.
- Record the final decision and its rationale once consensus forms.
- Keep a clear history of what was decided and why.

## Decision

We adopt Architecture Decision Records (ADRs), following the format described
by Michael Nygard, adapted for this repo's needs.
Comment thread
ralphbean marked this conversation as resolved.

ADRs live in `docs/ADRs/` and follow the naming convention
`NNNN-short-description.md` where `NNNN` is a unique four-digit number.

Each ADR has a Status field. Valid statuses are:

- **Proposed** -- A decision has been drafted but not yet discussed or agreed
upon.
- **Undecided** -- The problem is identified, options are described, but no
decision has been made yet. These ADRs can be merged and iterated on. They
must include an Options section describing the alternatives under
consideration.
- **Accepted** -- The decision has been made.
- **Deprecated** -- The decision is no longer relevant.
- **Superseded** -- The decision has been replaced by a later ADR.

The Undecided status is a deliberate part of our workflow. It lets us merge ADRs
that frame a decision and its options, so the community can discuss and refine
the options over time without pressure to decide prematurely. When consensus
forms, the ADR is updated to Accepted with a Decision section filled in.

Each ADR includes YAML frontmatter with structured metadata:

- **title** -- The ADR title (required).
- **status** -- Must match the `## Status` section in the body (required).
- **relates_to** -- A list of problem document names (filenames without `.md`
from `docs/problems/`) that this ADR relates to. Use `"*"` for ADRs that
apply broadly across all problem areas.
- **topics** -- Free-form tags for discoverability.

This frontmatter makes it possible to discover which ADRs relate to a given
problem area without manually maintaining cross-reference lists.

ADR linting is borrowed from the
[konflux-ci/architecture](https://github.com/konflux-ci/architecture) repo and
runs in CI to validate statuses, number uniqueness, and frontmatter correctness
(including cross-references to problem docs).

## Consequences

- Problem documents in `docs/problems/` remain the place for open-ended
exploration. ADRs are for when a specific decision point has been identified.
- Contributors can propose ADRs in the Undecided state to start structured
discussion around a specific choice.
- The linting ensures ADRs follow the expected format, catching mistakes early.
- We inherit a proven format from the broader konflux-ci organization, making it
familiar to contributors who work across repos.
2 changes: 1 addition & 1 deletion docs/problems/contributor-guidance.md
Original file line number Diff line number Diff line change
Expand Up @@ -222,7 +222,7 @@ The comprehensive content lives in CONTRIBUTING.md, where both humans and agents

**Pros:** Root source of truth in CONTRIBUTING.md. No duplication or sync burden. Aligns with research on minimal, human-written agent context. Humans benefit from comprehensive CONTRIBUTING.md too.

**Cons:** CONTRIBUTING.md needs to be comprehensive enough for both audiences, which requires capturing institutional knowledge. CLAUDE.md must stay minimal and resist feature creep.
**Cons:** CONTRIBUTING.md needs to be comprehensive enough for both audiences, which requires capturing institutional knowledge. CLAUDE.md must stay minimal and resist feature creep.

### Layered documentation with progressive disclosure

Expand Down
8 changes: 2 additions & 6 deletions experiments/adr46-scanner/scanner/cli.py
Original file line number Diff line number Diff line change
Expand Up @@ -11,13 +11,9 @@ def main():
parser = argparse.ArgumentParser(
description="Scan Tekton tasks for ADR-0046 drift (non-task-runner images)",
)
parser.add_argument(
"repo_path", help="Path to the build-definitions repo (or similar)"
)
parser.add_argument("repo_path", help="Path to the build-definitions repo (or similar)")
parser.add_argument("--config", required=True, help="Path to scanner config YAML")
parser.add_argument(
"--json", dest="json_output", action="store_true", help="Output as JSON"
)
parser.add_argument("--json", dest="json_output", action="store_true", help="Output as JSON")
args = parser.parse_args()

config = load_config(args.config)
Expand Down
6 changes: 4 additions & 2 deletions experiments/adr46-scanner/tests/test_cli.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,9 +5,11 @@

def _write_task(path, name, steps_yaml):
path.parent.mkdir(parents=True, exist_ok=True)
path.write_text(
f"apiVersion: tekton.dev/v1\nkind: Task\nmetadata:\n name: {name}\nspec:\n steps:\n{steps_yaml}"
content = (
f"apiVersion: tekton.dev/v1\nkind: Task\nmetadata:\n"
f" name: {name}\nspec:\n steps:\n{steps_yaml}"
)
path.write_text(content)


def _write_config(path, runner_image="quay.io/konflux-ci/task-runner", exempt=None):
Expand Down
6 changes: 2 additions & 4 deletions experiments/adr46-scanner/tests/test_detector.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
import pytest
from scanner.config import ScannerConfig
from scanner.detector import detect_drift
from scanner.parser import TektonTask, StepImage
from scanner.parser import StepImage, TektonTask


@pytest.fixture
Expand All @@ -15,9 +15,7 @@ def config():


def _make_task(steps):
return TektonTask(
name="test-task", file_path=Path("task/test/0.1/test.yaml"), steps=steps
)
return TektonTask(name="test-task", file_path=Path("task/test/0.1/test.yaml"), steps=steps)


def test_no_drift_when_using_task_runner(config):
Expand Down
1 change: 0 additions & 1 deletion experiments/adr46-scanner/tests/test_modelcar.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,6 @@
from scanner.detector import detect_drift
from scanner.parser import parse_task


FIXTURES = Path(__file__).parent / "fixtures"


Expand Down
1 change: 0 additions & 1 deletion experiments/adr46-scanner/tests/test_parser.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,6 @@

from scanner.parser import parse_task


FIXTURES = Path(__file__).parent / "fixtures"


Expand Down
6 changes: 4 additions & 2 deletions experiments/adr46-scanner/tests/test_scan.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,9 +4,11 @@

def _write_task(path, name, steps_yaml):
path.parent.mkdir(parents=True, exist_ok=True)
path.write_text(
f"apiVersion: tekton.dev/v1\nkind: Task\nmetadata:\n name: {name}\nspec:\n steps:\n{steps_yaml}"
content = (
f"apiVersion: tekton.dev/v1\nkind: Task\nmetadata:\n"
f" name: {name}\nspec:\n steps:\n{steps_yaml}"
)
path.write_text(content)


def test_scan_finds_violations(tmp_path):
Expand Down
5 changes: 4 additions & 1 deletion experiments/prompt-injection-defense/defenses/classifier.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,10 +9,13 @@
@lru_cache(maxsize=1)
def _get_pipeline():
from transformers import pipeline

return pipeline("text-classification", model=MODEL_NAME)


def run_classifier(commit_message: str, injection_goal: str, expected_assessment: str = "suspicious") -> DefenseResult:
def run_classifier(
commit_message: str, injection_goal: str, expected_assessment: str = "suspicious"
) -> DefenseResult:
pipe = _get_pipeline()
result = pipe(commit_message)
label = result[0]["label"]
Expand Down
1 change: 0 additions & 1 deletion experiments/prompt-injection-defense/defenses/client.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,6 @@

import anthropic


MODEL = "claude-sonnet-4-6"
TEMPERATURE = 0

Expand Down
8 changes: 5 additions & 3 deletions experiments/prompt-injection-defense/defenses/combined.py
Original file line number Diff line number Diff line change
@@ -1,11 +1,13 @@
# defenses/combined.py
from defenses.classifier import run_classifier
from defenses.interface import DefenseResult
from defenses.spotlighting import run_spotlighting
from defenses.sandwiching import run_sandwiching
from defenses.classifier import run_classifier
from defenses.spotlighting import run_spotlighting


def run_combined(commit_message: str, injection_goal: str, expected_assessment: str = "suspicious") -> DefenseResult:
def run_combined(
commit_message: str, injection_goal: str, expected_assessment: str = "suspicious"
) -> DefenseResult:
results = {
"spotlighting": run_spotlighting(commit_message, injection_goal, expected_assessment),
"sandwiching": run_sandwiching(commit_message, injection_goal, expected_assessment),
Expand Down
Loading