Repository navigation
Ci dsl dag design - #94
Merged
Merged
Conversation
Design for migrating CI YAML generation from Rust string concatenation to DSL-first modeling. Covers: - CG-1: CI model types (CiWorkflow, CiJob, CiStep, etc.) in dsl/std/ci.dag - CG-2: Pure rendering functions in dsl/std/ci_render.dag - CG-3: cigen tool entrypoint in dsl/tools/cigen.dag - CG-4: Rust cigen code deletion plan (~200 lines) - CG-5: RenderConfig/SharedStep cleanup plan (~500 lines) Follows makegen pattern: extern discovery -> pure DSL rendering -> content_upsert. Co-authored-by: Brian Searls <briansrls@users.noreply.github.com>
|
Cursor Agent can help with this pull request. Just |
Major revision of the CI DSL DAG design. Key change: introduces std/format_render.dag as a bridge between std/languages.dag (language models) and shared/dag_util.dag (Document rendering engine). The format-aware layer derives comment syntax, headers, and section markers from ConfigFormat/Language data declarations rather than hardcoding '#' strings. CI rendering becomes a consumer of this layer — config_format_header(yaml_format, ...) instead of literal '# Generated by...'. Includes consolidation inventory: 8+ locations across pragma.dag, bootstrap.dag, deps.dag, makegen.dag that currently hardcode comment prefixes, all replaceable via format_render.dag. Architecture: Layer 0: std/languages.dag (what is YAML? — CommentSyntax) Layer 1: std/format_render.dag (format-aware primitives) Layer 2: std/ci_render.dag (CI rendering, consumes Layer 1) Layer 3: tools/cigen.dag (entrypoint tool) Co-authored-by: Brian Searls <briansrls@users.noreply.github.com>
New files following the compositional modeling philosophy:
Layer 0 (std/):
- std/format_render.dag: Format-aware rendering bridge. Derives
comment syntax from ConfigFormat/Language models instead of
hardcoded strings. Connects std/languages.dag to Document engine.
- std/yaml_render.dag: YAML serialization primitives (mappings,
sequences, indentation, comments). Follows markdown_render.dag
pattern.
Layer 1 (extdeps/):
- extdeps/github_actions.dag: 'What is GitHub Actions?' Tautological
types — Workflow, Job, Step (Run|Uses), Trigger, Permission,
Cache, RunnerImage. Well-known actions and runners as data.
- extdeps/gitlab_ci.dag: 'What is GitLab CI?' Tautological types —
Pipeline, Job, Variable, Need, CacheConfig.
Layer 2 (config/):
- config/ci.dag: Our CI configuration. Static data declarations
composing extdeps types with our repo's requirements (branches,
runner, permissions, env, cache paths).
Layer 3 (tools/):
- tools/cigen.dag: Rendering + entrypoint. Composes YAML primitives
with provider schemas. Extern bridge for dynamic values (secrets,
tool command). Renders both GitHub Actions and GitLab CI YAML via
content_upsert.
Architecture: YAML (language) -> GitHub Actions / GitLab CI (interfaces)
-> our config (policy) -> cigen tool (rendering + execution). Each layer
is a tautology stacked on the layer below.
Co-authored-by: Brian Searls <briansrls@users.noreply.github.com>
Rewrites the design doc to match the implemented architecture: - YAML as a language (std/yaml_render.dag) - GitHub Actions as an external dependency (extdeps/github_actions.dag) - GitLab CI as an external dependency (extdeps/gitlab_ci.dag) - Our config as policy composition (config/ci.dag) - Clean layer separation matching docs/modeling.md patterns Co-authored-by: Brian Searls <briansrls@users.noreply.github.com>
format_render.dag now has two parts:
1. Format-aware primitives — comment syntax derived from language models
2. Structured-text primitives — indent_text, kv_pair, list_item,
text_block, list_block, kv_block. Format-agnostic building blocks.
yaml_render.dag is now a thin wrapper fixing YAML parameters:
indent_unit=' ', kv_sep=': ', list_prefix='- '
Each yaml_* function is a one-liner delegating to format_render.
The common layer is reusable for Makefile (unit='\t', sep=' = ')
and other formats.
Co-authored-by: Brian Searls <briansrls@users.noreply.github.com>
- Replace cmd_cigen() in codegen_cli.rs with DSL-based execution: builds graph from tools/cigen.dag, executes via execute_and_display. Deletes ~260 lines of template generation and validation code. - Add DiscoverCiConfigOp extern bridge in extern_impls.rs: returns secrets (from testgen metadata), tool_command (from CargoInvocation), and bootstrap_script. Static config (branches, runner, permissions, env) comes from config/ci.dag. - Delete: generate_github_actions_template(), generate_gitlab_ci_template(), validate_github_actions_template(), validate_gitlab_ci_template(), validate_generated_ci_template(), validate_required_sections(), CiTemplateKind enum, and associated tests. - Clean up unused imports: RenderConfig, CacheConfig, yaml_block, GitHubActionsProvider, GitLabCiProvider, CiRenderer, WorkspaceBinary, fmt::Write. Co-authored-by: Brian Searls <briansrls@users.noreply.github.com>
Delete the WorkflowSpec-to-DAG bridge (workflow_specs_to_dag, render_github_actions_from_workflow_specs, render_gitlab_ci_from_workflow_specs) — ~190 lines. The transport/ci/render.rs types (RenderConfig, SharedStep, CacheConfig) are still used by the runtime CI provider implementations (GitHubActionsProvider, GitLabCiProvider) for step-mode progress rendering. Cleaning those up requires migrating the runtime rendering path and is deferred. Co-authored-by: Brian Searls <briansrls@users.noreply.github.com>
- Merge format_render.dag into std/render.dag: one unified render module with terminal rendering IR, format-aware primitives (derived from language models), and structured-text building blocks. - Move yaml_render.dag to extdeps/yaml.dag: YAML is an external dependency (like Make, Clippy), not a standard library concept. Thin parameterization of std/render.dag for YAML syntax. - Delete std/format_render.dag and std/yaml_render.dag. - Update tools/cigen.dag import: std.yaml_render -> extdeps.yaml. Co-authored-by: Brian Searls <briansrls@users.noreply.github.com>
briansrls
marked this pull request as ready for review
February 28, 2026 13:14
Co-authored-by: Brian Searls <briansrls@users.noreply.github.com>
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: aa1295ce6c
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
- render.dag: replace ?? with match for comment_prefix_for (typechecker
infers String? from ?? despite fallback being String)
- cigen.dag: restructure to avoid patterns the typechecker can't handle:
- Remove typed intermediate records (Workflow, Job, Step, Cache,
Pipeline) — render directly to strings instead. The typechecker
doesn't coerce sum type variants to parent types or record literals
to named types across module boundaries.
- Inline config data (branches, permissions, env) as literals instead
of cross-module data references that resolve as Any.
- All 316 daglang-cli tests pass (was 298/316).
Co-authored-by: Brian Searls <briansrls@users.noreply.github.com>
- module_graph: add config.ci, extdeps.github_actions, extdeps.gitlab_ci, extdeps.yaml, tools.cigen; update std.render dep count (3→6). - extern_ratchet: bump baselines (extern func 2→3, extern impl 6→7) for discover_ci_config extern bridge. Co-authored-by: Brian Searls <briansrls@users.noreply.github.com>
- Remove shared.dag_util import from std/render.dag (layer 0 cannot import from shared/ layer 5). The format_document/render_format_document functions that depended on it are removed — tools that need Document construction import dag_util directly. - Update std.render dep count snapshot (6→4). - Add new files to daglang-syntax corpus inventory. Co-authored-by: Brian Searls <briansrls@users.noreply.github.com>
DSL discovery finds tools/cigen.dag and generates cigen + cigen-dry Make targets alongside the existing tool targets. Co-authored-by: Brian Searls <briansrls@users.noreply.github.com>
- Add Cigen variant to WorkspaceBinary enum (binaries.rs) - Add tools.cigen::cigen and std.width::truncate_text to passthrough callable allowlist (tool_registration.rs) Co-authored-by: Brian Searls <briansrls@users.noreply.github.com>
The codegen-generated binary needs a Cargo.toml declaration pointing to target/codegen/bin/cigen/main.rs, matching the pattern of other DSL-discovered tools (makegen, pragma, etc.). Co-authored-by: Brian Searls <briansrls@users.noreply.github.com>
Passing Some("written") to execute_and_display made cigen fail when
content_upsert returned written=false (file already up-to-date). This
broke idempotent reruns. Pass None instead — cigen succeeds if the
graph executes without errors, regardless of whether files changed.
Co-authored-by: Brian Searls <briansrls@users.noreply.github.com>
21-item queue (ED-1 through ED-21) for tautological modeling of all
external systems the SDLC scenario touches: GitHub APIs, GCP services,
AWS services, Azure services, LLM providers, Git, Cargo.
Includes design direction:
- Pattern to follow (types + data, zero opinions)
- File layout (extdeps/cloud/{gcp,aws,azure}/, extdeps/github/, etc.)
- Layering explanation (extdeps -> config -> services -> tools)
- Design decisions (CAS modeling, auth layering, granularity)
- Priority order (GitHub first, then GCP, then LLM, then CLI tools)
Also marks CG-1:5 as done in blue backlog.
Co-authored-by: Brian Searls <briansrls@users.noreply.github.com>
This was referenced May 7, 2026
This was referenced May 7, 2026
Merged
briansrls
added a commit
that referenced
this pull request
May 14, 2026
infer.rs is now PB-0 cycle-4 generated output; keep gate #94 helpers in standalone memory_peak_cost.rs and re-list it in SG-0 NON_TEST. Co-authored-by: Cursor <cursoragent@cursor.com>
This was referenced May 14, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add design document for migrating CI YAML generation to a DSL-first DAG model.
This document outlines the design for CG-1 through CG-5 from the Blue Team backlog, focusing on migrating CI YAML generation from ~700 lines of Rust string concatenation to a DSL-first modeling approach. It details key design decisions for
dsl/std/ci.dag,dsl/std/ci_render.dag, anddsl/tools/cigen.dag, along with a deletion plan for existing Rust code.