Appeng 4932/ab eval flow templates - #6
Merged
Conversation
Replace Dockerfile.skilled.j2 and Dockerfile.unskilled.j2 with a single Dockerfile.j2 that uses copy_pairs loop. Update task.toml.j2 to use skills_dir check instead of variant name. Remove variant-to-template bridge code from scaffold.py.
Rename skilled-task-dir/unskilled-task-dir results to treatment-task-dir/control-task-dir. Update step script to use tasks-treatment/ and tasks-control/ output directories.
Update implementation_plan.md phases 2-6, README "How It Works", and ab_testing_generalization_plan.md checklists. Add workstreams roadmap. No code changes — terminology alignment only.
GuyZivRH
force-pushed
the
APPENG-4932/ab-eval-flow-templates
branch
from
April 14, 2026 09:50
283ba17 to
a998b1b
Compare
Remove has_docs from _build_template_context — no template references it after Dockerfile unification (docs handled via copy_pairs). Fix scaffold_submission() return order in generalization plan doc: (treatment_dir, control_dir), not (control_dir, treatment_dir).
GuyZivRH
force-pushed
the
APPENG-4932/ab-eval-flow-templates
branch
from
April 14, 2026 10:03
a998b1b to
5793f45
Compare
Sync code snippets with actual validators (CopySpec._validate_dest, customize_context submission_dir param, _PerVariantStrategy base). Check all commit checklists, fix Dockerfile.j2 snippet, correct copy_pairs descriptions, update branch/PR info and files table.
Stop tracking Docs/ab_testing_generalization_plan.md — it is a local working document, not a deliverable. The file remains on disk for reference.
r2dedios
reviewed
Apr 14, 2026
r2dedios
left a comment
r2dedios
left a comment
There was a problem hiding this comment.
Looks good, Just a small question from my side
| @@ -60,13 +60,13 @@ spec: | |||
There was a problem hiding this comment.
Is pydantic missing here? It is not installed by default in: registry.access.redhat.com/ubi9/python-311:latest
Collaborator
Author
There was a problem hiding this comment.
you are right! good catch! validate.yaml task already installs pydantic but it was only missing from the scaffold task. the fix I adds pydantic to the pip install line in scaffold.yaml
scaffold.py imports abevalflow.schemas (Pydantic models) but the Tekton task only installed jinja2 and pyyaml. pydantic is not bundled in ubi9/python-311.
5 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Completes the A/B generalisation started in PR #5 by eliminating the last skilled/unskilled remnants in templates, Tekton task definitions, and documentation.
Changes
Dockerfile.skilled.j2+Dockerfile.unskilled.j2into a singleDockerfile.j2driven bycopy_pairsloop; updatetask.toml.j2to key offskills_dirvalue instead of variant name; remove variant-to-template bridge code fromscaffold.pyskilled-task-dir→treatment-task-dir,unskilled-task-dir→control-task-dir) and align step variables/pathsimplementation_plan.md,README.md,ab_testing_generalization_plan.md; addworkstreams_roadmap.mdhas_docscontext key (no template uses it after unification); fixscaffold_submission()return order in generalization plan docReview feedback addressed
has_docscontext key from_build_template_context(R1 Appeng 4903/tekton triggers and validate task #2, R2 Appeng 4903/tekton triggers and validate task #2)(treatment_dir, control_dir)not(control_dir, treatment_dir)(R2 Appeng 4903/phase 1 validation #1)_VARIANT_TEMPLATE_MAPwhich was already removed in PR Appeng 4932/ab eval flow conversion #5 (R1 Appeng 4903/phase 1 validation #1)copy_pairs/COMMON_COPY_DIRSguard (R1 Appeng 4904/phase 2 scaffolding #3),skill-nameTekton param rename (R1 refactor: rename project from SkillsEvalFlow to ABEvalFlow #4), ADR terminology (R2 Appeng 4904/phase 2 scaffolding #3)Test plan
uv run pytest -q— 132 passedhas_docsscaffold_submission()return order matches docRelated