Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Override the existing_tasks explicitly provided in the action's input #683

Merged
merged 3 commits into from
Jun 19, 2024
Merged

Override the existing_tasks explicitly provided in the action's input #683

Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
62966b0
Override the existing_tasks explicitly provided in the action's input
gabrielBusta Jun 18, 2024
a6f0fe1
Log the labels and task ids of any existing tasks that are overridden
gabrielBusta Jun 18, 2024
658adc2
Merge branch 'release' into escape-hatch-to-put-existing-tasks-in-the…
eu9ene Jun 19, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
33 changes: 33 additions & 0 deletions taskcluster/translations_taskgraph/actions/train.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
# This Source Code Form is subject to the terms of the Mozilla Public
# License, v. 2.0. If a copy of the MPL was not distributed with this
# file, You can obtain one at http://mozilla.org/MPL/2.0/.
import json
import logging

from taskgraph.actions.registry import register_callback_action
from taskgraph.decision import taskgraph_decision
Expand All @@ -10,6 +12,8 @@

from translations_taskgraph.parameters import get_defaults

logger = logging.getLogger(__name__)

TRAIN_ON_PROJECTS = (
"https://github.com/mozilla/firefox-translations-training",
"https://github.com/mozilla-releng/staging-firefox-translations-training",
Expand Down Expand Up @@ -398,6 +402,35 @@ def train_action(parameters, graph_config, input, task_group_id, task_id):
# an identical name.
parameters["existing_tasks"] = get_ancestors(start_task_ids)

# Override the `existing_tasks` explicitly provided in the action's input
existing_tasks = input.pop("existing_tasks", {})

# Find and log `overridden_existing_tasks`
overridden_existing_tasks = {
existing_task: parameters["existing_tasks"][existing_task]
for existing_task in existing_tasks.keys()
if existing_task in parameters["existing_tasks"]
}

if overridden_existing_tasks:
logger.info(
f"Old values for `overridden_existing_tasks`: {json.dumps(overridden_existing_tasks, indent=2)}"
)

# Do the override!
parameters["existing_tasks"].update(existing_tasks)

# Log the new values for the `overridden_existing_tasks`
new_values_for_overridden = {
existing_task: parameters["existing_tasks"][existing_task]
for existing_task in overridden_existing_tasks.keys()
}

if new_values_for_overridden:
logger.info(
f"New values for `overridden_existing_tasks`: {json.dumps(new_values_for_overridden, indent=2)}"
)

parameters["target_tasks_method"] = "train-target-tasks"
parameters["optimize_target_tasks"] = True
parameters["tasks_for"] = "action"
Expand Down