trainer: Adding local mode guides #4221

Fiona-Waters · 2025-10-24T11:32:16Z

Description of Changes

Added 3 user-guides for local process, docker and podman local mode backends. This PR should not be merged until after kubeflow/sdk#119 is merged.

Related Issues

Closes: #

Related: # kubeflow/sdk#95 (PR) The details in this PR have been included in this PR as per this comment.

Checklist

You have signed off your commits
Ensure you follow best practices from our contributing guide.
(for big changes) I will post screenshots of the changes in a PR comment

google-oss-prow · 2025-10-24T11:32:27Z

Hi @Fiona-Waters. Thanks for your PR.

I'm waiting for a kubeflow member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

github-actions · 2025-10-24T11:32:42Z

🚫 This command cannot be processed. Only organization members or owners can use the commands.

Arhell

/ok-to-test

astefanutti · 2025-10-28T17:20:16Z

Thanks @Fiona-Waters awesome work, very useful!

/lgtm

andreyvelich

@Fiona-Waters Thanks for this awesome work, I will take a look soon!
Shall we merge it once the Container PR is complete: kubeflow/sdk#119?
/hold

Fiona-Waters · 2025-10-29T09:43:10Z

@Fiona-Waters Thanks for this awesome work, I will take a look soon! Shall we merge it once the Container PR is complete: kubeflow/sdk#119? /hold

@andreyvelich yes this can be merged after the Container PR. I will have a PR in trainer repo for examples soon too, which can also be merged after. Thanks

Signed-off-by: Fiona Waters <[email protected]> Co-authored-by: Saad Zaher <[email protected]>

astefanutti · 2025-11-03T12:41:05Z

Thanks @Fiona-Waters!

/lgtm
/assign @andreyvelich

andreyvelich

Thank you for this @Fiona-Waters!
Overall looks great, I would move common sections to the overview page to reduce duplicated content.
cc @kubeflow/kubeflow-sdk-team @kubeflow/kubeflow-trainer-team

content/en/docs/components/trainer/user-guides/local-execution-mode/_index.md

content/en/docs/components/trainer/user-guides/local-execution-mode/docker.md

content/en/docs/components/trainer/user-guides/local-execution-mode/local_process.md

andreyvelich · 2025-11-06T16:30:08Z

content/en/docs/components/trainer/user-guides/local-execution-mode/local_process.md

+rm -rf /tmp/a1b2c3d4e5f_xyz/
+```
+
+## Architecture


We might need to re-draw it as a diagram in https://www.drawio.com/
we can do it later.

I think I'll remove this section for now and create a follow on issue to create and add a proper diagram. wdyt?

Sure, thank you!

content/en/docs/components/trainer/user-guides/local-execution-mode/local_process.md

content/en/docs/components/trainer/user-guides/local-execution-mode/podman.md

Signed-off-by: Fiona Waters <[email protected]>

google-oss-prow · 2025-11-07T16:17:43Z

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please ask for approval from andreyvelich. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

content/en/docs/components/trainer/OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

Fiona-Waters · 2025-11-07T16:18:56Z

@andreyvelich thanks for the review - I've addressed your feedback, please take a look. Thanks!

andreyvelich

Thanks for this great effort @Fiona-Waters!
/lgtm
/assign @kramaranya @szaher @astefanutti

astefanutti · 2025-11-12T18:51:11Z

content/en/docs/components/trainer/user-guides/local-execution-mode/docker.md

+from kubeflow.trainer import CustomTrainer, TrainerClient
+from kubeflow.trainer.backends.container.types import ContainerBackendConfig


Suggested change

from kubeflow.trainer import CustomTrainer, TrainerClient

from kubeflow.trainer.backends.container.types import ContainerBackendConfig

from kubeflow.trainer import CustomTrainer, TrainerClient, ContainerBackendConfig

astefanutti · 2025-11-12T18:54:40Z

content/en/docs/components/trainer/user-guides/local-execution-mode/docker.md

+- **Python 3.9+**
+- **Kubeflow SDK**: Install with Docker support:
+  ```bash
+  pip install kubeflow[docker]


Suggested change

pip install kubeflow[docker]

pip install "kubeflow[docker]"

To avoid some issues with shell expansion like with ZSH.

astefanutti · 2025-11-12T18:55:54Z

content/en/docs/components/trainer/user-guides/local-execution-mode/docker.md

+from kubeflow.trainer import CustomTrainer, TrainerClient
+from kubeflow.trainer.backends.container.types import ContainerBackendConfig


Suggested change

from kubeflow.trainer import CustomTrainer, TrainerClient

from kubeflow.trainer.backends.container.types import ContainerBackendConfig

from kubeflow.trainer import CustomTrainer, TrainerClient, ContainerBackendConfig

astefanutti · 2025-11-12T18:57:50Z

content/en/docs/components/trainer/user-guides/local-execution-mode/local_process.md

+from kubeflow.trainer import CustomTrainer, TrainerClient
+from kubeflow.trainer.backends.localprocess import LocalProcessBackendConfig


Suggested change

from kubeflow.trainer import CustomTrainer, TrainerClient

from kubeflow.trainer.backends.localprocess import LocalProcessBackendConfig

from kubeflow.trainer import CustomTrainer, TrainerClient, LocalProcessBackendConfig

astefanutti · 2025-11-12T18:58:20Z

content/en/docs/components/trainer/user-guides/local-execution-mode/local_process.md

+**Example:**
+
+```python
+from kubeflow.trainer.backends.localprocess import LocalProcessBackendConfig


Suggested change

from kubeflow.trainer.backends.localprocess import LocalProcessBackendConfig

from kubeflow.trainer import LocalProcessBackendConfig

astefanutti · 2025-11-12T19:01:58Z

content/en/docs/components/trainer/user-guides/local-execution-mode/podman.md

+- **Python 3.9+**
+- **Kubeflow SDK**: Install with Podman support:
+  ```bash
+  pip install kubeflow[podman]


Suggested change

pip install kubeflow[podman]

pip install "kubeflow[podman]"

astefanutti · 2025-11-12T19:04:39Z

content/en/docs/components/trainer/user-guides/local-execution-mode/podman.md

+
+```bash
+# Install Podman
+brew install podman


Maybe we want follow the official recommendation:

Though not recommended, Podman can also be obtained through Homebrew
Also maybe linking to the official installation instructions would be simpler.

astefanutti · 2025-11-12T19:04:55Z

content/en/docs/components/trainer/user-guides/local-execution-mode/podman.md

+from kubeflow.trainer import CustomTrainer, TrainerClient
+from kubeflow.trainer.backends.container.types import ContainerBackendConfig


Suggested change

from kubeflow.trainer import CustomTrainer, TrainerClient

from kubeflow.trainer.backends.container.types import ContainerBackendConfig

from kubeflow.trainer import CustomTrainer, TrainerClient, ContainerBackendConfig

astefanutti · 2025-11-12T19:05:23Z

content/en/docs/components/trainer/user-guides/local-execution-mode/podman.md

+from kubeflow.trainer import CustomTrainer, TrainerClient
+from kubeflow.trainer.backends.container.types import ContainerBackendConfig


Suggested change

from kubeflow.trainer import CustomTrainer, TrainerClient

from kubeflow.trainer.backends.container.types import ContainerBackendConfig

from kubeflow.trainer import CustomTrainer, TrainerClient, ContainerBackendConfig

astefanutti · 2025-11-12T19:06:45Z

content/en/docs/components/trainer/user-guides/local-execution-mode/podman.md

+```python
+backend_config = ContainerBackendConfig(
+    container_runtime="podman",
+    auto_remove=False  # Containers remain after job completion
+)


"```" is missing

google-oss-prow bot added do-not-merge/work-in-progress area/trainer AREA: Kubeflow Trainer / Kubeflow Training Operator labels Oct 24, 2025

google-oss-prow bot requested review from ChanYiLin and terrytangyuan October 24, 2025 11:32

google-oss-prow bot added the needs-ok-to-test label Oct 24, 2025

google-oss-prow bot added the size/XXL label Oct 24, 2025

Fiona-Waters mentioned this pull request Oct 24, 2025

feat: Add ContainerBackend with Docker and Podman kubeflow/sdk#119

Merged

1 task

Fiona-Waters force-pushed the local-guides branch 5 times, most recently from dadef0b to cba13a5 Compare October 24, 2025 12:21

Arhell reviewed Oct 25, 2025

View reviewed changes

google-oss-prow bot added ok-to-test and removed needs-ok-to-test labels Oct 25, 2025

google-oss-prow bot assigned astefanutti Oct 28, 2025

google-oss-prow bot added the lgtm label Oct 28, 2025

andreyvelich reviewed Oct 28, 2025

View reviewed changes

Fiona-Waters mentioned this pull request Oct 29, 2025

feat: Adding local execution example notebook kubeflow/trainer#2907

Merged

1 task

Fiona-Waters force-pushed the local-guides branch from cba13a5 to 02fdc04 Compare October 30, 2025 11:14

google-oss-prow bot removed the lgtm label Oct 30, 2025

Fiona-Waters force-pushed the local-guides branch from 02fdc04 to 62b2611 Compare October 30, 2025 17:12

Trainer: Adding local mode guides

b20b122

Signed-off-by: Fiona Waters <[email protected]> Co-authored-by: Saad Zaher <[email protected]>

Fiona-Waters force-pushed the local-guides branch from 62b2611 to b20b122 Compare November 3, 2025 11:40

Fiona-Waters changed the title ~~[WIP] Trainer: Adding local mode guides~~ Trainer: Adding local mode guides Nov 3, 2025

google-oss-prow bot removed the do-not-merge/work-in-progress label Nov 3, 2025

google-oss-prow bot assigned andreyvelich Nov 3, 2025

google-oss-prow bot added the lgtm label Nov 3, 2025

andreyvelich changed the title ~~Trainer: Adding local mode guides~~ trainer: Adding local mode guides Nov 6, 2025

andreyvelich reviewed Nov 6, 2025

View reviewed changes

Addressing feedback

0c5370b

Signed-off-by: Fiona Waters <[email protected]>

google-oss-prow bot removed the lgtm label Nov 7, 2025

Fiona-Waters mentioned this pull request Nov 7, 2025

chore(trainer): Add architecture section and diagram to Execute TrainJobs Locally section #4231

Open

andreyvelich reviewed Nov 7, 2025

View reviewed changes

google-oss-prow bot assigned kramaranya and szaher Nov 7, 2025

google-oss-prow bot added the lgtm label Nov 7, 2025

astefanutti reviewed Nov 12, 2025

View reviewed changes

		from kubeflow.trainer import CustomTrainer, TrainerClient
		from kubeflow.trainer.backends.container.types import ContainerBackendConfig

		from kubeflow.trainer import CustomTrainer, TrainerClient
		from kubeflow.trainer.backends.localprocess import LocalProcessBackendConfig

	from kubeflow.trainer.backends.localprocess import LocalProcessBackendConfig
	from kubeflow.trainer import LocalProcessBackendConfig

trainer: Adding local mode guides #4221

Are you sure you want to change the base?

trainer: Adding local mode guides #4221

Conversation

Fiona-Waters commented Oct 24, 2025

Description of Changes

Related Issues

Checklist

Uh oh!

google-oss-prow bot commented Oct 24, 2025

Uh oh!

github-actions bot commented Oct 24, 2025

Uh oh!

Arhell left a comment

Choose a reason for hiding this comment

Uh oh!

astefanutti commented Oct 28, 2025

Uh oh!

andreyvelich left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Fiona-Waters commented Oct 29, 2025

Uh oh!

astefanutti commented Nov 3, 2025

Uh oh!

andreyvelich left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

google-oss-prow bot commented Nov 7, 2025

Uh oh!

Fiona-Waters commented Nov 7, 2025

Uh oh!

andreyvelich left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

andreyvelich left a comment •

edited

Loading