-
Notifications
You must be signed in to change notification settings - Fork 35
feat: Add docker backend #111
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: The full list of commands accepted by this bot can be found here.
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
Pull Request Test Coverage Report for Build 18128142359Details
💛 - Coveralls |
7546cfc
to
92fa6c8
Compare
Signed-off-by: Brian Gallagher <[email protected]>
92fa6c8
to
407c08e
Compare
logger = logging.getLogger(__name__) | ||
|
||
|
||
DOCKER_LABEL_PREFIX = "trainer.kubeflow.ai" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
DOCKER_LABEL_PREFIX = "trainer.kubeflow.ai" | |
DOCKER_LABEL_PREFIX = "trainer.kubeflow.org" |
import uuid | ||
|
||
try: | ||
import docker # type: ignore |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
How difficult would it be to support Podman as well: https://pypi.org/project/podman/
Should it be a different backend or could this be a "container" backend that could work for both Docker and Podman depending on what's installed / configured?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think it's a little bit cleaner to have a separate backend as it uses the podman client and the code is a bit different. The networking is slightly different.
But, a nice addition might be some auto discovery after all backends are added.
Meaning, look for Podman, then docker finally fall back to subprocess.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I would agree with @astefanutti to unify all container-based functionality under single backend.
From the end-user point of view it doesn't matter whether they use Docker, Podman, or Container backend to run local containers.
What this PR does / why we need it:
This PR adds a docker backend, extending the current Kubernetes and Local Process backends.
Its designed for Local Execution
Which issue(s) this PR fixes (optional, in
Fixes #<issue number>, #<issue number>, ...
format, will close the issue(s) when PR gets merged):Fixes #108
Verification
This WIP PR to add an example notebook for local docker execution can be used to verify this PR
Update the pip install to:
!pip install "git+https://github.com/briangallagher/sdk.git@docker-backend#egg=kubeflow[docker]"