Skip to content

Conversation

@TomSweeneyRedHat
Copy link
Member

@TomSweeneyRedHat TomSweeneyRedHat commented Sep 17, 2019

WIP - Show c/storage (Buildah/CRI-O) containers in ps

Ready for review. I still have some work to do with testing, varlink and documentation. But thought I'd get this out for a look/feel kind of review at the moment.

The podman ps --all command will now show containers that are under the control of other c/storage container systems and the new ps --external option will show only containers that are
in c/storage but are not controlled by libpod.

In the below examples, the '*working-container' entries were created by Buildah.

podman ps -a
CONTAINER ID  IMAGE                             COMMAND  CREATED       STATUS                   PORTS  NAMES
9257ef8c786c  docker.io/library/busybox:latest  ls /etc  8 hours ago   Exited (0) 8 hours ago          gifted_jang
d302c81856da  docker.io/library/busybox:latest  NA       30 hours ago  NA                              busybox-working-container
7a5a7b099d33  localhost/tom:latest              ls -alF  30 hours ago  Exited (0) 30 hours ago         hopeful_hellman
01d601fca090  localhost/tom:latest              ls -alf  30 hours ago  Exited (1) 30 hours ago         determined_panini
ee58f429ff26  localhost/tom:latest              NA       33 hours ago  NA                              alpine-working-container

podman ps --external
CONTAINER ID  IMAGE                             COMMAND  CREATED       STATUS  PORTS  NAMES
d302c81856da  docker.io/library/busybox:latest  NA       30 hours ago  NA             busybox-working-container
ee58f429ff26  localhost/tom:latest              NA       33 hours ago  NA             alpine-working-container

Signed-off-by: TomSweeneyRedHat tsweeney@redhat.com

@openshift-ci-robot openshift-ci-robot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Sep 17, 2019
@openshift-ci-robot
Copy link
Collaborator

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: TomSweeneyRedHat

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci-robot openshift-ci-robot added approved Indicates a PR has been approved by an approver from all required OWNERS files. size/M and removed size/M labels Sep 17, 2019
@AkihiroSuda
Copy link
Collaborator

Can they be treated as if they belong to namespaces? eg. podman --namespace=crio ps

@TomSweeneyRedHat
Copy link
Member Author

bot, retest this please.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Podman typo

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ty and fixed.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Horrible Function name. GetNonLibpodContainers? GetExternalContainers?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just return r.Runtime.ImageRuntime().GetImage(imageID)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually looking at it further, I don't need this function and don't call it. Backing this change out.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this not implemented?

@rhatdan
Copy link
Member

rhatdan commented Sep 20, 2019

I hate the name of the option. I would figure --storage would be to return all containers in storage and
--storage=false would return only libpod containers.

Can we change it to --external

podman ps -a --storage external

@AkihiroSuda has a good idea, what the user would really want is something like

--namespace buildah or --namespace podman --namespace CRI-O, but all we can identify right now is whether or not the container was created via libpod or not. Since Buildah and CRI-O do not use LIBPOD yet, we have no good way to identify where the storage containers came from.

@mheon
Copy link
Member

mheon commented Sep 20, 2019

We already have a --namespace flag, though it's exclusively a libpod feature right now

@rhatdan
Copy link
Member

rhatdan commented Sep 20, 2019

--namespace undefined? --namespace external?
Perhaps this would give me what I want.

  • podman ps --all
    Gives all containers in storage

  • podman ps --all --namespace=podman
    Gives all containers created by podman

  • podman ps --all --namespace=external
    Gives all containers not created with libpod

If CRI-O eventually grows libpod support

  • podman ps --all --namespace=cri-o
    Would give only cri-o containers.

@mheon
Copy link
Member

mheon commented Sep 20, 2019

--namespace accepts arbitrary strings. It's used for Libpod namespaces, which were added with the intention of namespacing off CRI-O containers once we merged into CRI-O. We can't really repurpose it here as such.

@mheon
Copy link
Member

mheon commented Sep 20, 2019

Also, I've just taken a brief look at the PR, and I'm pretty concerned that the logic here is leaking into libpod.

Libpod should never see a container not in the DB. We added separate functions for dealing with c/storage containers in runtime_cstorage.go; going through there prevents us from having to convince Libpod proper to deal with containers that don't exist in the DB. The cstorage functions don't give particularly much information, but there isn't much we can say about a container in the storage library if it doesn't exist in our DB - trying to populate ps as normal won't work, 90% of the fields can't be filled.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would not expect building containers like this to work; soe of this, like lock allocation, has some definite undesirable side-effects.

We should confine the storage logic to cmd/podman, and ideally keep it separate from the rest of ps. If a storage container hits Libpod, I do not expect good things to happen.

@TomSweeneyRedHat
Copy link
Member Author

I've moved the external container list building logic from libpod into the podman ps logic. I still need to test further, especially the remote stuff, but thought I'd put this up for review. I've also renamed the option to --external and changed the underlying function.

@mheon
Copy link
Member

mheon commented Sep 22, 2019

We should discuss this more on Monday - I still don't think that returning a *libpod.Container is viable here. We're going to unavoidably leak locks, for example.

I'd honestly prefer if we did not reuse the display logic from the rest of ps when displaying storage containers. We don't know enough about them to meaningfully populate all the fields in ps... But this is worth a more detailed discussion. Postscrum, maybe.

@TomSweeneyRedHat
Copy link
Member Author

Postscrum chat sounds good to me.

FWIW, the main reason I'm stuffing a storage container into a libpod container object is so that it can be shown in the ps --all and go through any filtering that a user asks to be done. Assuming we keep this approach, is there something I should do to deallocate the locks at the end of the run? I assumed, maybe badly, that they'd just go away at the end of the run when the in memory objects got cleaned up.

The other approach would be to just forget about showing the storage containers with the --all command. But I think that's helpful for some one looking for one of these containers after they can't remove an image that has a storage container and not a libpod container still attached.

FWIW, I think I've found the remote compilation issue. I'd to update a struct in vi ./cmd/podman/shared/container.go. 'make podman-remote' worked in my sandbox, does the CI do other compiling/linking? It failed as I'd not included External to the PsOpts struct in that file.

@mheon
Copy link
Member

mheon commented Sep 22, 2019 via email

@rh-atomic-bot
Copy link
Collaborator

☔ The latest upstream changes (presumably #4355) made this pull request unmergeable. Please resolve the merge conflicts.

@rhatdan
Copy link
Member

rhatdan commented Nov 7, 2019

@TomSweeneyRedHat Needs a rebase. Is this something we should talk about at the f2f?

@TomSweeneyRedHat
Copy link
Member Author

I think we're all set talk wise, I just need to carve out some time and get back to this.

@github-actions
Copy link

github-actions bot commented Dec 8, 2019

This pull request had no activity for 30 days. In the absence of activity or the "do-not-close" label, the pull request will be automatically closed within 7 days.

@openshift-ci-robot openshift-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Dec 8, 2019
@rhatdan rhatdan removed the stale-pr label Dec 8, 2019
@rhatdan
Copy link
Member

rhatdan commented Dec 8, 2019

@TomSweeneyRedHat I guess this has bubbled back up to importance, at least do a rebase, Please.

@github-actions
Copy link

github-actions bot commented Jan 8, 2020

A friendly reminder that this PR had no activity for 30 days.

@rhatdan
Copy link
Member

rhatdan commented Jan 8, 2020

@TomSweeneyRedHat Could you rebase this one, and we can start the conversation again.

Ready for review.  I still have some work to do with
testing, varlink and documentation.  But thought I'd get
this out for a look/feel kind of review at the moment.

The `podman ps --all` command will now show containers that
are under the control of other c/storage container systems and
the new `ps --external` option will show only containers that are
in c/storage but are not controlled by libpod.

In the below examples, the '*working-container' entries were created
by Buildah.

```
podman ps -a
CONTAINER ID  IMAGE                             COMMAND  CREATED       STATUS                   PORTS  NAMES
9257ef8c786c  docker.io/library/busybox:latest  ls /etc  8 hours ago   Exited (0) 8 hours ago          gifted_jang
d302c81856da  docker.io/library/busybox:latest  NA       30 hours ago  NA                              busybox-working-container
7a5a7b099d33  localhost/tom:latest              ls -alF  30 hours ago  Exited (0) 30 hours ago         hopeful_hellman
01d601fca090  localhost/tom:latest              ls -alf  30 hours ago  Exited (1) 30 hours ago         determined_panini
ee58f429ff26  localhost/tom:latest              NA       33 hours ago  NA                              alpine-working-container

podman ps --external
CONTAINER ID  IMAGE                             COMMAND  CREATED       STATUS  PORTS  NAMES
d302c81856da  docker.io/library/busybox:latest  NA       30 hours ago  NA             busybox-working-container
ee58f429ff26  localhost/tom:latest              NA       33 hours ago  NA             alpine-working-container

```
Signed-off-by: TomSweeneyRedHat <tsweeney@redhat.com>
@openshift-ci-robot openshift-ci-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jan 9, 2020
@TomSweeneyRedHat
Copy link
Member Author

I've just rebased this, no changes yet, so please don't review.

@rh-atomic-bot
Copy link
Collaborator

☔ The latest upstream changes (presumably #4953) made this pull request unmergeable. Please resolve the merge conflicts.

@openshift-ci-robot openshift-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jan 23, 2020
@openshift-ci-robot
Copy link
Collaborator

@TomSweeneyRedHat: PR needs rebase.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@vrothberg vrothberg removed the size/L label Jul 21, 2020
@rhatdan
Copy link
Member

rhatdan commented Aug 11, 2020

Replacing with #7290

@rhatdan rhatdan closed this Aug 11, 2020
@github-actions github-actions bot added the locked - please file new issue/PR Assist humans wanting to comment on an old issue or PR with locked comments. label Sep 24, 2023
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Sep 24, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. locked - please file new issue/PR Assist humans wanting to comment on an old issue or PR with locked comments. needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. stale-pr

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants