How to view log of a container created by runc?

[nqbao]

Hi,

I was searching around a lot before asking this question. Once I create a detached runc container, how do i retrieve the log (stdout, stderr) of the container? In docker, I would use docker logs but i don't know how to do this with runc yet.

Please advise.

Thanks!

[cyphar]

runc doesn't have this feature natively (there is no runc logs command). However, you can make your own logging system by either having a small wrapper around runc or by disabling the terminal and then setting the stdio FDs of the runc create (or runc run) process -- these will be forwarded to the container's stdio.

The reason why runc doesn't have this feature natively is for two reasons:

1. By having more raw access to the related file descriptors, users have more control over the stdio of a container. This is unlike Docker, where if you wanted to have something more complicated than just "create a pty for me" or "don't create a pty at all", you're out of luck.

2. If you have a logging system you need to have a daemon (which is counter to runc's design principles) -- either for multiplexing the logs to a file while also outputting them to the user, or a management program that keeps logs truncated.

[nqbao]

Hi @cyphar ,

Thanks for your reply. Do you mean that i can use preserve-fds to redirect the stdio to a file right? Would you mind providing a sample?

Thanks!

[nqbao]

I tried the following thing but it does not work ...

# open 512 for log.txt
exec 512<> log.txt

# make sure it works
echo test >&512
cat log.txt


# redirect stdout stderr to log.txt
runc run --preserve-fds 0 --preserve-fds 512 --preserve-fds 512 test

The error is

container_linux.go:265: starting container process caused "process_linux.go:244: starting init process command caused \"fork/exec /proc/self/exe: bad file descriptor\""

[nqbao]

After digging into the code, you can redirect the stdio as you do in a normal bash script if you disable terminal in the config.json and you have to run in detach mode. So the below will work

runc -d run test > log.txt 2>&1

The 2nd is quite confusing for me. Because usually you don't do IO redirection for detached process. I think it would be a lot better if I can use flag to redirect the log to a file.

[cyphar]

@nqbao

The semantics you've just described is what I meant when I said:

or by disabling the terminal and then setting the stdio FDs of the runc create (or runc run) process -- these will be forwarded to the container's stdio.

Now, the reason why this is the case is quite complicated (and you've stepped into quite a long and complicated history here). I'll try to explain this as best I can.

One of the key features (and design constraints) of runc is that we do not have any daemons at all. No long-running processes are used in the maintenance of containers. While this is nice from the point of view of people who build stuff on top of runc (namely they don't need to care about what our code does after the container has been started), it does mean that certain APIs are a bit stranger than what you're probably used to. This is because our constraints change what we can reasonably do.

If you run a container in detached mode, then all of the IO has to be handled by the user of runc. In the case of "terminal": true this means that you have to get the master PTY through --console-socket and then you're on your own for managing the container process's IO. In the case of "terminal": false you have to specify what file descriptors are going to be used by the container process for stdio. In order for the most common usecase to work out-of-the-box (people running runc from their terminal), the API for specifying what file descriptors are going to be used by the container process is that the file descriptors of runc are inherited.

The idea is that this is semantically equivalent to when you run most programs that exec (the file descriptors are maintained across fork and exec).

Because usually you don't do IO redirection for detached process.

"Detach" is probably not the right choice of words on our part. When you run a container in "detached" mode (or you start the container with runc create -- which is semantically similar when it comes to IO), you're actually saying "the container runtime will detach itself from the lifecycle of the container". However, the process does act like a detached process (in some ways).

But this really is a fairly common source of confusion for people, so I'm wondering if there's a better way of explaining this. My main problem with proposals like this:

I think it would be a lot better if I can use flag to redirect the log to a file.

Is that having a --log-file API would be less powerful than the current API. Currently you can make the file descriptors used for the container anything (even network sockets if you're feeling particularly crazy), but a --log-file API would restrict you to file paths. Even with /proc/self/fd/... you couldn't emulate something like using network sockets -- because you cannot open(2) a non-regular-file /proc/self/fd/....

---

I also wanted to mention that your usage of --preserve-fds wasn't entirely correct -- that flag specifies that you want to inherit every file descriptor up to and including the one you specify. So it doesn't make sense to specify --preserve-fds 0 --preserve-fds 512. Also, the stdio file descriptors (0 through 2) are always preserved.

[nqbao]

Hi @cyphar ,

Thank for your very detail response. I understand the reason why the cli interface is designed this way. Actually, if I approach at library level and use libcontainer then the API makes a lot of sense (process.Stdin, process.Stdout, ...). Only at the CLI level it is a bit confusing for me. IMO, the audience of the library and the cli should be different, for example I am used to file, path and socket in the terminal than the file descriptor.

Anyway, this is great and thanks again for guiding me through the ticket!

[cyphar]

Well, I also happen to think the libcontainer interface is even more confusing (because it's trying to mimic os.Process -- which doesn't make sense since containers have different semantics especially when you're trying to join one as opposed to creating one).

[cenfei]

hello , i don not konw ,how to set runc log path? can you help me ?