diff --git a/README.md b/README.md index a98dc4acd..90074eab4 100644 --- a/README.md +++ b/README.md @@ -5,9 +5,11 @@ This project is where the [Open Container Initiative](http://www.opencontainers. Table of Contents - [Filesystem Bundle](bundle.md) -- [Container Configuration](config.md) - - [Linux Specific Configuration](config-linux.md) -- [Runtime and Lifecycle](runtime.md) +- [Host-independent Container Configuration](config.md) + - [Linux-specific Configuration](config-linux.md) +- [Host-specific Container Configuration](runtime.md) + - [Linux-specific Configuration](runtime-linux.md) +- [Runtime and Lifecycle](lifecycle.md) - [Implementations](implementations.md) ## Use Cases diff --git a/bundle.md b/bundle.md index 58f4146f6..fed726511 100644 --- a/bundle.md +++ b/bundle.md @@ -19,6 +19,7 @@ The `runtime.json` file contains settings that are host specific such as memory The goal is that the bundle can be moved as a unit to another machine and run the same application if `runtime.json` is removed or reconfigured. The syntax and semantics for `config.json` are described in [this specification](config.md). +The syntax and semantics for `runtime.json` are described in [this specification](runtime.md). A single `rootfs` directory MUST be in the same directory as the `config.json`. The names of the directories may be arbitrary, but users should consider using conventional names as in the example below. diff --git a/config_linux.go b/config-linux.go similarity index 100% rename from config_linux.go rename to config-linux.go diff --git a/config-linux.md b/config-linux.md index aab976967..e695edef2 100644 --- a/config-linux.md +++ b/config-linux.md @@ -1,4 +1,4 @@ -# Linux-specific configuration +# Linux-specific, host-independent configuration The Linux container specification uses various kernel features like namespaces, cgroups, capabilities, LSM, and file system jails to fulfill the spec. diff --git a/config.md b/config.md index 00f24aa6f..41baab147 100644 --- a/config.md +++ b/config.md @@ -1,4 +1,4 @@ -# Configuration file +# Host-independent container configuration The container's top-level directory MUST contain a configuration file called `config.json`. For now the canonical schema is defined in [spec.go](spec.go) and [spec_linux.go](spec_linux.go), but this will be moved to a formal JSON schema over time. diff --git a/lifecycle-linux.md b/lifecycle-linux.md new file mode 100644 index 000000000..2393a8db1 --- /dev/null +++ b/lifecycle-linux.md @@ -0,0 +1,13 @@ +# Linux-specific lifecycle notes + +The [platform-independent lifecycle](lifecycle.md) has a few Linux-specific extensions. + +## Start (process) + +### File descriptors + +By default, only the `stdin`, `stdout` and `stderr` file descriptors are kept open for the application by the runtime. + +The runtime may pass additional file descriptors to the application to support features such as [socket activation](http://0pointer.de/blog/projects/socket-activated-containers.html). + +Some of the file descriptors may be redirected to `/dev/null` even though they are open. diff --git a/lifecycle.md b/lifecycle.md new file mode 100644 index 000000000..9eb6ebe8b --- /dev/null +++ b/lifecycle.md @@ -0,0 +1,42 @@ +# Lifecycle + +A typical lifecyle progresses like this: + +1. There is no container or running application +2. A user tells the runtime to [create](#create) a container +3. The runtime creates the container +4. A user tells the runtime to [start](#start-process) an application +5. The runtime executes any [pre-start hooks](runtime.md#pre-start) +6. The runtime executes the application +7. The application is running +8. A user tells the runtime to [stop](#stop) an application +9. The runtime sends a termination signal to the application +10. The application exits +11. The runtime executes any [post-stop hooks](runtime.md#post-stop) +12. A user tells the runtime to [destroy](#destroy) the container +13. The runtime removes the container + +With steps 8 and 9, the user is explicitly stopping the application +(via the runtime), but it's also possible that the application could +exit for other reasons. In that case we skip directly from 7 to 10. + +## Create + +Creates the container: file system, namespaces, cgroups, capabilities. + +## Start (process) + +Runs a process in a container. Can be invoked several times. +On Linux hosts, some information for this execution may come from outside the [`config.json`](config.md) and [`runtime.json`](runtime.md) specifications. +See [the Linux-specific notes](lifecycle-linux.md#start-process) for details. + +## Stop (process) + +Not sure we need that from runc cli. Process is killed from the outside. + +This event needs to be captured by runc to run onstop event handlers. + +## Destroy + +Remove the container: unmount file systems, remove namespaces, etc. +This is the inverse of [create](#create). diff --git a/runtime-config-linux.md b/runtime-config-linux.md deleted file mode 100644 index 1cabc6156..000000000 --- a/runtime-config-linux.md +++ /dev/null @@ -1,195 +0,0 @@ -## Namespaces - -A namespace wraps a global system resource in an abstraction that makes it appear to the processes within the namespace that they have their own isolated instance of the global resource. -Changes to the global resource are visible to other processes that are members of the namespace, but are invisible to other processes. -For more information, see [the man page](http://man7.org/linux/man-pages/man7/namespaces.7.html). - -Namespaces are specified in the spec as an array of entries. -Each entry has a type field with possible values described below and an optional path element. -If a path is specified, that particular file is used to join that type of namespace. - -```json - "namespaces": [ - { - "type": "pid", - "path": "/proc/1234/ns/pid" - }, - { - "type": "net", - "path": "/var/run/netns/neta" - }, - { - "type": "mnt", - }, - { - "type": "ipc", - }, - { - "type": "uts", - }, - { - "type": "user", - }, - ] -``` - -#### Namespace types - -* **pid** processes inside the container will only be able to see other processes inside the same container. -* **network** the container will have its own network stack. -* **mnt** the container will have an isolated mount table. -* **ipc** processes inside the container will only be able to communicate to other processes inside the same -container via system level IPC. -* **uts** the container will be able to have its own hostname and domain name. -* **user** the container will be able to remap user and group IDs from the host to local users and groups -within the container. - -### Access to devices - -Devices is an array specifying the list of devices to be created in the container. -Next parameters can be specified: - -* type - type of device: 'c', 'b', 'u' or 'p'. More info in `man mknod` -* path - full path to device inside container -* major, minor - major, minor numbers for device. More info in `man mknod`. - There is special value: `-1`, which means `*` for `device` - cgroup setup. -* permissions - cgroup permissions for device. A composition of 'r' - (read), 'w' (write), and 'm' (mknod). -* fileMode - file mode for device file -* uid - uid of device owner -* gid - gid of device owner - -```json - "devices": [ - { - "path": "/dev/random", - "type": "c", - "major": 1, - "minor": 8, - "permissions": "rwm", - "fileMode": 0666, - "uid": 0, - "gid": 0 - }, - { - "path": "/dev/urandom", - "type": "c", - "major": 1, - "minor": 9, - "permissions": "rwm", - "fileMode": 0666, - "uid": 0, - "gid": 0 - }, - { - "path": "/dev/null", - "type": "c", - "major": 1, - "minor": 3, - "permissions": "rwm", - "fileMode": 0666, - "uid": 0, - "gid": 0 - }, - { - "path": "/dev/zero", - "type": "c", - "major": 1, - "minor": 5, - "permissions": "rwm", - "fileMode": 0666, - "uid": 0, - "gid": 0 - }, - { - "path": "/dev/tty", - "type": "c", - "major": 5, - "minor": 0, - "permissions": "rwm", - "fileMode": 0666, - "uid": 0, - "gid": 0 - }, - { - "path": "/dev/full", - "type": "c", - "major": 1, - "minor": 7, - "permissions": "rwm", - "fileMode": 0666, - "uid": 0, - "gid": 0 - } - ] -``` - -## Control groups - -Also known as cgroups, they are used to restrict resource usage for a container and handle -device access. cgroups provide controls to restrict cpu, memory, IO, and network for -the container. For more information, see the [kernel cgroups documentation](https://www.kernel.org/doc/Documentation/cgroups/cgroups.txt). - -## Sysctl - -sysctl allows kernel parameters to be modified at runtime for the container. -For more information, see [the man page](http://man7.org/linux/man-pages/man8/sysctl.8.html) - -```json - "sysctl": { - "net.ipv4.ip_forward": "1", - "net.core.somaxconn": "256" - } -``` - -## Rlimits - -```json - "rlimits": [ - { - "type": "RLIMIT_NPROC", - "soft": 1024, - "hard": 102400 - } - ] -``` - -rlimits allow setting resource limits. The type is from the values defined in [the man page](http://man7.org/linux/man-pages/man2/setrlimit.2.html). The kernel enforces the soft limit for a resource while the hard limit acts as a ceiling for that value that could be set by an unprivileged process. - -## SELinux process label - -SELinux process label specifies the label with which the processes in a container are run. -For more information about SELinux, see [Selinux documentation](http://selinuxproject.org/page/Main_Page) -```json - "selinuxProcessLabel": "system_u:system_r:svirt_lxc_net_t:s0:c124,c675" -``` - -## Apparmor profile - -Apparmor profile specifies the name of the apparmor profile that will be used for the container. -For more information about Apparmor, see [Apparmor documentation](https://wiki.ubuntu.com/AppArmor) - -```json - "apparmorProfile": "acme_secure_profile" -``` - -## seccomp - -Seccomp provides application sandboxing mechanism in the Linux kernel. -Seccomp configuration allows one to configure actions to take for matched syscalls and furthermore also allows -matching on values passed as arguments to syscalls. -For more information about Seccomp, see [Seccomp kernel documentation](https://www.kernel.org/doc/Documentation/prctl/seccomp_filter.txt) -The actions and operators are strings that match the definitions in seccomp.h from [libseccomp](https://github.com/seccomp/libseccomp) and are translated to corresponding values. - -```json - "seccomp": { - "defaultAction": "SCMP_ACT_ALLOW", - "syscalls": [ - { - "name": "getcwd", - "action": "SCMP_ACT_ERRNO" - } - ] - } -``` diff --git a/runtime-config.md b/runtime-config.md deleted file mode 100644 index 6074e98bf..000000000 --- a/runtime-config.md +++ /dev/null @@ -1,54 +0,0 @@ -## Mount Configuration - -Additional filesystems can be declared as "mounts", specified in the *mounts* array. The parameters are similar to the ones in Linux mount system call. [http://linux.die.net/man/2/mount](http://linux.die.net/man/2/mount) - -* **type** (string, required) Linux, *filesystemtype* argument supported by the kernel are listed in */proc/filesystems* (e.g., "minix", "ext2", "ext3", "jfs", "xfs", "reiserfs", "msdos", "proc", "nfs", "iso9660"). Windows: ntfs -* **source** (string, required) a device name, but can also be a directory name or a dummy. Windows, the volume name that is the target of the mount point. \\?\Volume\{GUID}\ (on Windows source is called target) -* **destination** (string, required) where the source filesystem is mounted relative to the container rootfs. -* **options** (list of strings, optional) in the fstab format [https://wiki.archlinux.org/index.php/Fstab](https://wiki.archlinux.org/index.php/Fstab). - -*Example (Linux)* - -```json -"mounts": [ - { - "type": "proc", - "source": "proc", - "destination": "/proc", - "options": [] - }, - { - "type": "tmpfs", - "source": "tmpfs", - "destination": "/dev", - "options": ["nosuid","strictatime","mode=755","size=65536k"] - }, - { - "type": "devpts", - "source": "devpts", - "destination": "/dev/pts", - "options": ["nosuid","noexec","newinstance","ptmxmode=0666","mode=0620","gid=5"] - }, - { - "type": "bind", - "source": "/volumes/testing", - "destination": "/data", - "options": ["rbind","rw"] - } -] -``` - -*Example (Windows)* - -```json -"mounts": [ - { - "type": "ntfs", - "source": "\\\\?\\Volume\\{2eca078d-5cbc-43d3-aff8-7e8511f60d0e}\\", - "destination": "C:\\Users\\crosbymichael\\My Fancy Mount Point\\", - "options": [] - } -] -``` - -See links for details about [mountvol](http://ss64.com/nt/mountvol.html) and [SetVolumeMountPoint](https://msdn.microsoft.com/en-us/library/windows/desktop/aa365561(v=vs.85).aspx) in Windows. diff --git a/runtime_config_linux.go b/runtime-linux.go similarity index 100% rename from runtime_config_linux.go rename to runtime-linux.go diff --git a/runtime-linux.md b/runtime-linux.md index dcfa24eb4..d3bbb649d 100644 --- a/runtime-linux.md +++ b/runtime-linux.md @@ -1,6 +1,199 @@ -## File descriptors -By default, only the `stdin`, `stdout` and `stderr` file descriptors are kept open for the application by the runtime. +# Linux-specific, host-dependent configuration -The runtime may pass additional file descriptors to the application to support features such as [socket activation](http://0pointer.de/blog/projects/socket-activated-containers.html). +The Linux-specific section of the [host-dependent configuration](runtime.md). -Some of the file descriptors may be redirected to `/dev/null` even though they are open. +## Namespaces + +A namespace wraps a global system resource in an abstraction that makes it appear to the processes within the namespace that they have their own isolated instance of the global resource. +Changes to the global resource are visible to other processes that are members of the namespace, but are invisible to other processes. +For more information, see [the man page](http://man7.org/linux/man-pages/man7/namespaces.7.html). + +Namespaces are specified in the spec as an array of entries. +Each entry has a type field with possible values described below and an optional path element. +If a path is specified, that particular file is used to join that type of namespace. + +```json + "namespaces": [ + { + "type": "pid", + "path": "/proc/1234/ns/pid" + }, + { + "type": "net", + "path": "/var/run/netns/neta" + }, + { + "type": "mnt", + }, + { + "type": "ipc", + }, + { + "type": "uts", + }, + { + "type": "user", + }, + ] +``` + +#### Namespace types + +* **pid** processes inside the container will only be able to see other processes inside the same container. +* **network** the container will have its own network stack. +* **mnt** the container will have an isolated mount table. +* **ipc** processes inside the container will only be able to communicate to other processes inside the same +container via system level IPC. +* **uts** the container will be able to have its own hostname and domain name. +* **user** the container will be able to remap user and group IDs from the host to local users and groups +within the container. + +### Access to devices + +Devices is an array specifying the list of devices to be created in the container. +Next parameters can be specified: + +* type - type of device: 'c', 'b', 'u' or 'p'. More info in `man mknod` +* path - full path to device inside container +* major, minor - major, minor numbers for device. More info in `man mknod`. + There is special value: `-1`, which means `*` for `device` + cgroup setup. +* permissions - cgroup permissions for device. A composition of 'r' + (read), 'w' (write), and 'm' (mknod). +* fileMode - file mode for device file +* uid - uid of device owner +* gid - gid of device owner + +```json + "devices": [ + { + "path": "/dev/random", + "type": "c", + "major": 1, + "minor": 8, + "permissions": "rwm", + "fileMode": 0666, + "uid": 0, + "gid": 0 + }, + { + "path": "/dev/urandom", + "type": "c", + "major": 1, + "minor": 9, + "permissions": "rwm", + "fileMode": 0666, + "uid": 0, + "gid": 0 + }, + { + "path": "/dev/null", + "type": "c", + "major": 1, + "minor": 3, + "permissions": "rwm", + "fileMode": 0666, + "uid": 0, + "gid": 0 + }, + { + "path": "/dev/zero", + "type": "c", + "major": 1, + "minor": 5, + "permissions": "rwm", + "fileMode": 0666, + "uid": 0, + "gid": 0 + }, + { + "path": "/dev/tty", + "type": "c", + "major": 5, + "minor": 0, + "permissions": "rwm", + "fileMode": 0666, + "uid": 0, + "gid": 0 + }, + { + "path": "/dev/full", + "type": "c", + "major": 1, + "minor": 7, + "permissions": "rwm", + "fileMode": 0666, + "uid": 0, + "gid": 0 + } + ] +``` + +## Control groups + +Also known as cgroups, they are used to restrict resource usage for a container and handle +device access. cgroups provide controls to restrict cpu, memory, IO, and network for +the container. For more information, see the [kernel cgroups documentation](https://www.kernel.org/doc/Documentation/cgroups/cgroups.txt). + +## Sysctl + +sysctl allows kernel parameters to be modified at runtime for the container. +For more information, see [the man page](http://man7.org/linux/man-pages/man8/sysctl.8.html) + +```json + "sysctl": { + "net.ipv4.ip_forward": "1", + "net.core.somaxconn": "256" + } +``` + +## Rlimits + +```json + "rlimits": [ + { + "type": "RLIMIT_NPROC", + "soft": 1024, + "hard": 102400 + } + ] +``` + +rlimits allow setting resource limits. The type is from the values defined in [the man page](http://man7.org/linux/man-pages/man2/setrlimit.2.html). The kernel enforces the soft limit for a resource while the hard limit acts as a ceiling for that value that could be set by an unprivileged process. + +## SELinux process label + +SELinux process label specifies the label with which the processes in a container are run. +For more information about SELinux, see [Selinux documentation](http://selinuxproject.org/page/Main_Page) +```json + "selinuxProcessLabel": "system_u:system_r:svirt_lxc_net_t:s0:c124,c675" +``` + +## Apparmor profile + +Apparmor profile specifies the name of the apparmor profile that will be used for the container. +For more information about Apparmor, see [Apparmor documentation](https://wiki.ubuntu.com/AppArmor) + +```json + "apparmorProfile": "acme_secure_profile" +``` + +## seccomp + +Seccomp provides application sandboxing mechanism in the Linux kernel. +Seccomp configuration allows one to configure actions to take for matched syscalls and furthermore also allows +matching on values passed as arguments to syscalls. +For more information about Seccomp, see [Seccomp kernel documentation](https://www.kernel.org/doc/Documentation/prctl/seccomp_filter.txt) +The actions and operators are strings that match the definitions in seccomp.h from [libseccomp](https://github.com/seccomp/libseccomp) and are translated to corresponding values. + +```json + "seccomp": { + "defaultAction": "SCMP_ACT_ALLOW", + "syscalls": [ + { + "name": "getcwd", + "action": "SCMP_ACT_ERRNO" + } + ] + } +``` diff --git a/runtime_config.go b/runtime.go similarity index 100% rename from runtime_config.go rename to runtime.go diff --git a/runtime.md b/runtime.md index dbd055f04..c99a7c211 100644 --- a/runtime.md +++ b/runtime.md @@ -1,28 +1,17 @@ -# Runtime and Lifecycle +# Host-specific container configuration -## Lifecycle - -### Create - -Creates the container: file system, namespaces, cgroups, capabilities. - -### Start (process) - -Runs a process in a container. Can be invoked several times. - -### Stop (process) - -Not sure we need that from runc cli. Process is killed from the outside. - -This event needs to be captured by runc to run onstop event handlers. +The `runtime.json` file contains host-specific information needed to create containers and launch applications. +There are also platform-specific extensions for [Linux](runtime-linux.md). ## Hooks + Hooks allow one to run code before/after various lifecycle events of the container. The state of the container is passed to the hooks over stdin, so the hooks could get the information they need to do their work. Hook paths are absolute and are executed from the host's filesystem. ### Pre-start + The pre-start hooks are called after the container process is spawned, but before the user supplied command is executed. They are called after the container namespaces are created on Linux, so they provide an opportunity to customize the container. In Linux, for e.g., the network namespace could be configured in this hook. @@ -30,6 +19,7 @@ In Linux, for e.g., the network namespace could be configured in this hook. If a hook returns a non-zero exit code, then an error including the exit code and the stderr is returned to the caller and the container is torn down. ### Post-stop + The post-stop hooks are called after the container process is stopped. Cleanup or debugging could be performed in such a hook. If a hook returns a non-zero exit code, then an error is logged and the remaining hooks are executed. @@ -57,3 +47,58 @@ If a hook returns a non-zero exit code, then an error is logged and the remainin ``` `path` is required for a hook. `args` and `env` are optional. + +## Mount Configuration + +Additional filesystems can be declared as "mounts", specified in the *mounts* array. The parameters are similar to the ones in Linux mount system call. [http://linux.die.net/man/2/mount](http://linux.die.net/man/2/mount) + +* **type** (string, required) Linux, *filesystemtype* argument supported by the kernel are listed in */proc/filesystems* (e.g., "minix", "ext2", "ext3", "jfs", "xfs", "reiserfs", "msdos", "proc", "nfs", "iso9660"). Windows: ntfs +* **source** (string, required) a device name, but can also be a directory name or a dummy. Windows, the volume name that is the target of the mount point. \\?\Volume\{GUID}\ (on Windows source is called target) +* **destination** (string, required) where the source filesystem is mounted relative to the container rootfs. +* **options** (list of strings, optional) in the fstab format [https://wiki.archlinux.org/index.php/Fstab](https://wiki.archlinux.org/index.php/Fstab). + +*Example (Linux)* + +```json +"mounts": [ + { + "type": "proc", + "source": "proc", + "destination": "/proc", + "options": [] + }, + { + "type": "tmpfs", + "source": "tmpfs", + "destination": "/dev", + "options": ["nosuid","strictatime","mode=755","size=65536k"] + }, + { + "type": "devpts", + "source": "devpts", + "destination": "/dev/pts", + "options": ["nosuid","noexec","newinstance","ptmxmode=0666","mode=0620","gid=5"] + }, + { + "type": "bind", + "source": "/volumes/testing", + "destination": "/data", + "options": ["rbind","rw"] + } +] +``` + +*Example (Windows)* + +```json +"mounts": [ + { + "type": "ntfs", + "source": "\\\\?\\Volume\\{2eca078d-5cbc-43d3-aff8-7e8511f60d0e}\\", + "destination": "C:\\Users\\crosbymichael\\My Fancy Mount Point\\", + "options": [] + } +] +``` + +See links for details about [mountvol](http://ss64.com/nt/mountvol.html) and [SetVolumeMountPoint](https://msdn.microsoft.com/en-us/library/windows/desktop/aa365561(v=vs.85).aspx) in Windows.