add new generate options for setting cpu/mem #175

haiyanmeng · 2016-08-02T18:59:24Z

Signed-off-by: Haiyan Meng [email protected]

haiyanmeng · 2016-08-02T18:59:54Z

haiyanmeng · 2016-08-02T19:00:24Z

The test command is here:

ocitools generate --output config.json --cpu-shares 512 \
   --cpu-period 10000 \
   --cpu-quota 20000 \
   --realtime-runtime 35000000 \
   --realtime-period 343439999 \
   --cpus 1-3 \
   --mems 0 \
   --mem-limit 999000000 \
   --mem-reservation 6660000 \
   --mem-swap 1000000000 \
   --mem-kernel-limit 450000000 \
   --mem-kernel-tcp 3000000 \
   --mem-swappiness 30

wking · 2016-08-02T19:03:12Z

man/ocitools-generate.1.md

@@ -234,6 +234,45 @@ inside of the container.
  The special *PATH* `host` removes any existing UTS namespace from
  the configuration.

+**--cpu-shares**=CPUSHARES
+  Sets the CPU shares (relative weight (ratio) vs. other cgroups with cpu shares).


Instead of having two descriptions to discuss, I'd rather just copy the runtime-spec wording verbatim here. For this setting, that's:

Specifies a relative share of CPU time available to the tasks in a cgroup.

@wking , thanks for pointing out.
In fact, the current description I am using is from https://github.com/opencontainers/runtime-spec/blob/v1.0.0-rc1/specs-go/config.go#L272 .
Is there any plan on runtime-spec side to sync the descriptions on the config-linux.md and config.go?

On Tue, Aug 02, 2016 at 12:28:18PM -0700, hmeng-19 wrote:

In fact, the current description I am using is from
https://github.com/opencontainers/runtime-spec/blob/v1.0.0-rc1/specs-go/config.go#L272
. Is there any plan on runtime-spec side to sync the descriptions
on the config-linux.md and config.go?

Heh. I'm not aware of a plan, but I'll file a PR updating the Go
comments this evening if nobody beats me to it. And if you feel moved
to PR the update, don't wait for me ;).

@wking , go ahead with the runtime-spec PR to fix it.

wking · 2016-08-02T19:07:32Z

On Tue, Aug 02, 2016 at 12:00:25PM -0700, hmeng-19 wrote:

The test command is here:

As ‘ocitools generate’ grows, it's probalby worth adding a
bats/sharness/… test suite locally to exercise all these options for
us. I think we've passed that tipping point already, and am happy to
PR something if ocitools maintainers agree (and have a test-suite
preference, otherwise I'll probably use sharness).

haiyanmeng · 2016-08-02T20:19:00Z

@wking , @mrunalp , PTAL.

wking · 2016-08-02T20:33:37Z

man/ocitools-generate.1.md

@@ -234,6 +234,45 @@ inside of the container.
  The special *PATH* `host` removes any existing UTS namespace from
  the configuration.

+**--cpu-shares**=CPUSHARES


Options in this man page are currently alphabetized (since #44). That's an easier pattern to follow than grouping related options.

@wking , will fix it. Thanks.

It may be worth using --linux-cpu-shares here to leave room for other platforms with their own interpretation of CPU shares. That will help avoid confusion when someone PRs --solaris-ncpus, etc. It's possible that there is common ground here and we could have a cross-platform --cpu-shares option, but if that's the case I'd rather have runtime-spec leading the push towards unification.

@mrunalp , how to think about this?

mrunalp · 2016-08-02T21:34:07Z

@wking 👍 to a test harness. I don't care what it is as long as we have something 😉

haiyanmeng · 2016-08-05T15:24:28Z

@wking , @mrunalp , PTAL.
Rename the options to be --linux-*;
Reorder the options in man/ocitools-generate.1.md.

haiyanmeng · 2016-08-05T15:25:06Z

The new test command is:

ocitools generate --output config.json \
   --linux-cpu-shares 512 \
   --linux-cpu-period 10000 \
   --linux-cpu-quota 20000 \
   --linux-realtime-runtime 35000000 \
   --linux-realtime-period 343439999 \
   --linux-cpus 1-3 \
   --linux-mems 0 \
   --linux-mem-limit 999000000 \
   --linux-mem-reservation 6660000 \
   --linux-mem-swap 1000000000 \
   --linux-mem-kernel-limit 450000000 \
   --linux-mem-kernel-tcp 3000000 \
   --linux-mem-swappiness 30

mrunalp · 2016-08-05T16:56:15Z

Needs rebase?

haiyanmeng · 2016-08-05T17:26:06Z

@mrunalp , rebased it. PTAL.

wking · 2016-08-05T17:45:14Z

man/ocitools-generate.1.md

+  Sets the limit of memory usage in bytes.
+
+**--linux-mem-reservation**=MEMRESERVATION
+  sets the soft limit of memory usage in bytes.


“sets” → “Sets”

wking · 2016-08-05T18:11:33Z

man/ocitools-generate.1.md

+  Specifies a relative share of CPU time available to the tasks in a cgroup.
+
+**--linux-cpu-period**=CPUPERIOD
+  Specifies a period of time in microseconds for how regularly a cgroup's access to CPU resources should be reallocated (CFS scheduler only).


“CFS only” seems like we want a --host-specific guard to error out if someone tries to set it with --host-specific when they aren't using a CFS scheduler. But after reading over sched(7) and sched_setscheduler(2), I'm not quite sure how to do that. It looks like you check for a SCHED_OTHER policy, but sched_getscheduler is per-thread. Is the scheduler policy really set on a per-thread basis? Is a host-specific check just supposed to look for internal consistency (e.g. the config doesn't set both a CFS-only and a realtime-only parameter)?

@wking , I think there are two ways we can use to determine the current scheduler.

Read the kernel config file, refer to https://en.wikipedia.org/wiki/Completely_Fair_Scheduler:

[hmeng@localhost boot]$ grep CONFIG_SCHED_AUTOGROUP /boot/config-4.6.4-201.fc23.x86_64 CONFIG_SCHED_AUTOGROUP=y

check the /sys/fs/cgroup/cpu dir to see whether cpu.cfs_period_us exists or not.

On Fri, Aug 05, 2016 at 11:58:48AM -0700, hmeng-19 wrote:

@wking , I think there are two ways we can use to determine the current scheduler.

Read the kernel config file, refer to https://en.wikipedia.org/wiki/Completely_Fair_Scheduler:

[hmeng@localhost boot]$ grep CONFIG_SCHED_AUTOGROUP /boot/config-4.6.4-201.fc23.x86_64 CONFIG_SCHED_AUTOGROUP=y

check the /sys/fs/cgroup/cpu dir to see whether cpu.cfs_period_us exists or not.

These both sound like “does your kernel support that scheduler”, but
if scheduler policy is per-thread, then how do you know the container
will use that available scheduler (and not another available
scheduler)?

@wking , @mrunalp , do correct me if I am wrong.
I always thought the scheduling policy is a system-level config. It is decided when you built your kernel. Right?

On Fri, Aug 05, 2016 at 08:25:27PM -0700, hmeng-19 wrote:

+--linux-cpu-period=CPUPERIOD

Specifies a period of time in microseconds for how regularly a cgroup's access to CPU resources should be reallocated (CFS scheduler only).

@wking , @mrunalp , do correct me if I am wrong.
I always thought the scheduling policy is a system-level config. It
is decided when you built your kernel. Right?

That was my impression before poking around in the man pages today.
But see also 1, which has:

Each process or thread is controlled by an associated scheduling
policy and priority…

And 2 has:

If at least one RT task is running, no other SCHED_OTHER task will
be allowed to run in any CPU.

which makes it fairly clear that you can have threads/processes with
different schedulers running concurrently. I'm not sure how that
works for the cgroup settings though. Maybe you can set them all
(both real-time and CFS parameters) and the kernel will apply the
appropriate limits to each task depending on its policy? In that
case, configs that specify settings for multiple schedulers are fair
game.

mrunalp · 2016-08-08T16:48:00Z

LGTM

Mashimiao · 2016-09-06T08:06:56Z

@hmeng-19 seems need rebase

Signed-off-by: Haiyan Meng <[email protected]>

haiyanmeng · 2016-09-06T15:06:55Z

@Mashimiao , @mrunalp , PTAL.

mrunalp · 2016-09-06T17:26:53Z

LGTM

Signed-off-by: Haiyan Meng <[email protected]> Backported to v1.0.0.rc1 from 092cb7c opencontainers#175 (cherry-pick applied cleanly). Signed-off-by: W. Trevor King <[email protected]>

wking reviewed Aug 2, 2016
View reviewed changes

haiyanmeng force-pushed the add_generate_options branch from 1fd83a3 to 71a9fea Compare August 2, 2016 20:18

wking reviewed Aug 2, 2016
View reviewed changes

This was referenced Aug 3, 2016

test: Stub in sharness framework #180

Open

config: Synchronize comments between Markdown and Go opencontainers/runtime-spec#525

Merged

haiyanmeng mentioned this pull request Aug 5, 2016

Configure memory swapiness for a container #111

Closed

haiyanmeng force-pushed the add_generate_options branch from 71a9fea to 8ff81a1 Compare August 5, 2016 15:23

haiyanmeng force-pushed the add_generate_options branch from 8ff81a1 to b812c1b Compare August 5, 2016 17:24

wking reviewed Aug 5, 2016
View reviewed changes

haiyanmeng force-pushed the add_generate_options branch from b812c1b to 84d9c19 Compare August 5, 2016 17:47

wking reviewed Aug 5, 2016
View reviewed changes

add new generate options for setting cpu/mem

092cb7c

Signed-off-by: Haiyan Meng <[email protected]>

haiyanmeng force-pushed the add_generate_options branch from 84d9c19 to 092cb7c Compare September 6, 2016 15:06

mrunalp merged commit 089d971 into opencontainers:master Sep 6, 2016

wking mentioned this pull request Sep 6, 2016

[1.0.0.rc1] add new generate options for setting cpu/mem #203

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

add new generate options for setting cpu/mem #175

add new generate options for setting cpu/mem #175

haiyanmeng commented Aug 2, 2016

haiyanmeng commented Aug 2, 2016

haiyanmeng commented Aug 2, 2016

wking Aug 2, 2016

haiyanmeng Aug 2, 2016

wking Aug 2, 2016

haiyanmeng Aug 2, 2016

wking commented Aug 2, 2016

haiyanmeng commented Aug 2, 2016

wking Aug 2, 2016

haiyanmeng Aug 2, 2016

wking Aug 2, 2016

haiyanmeng Aug 3, 2016

mrunalp commented Aug 2, 2016

haiyanmeng commented Aug 5, 2016

haiyanmeng commented Aug 5, 2016

mrunalp commented Aug 5, 2016

haiyanmeng commented Aug 5, 2016

wking Aug 5, 2016

haiyanmeng Aug 5, 2016

wking Aug 5, 2016

haiyanmeng Aug 5, 2016

wking Aug 5, 2016

haiyanmeng Aug 6, 2016

wking Aug 6, 2016

mrunalp commented Aug 8, 2016

Mashimiao commented Sep 6, 2016

haiyanmeng commented Sep 6, 2016

mrunalp commented Sep 6, 2016

add new generate options for setting cpu/mem #175

add new generate options for setting cpu/mem #175

Conversation

haiyanmeng commented Aug 2, 2016

haiyanmeng commented Aug 2, 2016

haiyanmeng commented Aug 2, 2016

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

wking commented Aug 2, 2016

haiyanmeng commented Aug 2, 2016

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mrunalp commented Aug 2, 2016

haiyanmeng commented Aug 5, 2016

haiyanmeng commented Aug 5, 2016

mrunalp commented Aug 5, 2016

haiyanmeng commented Aug 5, 2016

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mrunalp commented Aug 8, 2016

Mashimiao commented Sep 6, 2016

haiyanmeng commented Sep 6, 2016

mrunalp commented Sep 6, 2016