Skip to content

Commit

Permalink
validation/linux_cgroups_*hugetlb: Use smaller limits
Browse files Browse the repository at this point in the history
The previous values were giving me:

  container_linux.go:348: starting container process caused
    "process_linux.go:402: container init caused
    \"process_linux.go:367: setting cgroup config for procHooks
    process caused \\\"failed to write 56892210544640 to
    hugetlb.1GB.limit_in_bytes: open
    /sys/fs/cgroup/hugetlb/.../hugetlb.1GB.limit_in_bytes: permission
    denied\\\"\""

The previous values are originally from 432615a (add cgroup hugetlb
test for runtime, 2017-12-05, #93), which doesn't motivate their
choice.  The new values are copy/pasted from the spec [1] (which
doesn't motivate its choice either ;).  I've kept something like
Alban's comment from 984dbc8 (Fix error messages in validation cgroup
tests, 2018-03-14, #605) to at least explain how the limit breaks
down.

In testing with my local system, the issue seems to be pageSize and
not the limit value.  That seems to be supported by the kernel docs,
which have [2]:

  hugepages=  [HW,X86-32,IA-64] HugeTLB pages to allocate at boot.
  hugepagesz= [HW,IA-64,PPC,X86-64] The size of the HugeTLB pages.
      On x86-64 and powerpc, this option can be specified
      multiple times interleaved with hugepages= to reserve
      huge pages of different sizes. Valid pages sizes on
      x86-64 are 2M (when the CPU supports "pse") and 1G
      (when the CPU supports the "pdpe1gb" cpuinfo flag).

My CPU supports both:

  $ cat /proc/cpuinfo | grep '^flags' | head -n1 | grep -o ' \(pse\|pdpe1gb\) '
   pse
   pdpe1gb

but I don't set hugepagesz, and I seem to only get 2M by default.  I
can get 1GB entries by booting with hugepagesz=1GB.  Longer-term, we
may want to auto-detect the value(s) currently enabled by the host
system, but for this commit I'm hard-coding 2MB.

[1]: https://github.com/opencontainers/runtime-spec/blob/v1.0.1/config-linux.md#example-8
[2]: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/Documentation/admin-guide/kernel-parameters.txt?h=v4.16#n1336

Signed-off-by: W. Trevor King <[email protected]>
  • Loading branch information
wking committed Apr 7, 2018
1 parent fcd5b23 commit f25a97d
Show file tree
Hide file tree
Showing 2 changed files with 6 additions and 4 deletions.
5 changes: 3 additions & 2 deletions validation/linux_cgroups_hugetlb.go
Original file line number Diff line number Diff line change
Expand Up @@ -9,8 +9,9 @@ import (
)

func main() {
page := "1GB"
var limit uint64 = 52985 * 1024 * 1024 * 1024 // multiple of hugepage size
page := "2MB"
var pageSize uint64 = 2 * 1024 * 1024 // 2MB in bytes
limit := 100 * pageSize
g := util.GetDefaultGenerator()
g.SetLinuxCgroupsPath(cgroups.AbsCgroupPath)
g.AddLinuxResourcesHugepageLimit(page, limit)
Expand Down
5 changes: 3 additions & 2 deletions validation/linux_cgroups_relative_hugetlb.go
Original file line number Diff line number Diff line change
Expand Up @@ -8,8 +8,9 @@ import (
)

func main() {
page := "1GB"
var limit uint64 = 56892210544640
page := "2MB"
var pageSize uint64 = 2 * 1024 * 1024 // 2MB in bytes
limit := 100 * pageSize
g := util.GetDefaultGenerator()
g.SetLinuxCgroupsPath(cgroups.RelCgroupPath)
g.AddLinuxResourcesHugepageLimit(page, limit)
Expand Down

0 comments on commit f25a97d

Please sign in to comment.