Cgroups: can't use cpuset with cgexec [solved]

On the web i saw some mention of cgoup-tools. So i had a go at it.

cgroups is a control group feature that lets you manage and limit process groups in linux, in case you did not know. Super awesome when it works. Trust me. :)

Problem is i can't use cpuset. Which is a nice subsystem of cgroups that lets you pin groups to specific cores.
But i can use the cpu subsystem without any problem. So i can set e.g. cpu.shares, but nothing in cpuset.

Running Debian 9 (testing) and doing the following steps:

sudo apt install cgroup-tools # installs the package
sudo cgcreate -t YOUR_USER:YOUR_USER -a YOUR_USER:YOUR_USER  -g cpuset:testgroup # creating the group testgroup, YOUR_USER is obviously replaced with my user
ls -al /sys/fs/cgroup/cpuset/testgroup/ # testing if i own the group, i do
cgexec -g cpuset:testgroup sleep 600 # trying to run a command under my user withing the testgroup group

When running cgexec i get this error:

cgroup change of group failed

Which is weird because i own all the files in that group. And i also get that error when i run everything as root. So there shouldn't be a permission error.

When running

strace cgexec -g cpuset:testgroup "sleep 6000"  
[…]
geteuid()                               = 1000
getegid()                               = 1000
getuid()                                = 1000
getgid()                                = 1000
socket(AF_UNIX, SOCK_STREAM, 0)         = 3
connect(3, {sa_family=AF_UNIX, sun_path="/var/run/cgred.socket"}, 23) = -1 ENOENT (No such file or directory)
close(3)                                = 0
setresuid(1000, 1000, 1000)             = 0
setresgid(1000, 1000, 1000)             = 0
open("/sys/fs/cgroup/cpuset/testgroup//tasks", O_WRONLY|O_CREAT|O_TRUNC|O_CLOEXEC, 0666) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=0, ...}) = 0
write(3, "20253", 5)                    = -1 ENOSPC (No space left on device)
close(3)                                = 0
write(2, "cgroup change of group failed\n", 30cgroup change of group failed
) = 30
exit_group(50016)                       = ?
+++ exited with 96 +++

I don't understand why there is no space on /sys/ and only for that file. It does not make sense.

   file /sys/fs/cgroup/cpuset/testgroup//tasks      
   /sys/fs/cgroup/cpuset/testgroup//tasks: empty
   ls -aln /sys/fs/cgroup/cpuset/testgroup//tasks 
   -rw-r--r-- 1 1000 1000 0 Mär 19 01:28 /sys/fs/cgroup/cpuset/testgroup//tasks

When using only the cpu subsystem:

 sudo cgcreate -t $USER:$USER -a $USER:$USER  -g cpu:testgroupcpu 
 ls -lna /sys/fs/cgroup/cpu/testgroupcpu    
drwxr-xr-x 2 1000 1000 0 Mär 19 01:33 ./
dr-xr-xr-x 6    0    0 0 Mär 19 01:33 ../
-rw-r--r-- 1 1000 1000 0 Mär 19 01:33 cgroup.clone_children
-rw-r--r-- 1 1000 1000 0 Mär 19 01:33 cgroup.procs
-r--r--r-- 1 1000 1000 0 Mär 19 01:33 cpuacct.stat
-rw-r--r-- 1 1000 1000 0 Mär 19 01:33 cpuacct.usage
-r--r--r-- 1 1000 1000 0 Mär 19 01:33 cpuacct.usage_all
-r--r--r-- 1 1000 1000 0 Mär 19 01:33 cpuacct.usage_percpu
-r--r--r-- 1 1000 1000 0 Mär 19 01:33 cpuacct.usage_percpu_sys
-r--r--r-- 1 1000 1000 0 Mär 19 01:33 cpuacct.usage_percpu_user
-r--r--r-- 1 1000 1000 0 Mär 19 01:33 cpuacct.usage_sys
-r--r--r-- 1 1000 1000 0 Mär 19 01:33 cpuacct.usage_user
-rw-r--r-- 1 1000 1000 0 Mär 19 01:33 cpu.cfs_period_us
-rw-r--r-- 1 1000 1000 0 Mär 19 01:33 cpu.cfs_quota_us
-rw-r--r-- 1 1000 1000 0 Mär 19 01:33 cpu.shares
-r--r--r-- 1 1000 1000 0 Mär 19 01:33 cpu.stat
-rw-r--r-- 1 1000 1000 0 Mär 19 01:33 notify_on_release
-rw-r--r-- 1 1000 1000 0 Mär 19 01:33 tasks

cgexec -g cpu:testgroupcpu sleep 6000 # WORKS!

Any ideas or suggestions?

Turns out cgcreate leaves the files empty and that this the problem.

Solution: https://serverfault.com/questions/579555/cgroup-no-space-left-on-device#579598

This fixes the error:

echo 0 > cpuset.cpus
echo 0 > cpuset.mems
1 Like