This is an old revision of the document!
Table of Contents
CPU shielding using /proc and /dev/cpuset
The goal of CPU shielding is to have one or more CPUs isolated/shielded from implicit disruption by the kernel and have the processors available to run undisturbed specific tasks.
Mainly, the CPUs are shielded against interrupts and the scheduler.
Interrupt shielding
User Space
The goal of CPU shielding is to have one or more CPUs isolated/shielded from implicit disruption, mainly interrupts and the kernel schedules and have those CPUs ready to run exclusively certain tasks (in this particular case, real time tasks). In order to do this we make sure that the interrupts are not automatically balanced, the irqbalance daemon needs to be disabled. This daemon is started from the irqbalance init script. To disable once do:
$ /etc/init.d/irqbalance stop
To disable after next reboot do:
$ chkconfig irqbalance off
After this you can change the CPU affinity mask of each interrupt by doing:
$ echo hex_mask > /proc/irq/<irq_number>/smp_affinity
To check that the affinity mask has been set you can check the contents of the smp_affinity file.
The mask is updated the next time an interrupt is serviced. So you may not see the change immediately.
More information can be found in IRQ affinity
Process shielding
The kernel has an cpuset feature that allows you to create cpusets for real-time purposes. The kernel interface is proc filesystem based. It is described in cpusets. Each cpuset is represented by a directory in the cgroup file system containing (on top of the standard cgroup files) the following files describing that cpuset:
- cpus: list of CPUs in that cpuset
- mems: list of Memory Nodes in that cpuset
- memory_migrate flag: if set, move pages to cpusets nodes
- cpu_exclusive flag: is cpu placement exclusive?
- mem_exclusive flag: is memory placement exclusive?
- mem_hardwall flag: is memory allocation hardwalled
- memory_pressure: measure of how much paging pressure in cpuset
- memory_spread_page flag: if set, spread page cache evenly on allowed nodes
- memory_spread_slab flag: if set, spread slab cache evenly on allowed nodes
- sched_load_balance flag: if set, load balance within CPUs on that cpuset
- sched_relax_domain_level: the searching range when migrating tasks
In addition, the root cpuset only has the following file:
- memory_pressure_enabled flag: compute memory_pressure?
Here is a quick example of how to use cpuset to reserve one cpu for your real-time process on a 4 cpu machine:
$ mkdir /dev/cpuset/rt0 $ echo 0 > /dev/cpuset/rt0/cpus $ echo 0 > /dev/cpuset/rt0/mems $ echo 1 > /dev/cpuset/rt0/cpu_exclusive $ echo $RT_PROC_PID > /dev/cpuset/rt0/tasks $ mkdir /dev/cpuset/system $ echo 1-3 > /dev/cpuset/system/cpus $ echo 0 > /dev/cpuset/system/mems $ echo 1 > /dev/cpuset/system/cpu_exclusive $ for pid in $(cat /dev/cpuset/tasks); do /bin/echo $pid > /dev/cpuset/system/tasks; done
