Home

Sources of CPU interference in core Linux code

For the the purpose of this document, CPU interference is any case where kernel management function will take CPU time from a pure CPU bound user space task making no system call running pinned to an isolated CPU with no other contending user tasks. Specifically, any kernel activity that occurred as a result of the the specific user task activity, is not considered interference, but kernel activity on the specific CPU happening as a result of work of other tasks or the general kernel infrastructure are considered interference.

Global IPIs

1. fs/buffer.c - Global IPI for each LRU drain [1]
2. kernel/hrtimer - Global IPI for setting high res-timer when system clock is changed [2]
3. kernel/profile.c - double flip buffer management, used during profiling only [2]
4. kernel/rcutree.c - rcu_barrier() implementation uses a global IPI to queue an RCU callback on every CPU
5. mm/page_alloc.c - global IPI for draining all per-cpu pages [1] [4]
6. mm/slab.c - global IPI during per cpu cache drain and tuning [3]
7. mm/slub.c - global IPI during per cpu caches drain, called when destroying a cache (also does an rcu_barrier) [1]
8. net/core/dev.c - global IPI to flush backlog during net device unregister (also does an rcu_barrier) [2]

Global work queue scheduling

1. mm/swap.c - global work scheduled on each cpu for draining pagevecs to LRU lists [5]
2. mm/slab.c - delayed work scheduled one per second to reap per cpu slab cache
3. mm/vmstat.c - delayed work scheduled one per second for pcp cache draining and VM statistics

Global kthread scheduling

?

Scheduler tick

1. Scheduler, run queue time keeping, timer management, RCU [6]

Notes

1. Dealt with in Reduce cross CPU IPI interference patch set (https://lkml.org/lkml/2012/1/8/109)
2. Does not occur during normal production run time
3. Since slab and slub are mutually exclusive it is enough to deal with one of them
4. Made a very rare event by the Mel Gorman's patch to not do a global drain for direct reclaim path of memory allocator (https://lkml.org/lkml/2012/1/11/93)
5. Might be dealt with by the effort to get rid of pagevecs (see: https://lkml.org/lkml/2012/1/4/376) or we can do schedule_on_each_cpu_mask(...)
6. Dealt with by Frederic Weisbecker NOHZ cpuset patchset (see: http://lwn.net/Articles/455044/)