Home

Sources of cross CPU interference in core Linux code

Global IPIs

1. fs/buffer.c - Global IPI for each LRU drain [1]
2. kernel/hrtimer - Global IPI for setting high res-timer when system clock is changed [2]
3. kernel/profile.c - double flip buffer management, used during profiling only [2]
4. kernel/rcutree.c - rcu_barrier() implementation uses a global IPI to queue an RCU callback on every CPU
5. mm/page_alloc.c - global IPI for draining all per-cpu pages [1] [4]
6. mm/slab.c - global IPI during per cpu cache drain and tuning [3]
7. mm/slub.c - global IPI during per cpu caches drain, called when destroying a cache (also does an rcu_barrier) [1]
8. net/core/dev.c - global IPI to flush backlog during net device unregister (also does an rcu_barrier) [2]

Global work queue scheduling

1. mm/swap.c - global work scheduled on each cpu for draining pagevecs to LRU lists [5]
2. mm/slab.c - delayed work scheduled one per second to reap per cpu slab cache
3. mm/vmstat.c - delayed work scheduled one per second for pcp cache draining and VM statistics

Global kthread scheduling

?

Notes

1. Dealt with in Reduce cross CPU IPI interference patch set (https://lkml.org/lkml/2012/1/8/109)
2. Does not occur during normal production run time
3. Since slab and slub are mutually exclusive it is enough to deal with one of them
4. Made a very rare event by the Mel Gorman's patch to not do a global drain for direct reclaim path of memory allocator (https://lkml.org/lkml/2012/1/11/93)
5. Might be dealt with by the effort to get rid of pagevecs (see: https://lkml.org/lkml/2012/1/4/376) or we can do schedule_on_each_cpu_mask(...)