Skip to content

Commit 00a8b4b

Browse files
committed
rcu-tasks: Set ->percpu_enqueue_shift to zero upon contention
Currently, call_rcu_tasks_generic() sets ->percpu_enqueue_shift to order_base_2(nr_cpu_ids) upon encountering sufficient contention. This does not shift to use of non-CPU-0 callback queues as intended, but rather continues using only CPU 0's queue. Although this does provide some decrease in contention due to spreading work over multiple locks, it is not the dramatic decrease that was intended. This commit therefore makes call_rcu_tasks_generic() set ->percpu_enqueue_shift to 0. Reported-by: Neeraj Upadhyay <quic_neeraju@quicinc.com> Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
1 parent 2bcd18e commit 00a8b4b

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

kernel/rcu/tasks.h

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -302,7 +302,7 @@ static void call_rcu_tasks_generic(struct rcu_head *rhp, rcu_callback_t func,
302302
if (unlikely(needadjust)) {
303303
raw_spin_lock_irqsave(&rtp->cbs_gbl_lock, flags);
304304
if (rtp->percpu_enqueue_lim != nr_cpu_ids) {
305-
WRITE_ONCE(rtp->percpu_enqueue_shift, order_base_2(nr_cpu_ids));
305+
WRITE_ONCE(rtp->percpu_enqueue_shift, 0);
306306
WRITE_ONCE(rtp->percpu_dequeue_lim, nr_cpu_ids);
307307
smp_store_release(&rtp->percpu_enqueue_lim, nr_cpu_ids);
308308
pr_info("Switching %s to per-CPU callback queuing.\n", rtp->name);

0 commit comments

Comments
 (0)