Skip to content

Commit 3bb1593

Browse files
kkdwivediAlexei Starovoitov
authored andcommitted
rqspinlock: Protect waiters in trylock fallback from stalls
When we run out of maximum rqnodes, the original queued spin lock slow path falls back to a try lock. In such a case, we are again susceptible to stalls in case the lock owner fails to make progress. We use the timeout as a fallback to break out of this loop and return to the caller. This is a fallback for an extreme edge case, when on the same CPU we run out of all 4 qnodes. When could this happen? We are in slow path in task context, we get interrupted by an IRQ, which while in the slow path gets interrupted by an NMI, whcih in the slow path gets another nested NMI, which enters the slow path. All of the interruptions happen after node->count++. We use RES_DEF_TIMEOUT as our spinning duration, but in the case of this fallback, no fairness is guaranteed, so the duration may be too small for contended cases, as the waiting time is not bounded. Since this is an extreme corner case, let's just prefer timing out instead of attempting to spin for longer. Reviewed-by: Barret Rhoden <brho@google.com> Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com> Link: https://lore.kernel.org/r/20250316040541.108729-12-memxor@gmail.com Signed-off-by: Alexei Starovoitov <ast@kernel.org>
1 parent 164c246 commit 3bb1593

File tree

1 file changed

+7
-1
lines changed

1 file changed

+7
-1
lines changed

kernel/bpf/rqspinlock.c

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -275,8 +275,14 @@ int __lockfunc resilient_queued_spin_lock_slowpath(rqspinlock_t *lock, u32 val)
275275
*/
276276
if (unlikely(idx >= _Q_MAX_NODES)) {
277277
lockevent_inc(lock_no_node);
278-
while (!queued_spin_trylock(lock))
278+
RES_RESET_TIMEOUT(ts, RES_DEF_TIMEOUT);
279+
while (!queued_spin_trylock(lock)) {
280+
if (RES_CHECK_TIMEOUT(ts, ret)) {
281+
lockevent_inc(rqspinlock_lock_timeout);
282+
break;
283+
}
279284
cpu_relax();
285+
}
280286
goto release;
281287
}
282288

0 commit comments

Comments
 (0)