Skip to content

Commit 1063973

Browse files
David Jefferyaxboe
authored andcommitted
sbitmap: fix batching wakeup
Current code supposes that it is enough to provide forward progress by just waking up one wait queue after one completion batch is done. Unfortunately this way isn't enough, cause waiter can be added to wait queue just after it is woken up. Follows one example(64 depth, wake_batch is 8) 1) all 64 tags are active 2) in each wait queue, there is only one single waiter 3) each time one completion batch(8 completions) wakes up just one waiter in each wait queue, then immediately one new sleeper is added to this wait queue 4) after 64 completions, 8 waiters are wakeup, and there are still 8 waiters in each wait queue 5) after another 8 active tags are completed, only one waiter can be wakeup, and the other 7 can't be waken up anymore. Turns out it isn't easy to fix this problem, so simply wakeup enough waiters for single batch. Cc: Kemeng Shi <shikemeng@huaweicloud.com> Cc: Chengming Zhou <zhouchengming@bytedance.com> Cc: Jan Kara <jack@suse.cz> Signed-off-by: David Jeffery <djeffery@redhat.com> Signed-off-by: Ming Lei <ming.lei@redhat.com> Reviewed-by: Gabriel Krisman Bertazi <krisman@suse.de> Reviewed-by: Keith Busch <kbusch@kernel.org> Link: https://lore.kernel.org/r/20230721095715.232728-1-ming.lei@redhat.com Signed-off-by: Jens Axboe <axboe@kernel.dk>
1 parent 013adcb commit 1063973

File tree

1 file changed

+7
-8
lines changed

1 file changed

+7
-8
lines changed

lib/sbitmap.c

Lines changed: 7 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -550,7 +550,7 @@ EXPORT_SYMBOL_GPL(sbitmap_queue_min_shallow_depth);
550550

551551
static void __sbitmap_queue_wake_up(struct sbitmap_queue *sbq, int nr)
552552
{
553-
int i, wake_index;
553+
int i, wake_index, woken;
554554

555555
if (!atomic_read(&sbq->ws_active))
556556
return;
@@ -567,13 +567,12 @@ static void __sbitmap_queue_wake_up(struct sbitmap_queue *sbq, int nr)
567567
*/
568568
wake_index = sbq_index_inc(wake_index);
569569

570-
/*
571-
* It is sufficient to wake up at least one waiter to
572-
* guarantee forward progress.
573-
*/
574-
if (waitqueue_active(&ws->wait) &&
575-
wake_up_nr(&ws->wait, nr))
576-
break;
570+
if (waitqueue_active(&ws->wait)) {
571+
woken = wake_up_nr(&ws->wait, nr);
572+
if (woken == nr)
573+
break;
574+
nr -= woken;
575+
}
577576
}
578577

579578
if (wake_index != atomic_read(&sbq->wake_index))

0 commit comments

Comments
 (0)