Skip to content

Commit f7ec1cd

Browse files
oleg-nesterovakpm00
authored andcommitted
getrusage: use sig->stats_lock rather than lock_task_sighand()
lock_task_sighand() can trigger a hard lockup. If NR_CPUS threads call getrusage() at the same time and the process has NR_THREADS, spin_lock_irq will spin with irqs disabled O(NR_CPUS * NR_THREADS) time. Change getrusage() to use sig->stats_lock, it was specifically designed for this type of use. This way it runs lockless in the likely case. TODO: - Change do_task_stat() to use sig->stats_lock too, then we can remove spin_lock_irq(siglock) in wait_task_zombie(). - Turn sig->stats_lock into seqcount_rwlock_t, this way the readers in the slow mode won't exclude each other. See https://lore.kernel.org/all/20230913154907.GA26210@redhat.com/ - stats_lock has to disable irqs because ->siglock can be taken in irq context, it would be very nice to change __exit_signal() to avoid the siglock->stats_lock dependency. Link: https://lkml.kernel.org/r/20240122155053.GA26214@redhat.com Signed-off-by: Oleg Nesterov <oleg@redhat.com> Reported-by: Dylan Hatch <dylanbhatch@google.com> Tested-by: Dylan Hatch <dylanbhatch@google.com> Cc: Eric W. Biederman <ebiederm@xmission.com> Cc: <stable@vger.kernel.org> Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
1 parent daa694e commit f7ec1cd

File tree

1 file changed

+13
-3
lines changed

1 file changed

+13
-3
lines changed

kernel/sys.c

Lines changed: 13 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1788,7 +1788,9 @@ void getrusage(struct task_struct *p, int who, struct rusage *r)
17881788
unsigned long maxrss;
17891789
struct mm_struct *mm;
17901790
struct signal_struct *sig = p->signal;
1791+
unsigned int seq = 0;
17911792

1793+
retry:
17921794
memset(r, 0, sizeof(*r));
17931795
utime = stime = 0;
17941796
maxrss = 0;
@@ -1800,8 +1802,7 @@ void getrusage(struct task_struct *p, int who, struct rusage *r)
18001802
goto out_thread;
18011803
}
18021804

1803-
if (!lock_task_sighand(p, &flags))
1804-
return;
1805+
flags = read_seqbegin_or_lock_irqsave(&sig->stats_lock, &seq);
18051806

18061807
switch (who) {
18071808
case RUSAGE_BOTH:
@@ -1829,14 +1830,23 @@ void getrusage(struct task_struct *p, int who, struct rusage *r)
18291830
r->ru_oublock += sig->oublock;
18301831
if (maxrss < sig->maxrss)
18311832
maxrss = sig->maxrss;
1833+
1834+
rcu_read_lock();
18321835
__for_each_thread(sig, t)
18331836
accumulate_thread_rusage(t, r);
1837+
rcu_read_unlock();
1838+
18341839
break;
18351840

18361841
default:
18371842
BUG();
18381843
}
1839-
unlock_task_sighand(p, &flags);
1844+
1845+
if (need_seqretry(&sig->stats_lock, seq)) {
1846+
seq = 1;
1847+
goto retry;
1848+
}
1849+
done_seqretry_irqrestore(&sig->stats_lock, seq, flags);
18401850

18411851
if (who == RUSAGE_CHILDREN)
18421852
goto out_children;

0 commit comments

Comments
 (0)