Skip to content

Commit f4edc66

Browse files
Martin KaFai LauAlexei Starovoitov
authored andcommitted
bpf: Only fails the busy counter check in bpf_cgrp_storage_get if it creates storage
The current cgrp storage has a percpu counter, bpf_cgrp_storage_busy, to detect potential deadlock at a spin_lock that the local storage acquires during new storage creation. There are false positives. It turns out to be too noisy in production. For example, a bpf prog may be doing a bpf_cgrp_storage_get on map_a. An IRQ comes in and triggers another bpf_cgrp_storage_get on a different map_b. It will then trigger the false positive deadlock check in the percpu counter. On top of that, both are doing lookup only and no need to create new storage, so practically it does not need to acquire the spin_lock. The bpf_task_storage_get already has a strategy to minimize this false positive by only failing if the bpf_task_storage_get needs to create a new storage and the percpu counter is busy. Creating a new storage is the only time it must acquire the spin_lock. This patch borrows the same idea. Unlike task storage that has a separate variant for tracing (_recur) and non-tracing, this patch stays with one bpf_cgrp_storage_get helper to keep it simple for now in light of the upcoming res_spin_lock. The variable could potentially use a better name noTbusy instead of nobusy. This patch follows the same naming in bpf_task_storage_get for now. I have tested it by temporarily adding noinline to the cgroup_storage_lookup(), traced it by fentry, and the fentry program succeeded in calling bpf_cgrp_storage_get(). Signed-off-by: Martin KaFai Lau <martin.lau@kernel.org> Link: https://lore.kernel.org/r/20250318182759.3676094-1-martin.lau@linux.dev Signed-off-by: Alexei Starovoitov <ast@kernel.org>
1 parent ae0a457 commit f4edc66

File tree

1 file changed

+6
-5
lines changed

1 file changed

+6
-5
lines changed

kernel/bpf/bpf_cgrp_storage.c

Lines changed: 6 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -161,6 +161,7 @@ BPF_CALL_5(bpf_cgrp_storage_get, struct bpf_map *, map, struct cgroup *, cgroup,
161161
void *, value, u64, flags, gfp_t, gfp_flags)
162162
{
163163
struct bpf_local_storage_data *sdata;
164+
bool nobusy;
164165

165166
WARN_ON_ONCE(!bpf_rcu_lock_held());
166167
if (flags & ~(BPF_LOCAL_STORAGE_GET_F_CREATE))
@@ -169,21 +170,21 @@ BPF_CALL_5(bpf_cgrp_storage_get, struct bpf_map *, map, struct cgroup *, cgroup,
169170
if (!cgroup)
170171
return (unsigned long)NULL;
171172

172-
if (!bpf_cgrp_storage_trylock())
173-
return (unsigned long)NULL;
173+
nobusy = bpf_cgrp_storage_trylock();
174174

175-
sdata = cgroup_storage_lookup(cgroup, map, true);
175+
sdata = cgroup_storage_lookup(cgroup, map, nobusy);
176176
if (sdata)
177177
goto unlock;
178178

179179
/* only allocate new storage, when the cgroup is refcounted */
180180
if (!percpu_ref_is_dying(&cgroup->self.refcnt) &&
181-
(flags & BPF_LOCAL_STORAGE_GET_F_CREATE))
181+
(flags & BPF_LOCAL_STORAGE_GET_F_CREATE) && nobusy)
182182
sdata = bpf_local_storage_update(cgroup, (struct bpf_local_storage_map *)map,
183183
value, BPF_NOEXIST, false, gfp_flags);
184184

185185
unlock:
186-
bpf_cgrp_storage_unlock();
186+
if (nobusy)
187+
bpf_cgrp_storage_unlock();
187188
return IS_ERR_OR_NULL(sdata) ? (unsigned long)NULL : (unsigned long)sdata->data;
188189
}
189190

0 commit comments

Comments
 (0)