Skip to content

Commit 2468b0e

Browse files
zhangtianyang-ztychenhuacai
authored andcommitted
LoongArch: Prevent cond_resched() occurring within kernel-fpu
When CONFIG_PREEMPT_COUNT is not configured (i.e. CONFIG_PREEMPT_NONE/ CONFIG_PREEMPT_VOLUNTARY), preempt_disable() / preempt_enable() merely acts as a barrier(). However, in these cases cond_resched() can still trigger a context switch and modify the CSR.EUEN, resulting in do_fpu() exception being activated within the kernel-fpu critical sections, as demonstrated in the following path: dcn32_calculate_wm_and_dlg() DC_FP_START() dcn32_calculate_wm_and_dlg_fpu() dcn32_find_dummy_latency_index_for_fw_based_mclk_switch() dcn32_internal_validate_bw() dcn32_enable_phantom_stream() dc_create_stream_for_sink() kzalloc(GFP_KERNEL) __kmem_cache_alloc_node() __cond_resched() DC_FP_END() This patch is similar to commit d021985 (x86/fpu: Improve crypto performance by making kernel-mode FPU reliably usable in softirqs). It uses local_bh_disable() instead of preempt_disable() for non-RT kernels so it can avoid the cond_resched() issue, and also extend the kernel-fpu application scenarios to the softirq context. Cc: stable@vger.kernel.org Signed-off-by: Tianyang Zhang <zhangtianyang@loongson.cn> Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
1 parent 82f2b0b commit 2468b0e

File tree

1 file changed

+20
-2
lines changed

1 file changed

+20
-2
lines changed

arch/loongarch/kernel/kfpu.c

Lines changed: 20 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -18,11 +18,28 @@ static unsigned int euen_mask = CSR_EUEN_FPEN;
1818
static DEFINE_PER_CPU(bool, in_kernel_fpu);
1919
static DEFINE_PER_CPU(unsigned int, euen_current);
2020

21+
static inline void fpregs_lock(void)
22+
{
23+
if (IS_ENABLED(CONFIG_PREEMPT_RT))
24+
preempt_disable();
25+
else
26+
local_bh_disable();
27+
}
28+
29+
static inline void fpregs_unlock(void)
30+
{
31+
if (IS_ENABLED(CONFIG_PREEMPT_RT))
32+
preempt_enable();
33+
else
34+
local_bh_enable();
35+
}
36+
2137
void kernel_fpu_begin(void)
2238
{
2339
unsigned int *euen_curr;
2440

25-
preempt_disable();
41+
if (!irqs_disabled())
42+
fpregs_lock();
2643

2744
WARN_ON(this_cpu_read(in_kernel_fpu));
2845

@@ -73,7 +90,8 @@ void kernel_fpu_end(void)
7390

7491
this_cpu_write(in_kernel_fpu, false);
7592

76-
preempt_enable();
93+
if (!irqs_disabled())
94+
fpregs_unlock();
7795
}
7896
EXPORT_SYMBOL_GPL(kernel_fpu_end);
7997

0 commit comments

Comments
 (0)