[BOLT] Force frame pointers off for runtimes

[peterwaller-arm]

Distributions are making the choice to turn frame pointers on by default. Nixpkgs recently turned them on, and the method they use to do so implies that everything is built with them on by default.

NixOS/nixpkgs#399014

Assuming that a well behaved distribution doing this puts -fno-omit-frame-pointer at the beginning of the compiler invocation, we can still re-enable omission by supplying -fomit-frame-pointer during compilation.

This fixes some segfaults from stack corruption in binaries rewritten by bolt with llvm-bolt -instrument.

See also: #147569

[llvmbot]

@llvm/pr-subscribers-bolt

Author: Peter Waller (peterwaller-arm)

Changes

Distributions are making the choice to turn frame pointers on by default. Nixpkgs recently turned them on, and the method they use to do so implies that everything is built with them on by default.

NixOS/nixpkgs#399014

Assuming that a well behaved distribution doing this puts -fno-omit-frame-pointer at the beginning of the compiler invocation, we can still re-enable it by supplying -fomit-frame-pointer during compilation.

See also: #147569

---

Full diff: https://github.com/llvm/llvm-project/pull/148009.diff

1 Files Affected:

• (modified) bolt/runtime/CMakeLists.txt (+4-1)

diff --git a/bolt/runtime/CMakeLists.txt b/bolt/runtime/CMakeLists.txt
index 87cc44812da11..4c496c2b800ef 100644
--- a/bolt/runtime/CMakeLists.txt
+++ b/bolt/runtime/CMakeLists.txt
@@ -35,7 +35,10 @@ set(BOLT_RT_FLAGS
   -fno-exceptions
   -fno-rtti
   -fno-stack-protector
-  -fPIC)
+  -fPIC
+  -mgeneral-regs-only
+  -fomit-frame-pointer # Runtime currently assumes omitted frame pointers. llvm/llvm-project#147569
+)
 if (CMAKE_SYSTEM_PROCESSOR MATCHES "x86_64")
   set(BOLT_RT_FLAGS ${BOLT_RT_FLAGS} 
     -mno-sse 

[peterwaller-arm]

llvm-project/bolt/runtime/instr.cpp

Line 1666 in dbb79c3

extern "C" __attribute((naked)) void __bolt_instr_indirect_call()

I currently suspect the issue is arising from void __bolt_instr_indirect_call() or related functions, which are marked __attribute((naked)). Naively I would have expected this to omit the instructions for preserving the frame pointer, but that's not what I see:

0000000000000000 <__bolt_instr_indirect_call>:
       0:       a9bf7bfd        stp     x29, x30, [sp, #-16]!
       4:       910003fd        mov     x29, sp
       8:       a9bf07e0        stp     x0, x1, [sp, #-16]!
       c:       a9bf0fe2        stp     x2, x3, [sp, #-16]!
      10:       a9bf17e4        stp     x4, x5, [sp, #-16]!
      14:       a9bf1fe6        stp     x6, x7, [sp, #-16]!
      18:       a9bf27e8        stp     x8, x9, [sp, #-16]!

Mentioning it in case anyone has an idea what's happening there, it looks like it relates to -mno-omit-leaf-frame-pointer, which nixpkgs is also supplying, -fno-omit-frame-pointer does not appear to be enough to cause the problem by itself. However, -fomit-frame-pointer does negate the issue.

[nikic]

@peterwaller-arm It looks like GCC just doesn't support naked functions on AArch64 at all, see https://gcc.gnu.org/bugzilla/show_bug.cgi?id=77882.

[nikic]

Looks reasonable to me, but a bolt maintainer should approve as well.