cmake: compiler / linker infrastructure improvement #89073

tejlmand · 2025-04-25T07:33:00Z

Cleanup and improve the design and logic flow of compiler and linker handling.

With this PR then FindTargetTools.cmake is now the entry point for
loading toolchain CMake file for configuring the toolchain and its
properties.

The main target toolchain handling structure is now:

Load compiler, linker, bintools template properties.
Load actual target compiler, linker, bintools properties.
Load target compiler, linker, bintools target CMake files.
Those are responsible for defining the actual CMake compiler settings,
setting up linker invocation, etc.

For highlevel toolchain verification, a new zephyr_verify_toolchain
function has been introduced which invokes the compiler to check its
working state.

Some compiler and linker properties requires testing the flags by
invoking the actual compiler or linker. This is done using
check_set_compiler_property and check_set_linker_property.
However such tests cannot be executed until CMake's project() function
has been called. Therefore calls to those functions are defered until
the basic toolchain has been verified using zephyr_verify_toolchain.

This allows us to introduce the above structure and thereby cleaning up
toolchain handling and improving the design. It also allows compiler and
linker target CMake files to use compiler properties as they are now
available at an earlier stage.

The above prepares for improved library path handling as it becomes easier and cleaner to decide compiler flags which should be used for C and runtime library path lookup, as presented in #87657

keith-packard · 2025-04-25T14:23:27Z

This appears to still use generator expressions. I think those can't be used to resolve the compiler library pathnames needed for crtbegin.o and crtend.o filenames.

tejlmand · 2025-04-28T11:54:41Z

This appears to still use generator expressions.

Correct, because this way the principle introduced with #24851 will still be used and thereby a single approach for describing compiler/linker, and not a mix of properties and macro/functions.

I think those can't be used to resolve the compiler library pathnames needed for crtbegin.o and crtend.o filenames.

Genex'es cannot, that's correct, but properties can still be retrieved and used at CMake configure time, albeit later adjustments to the property will not be picked up, but this also goes for a macro/function, and should not be a blocker in this case.

This was what I was referring to with this comment #87657 (comment):

The original intent with compiler / linker flags properties was to avoid if(CONFIG_X) checking when dealing with flag support in compiler_flags.cmake and linker_flags.cmake and instead just describe the flags, and then where they are used do the config check. The is almost true, although for historic reasons there does exists some checks.

Following this approach I think the linker should have a property describing flags which might impact path calc, but as the actual flag usage is config dependent we need improved Kconfig -> mapping, which I guess will be on my shoulders 😃 .

keith-packard · 2025-04-28T20:01:11Z

Genex'es cannot, that's correct, but properties can still be retrieved and used at CMake configure time, albeit later adjustments to the property will not be picked up, but this also goes for a macro/function, and should not be a blocker in this case.

I think this will still break the computation of paths for crtbegin.o/crtend.o. The directory name for those is currently computed at cmake time in either gcc/target.cmake or clang/target.cmake using

execute_process(
    COMMAND ${CMAKE_C_COMPILER} ${clang_target_flag} ${TOOLCHAIN_C_FLAGS}
            --print-libgcc-file-name
  OUTPUT_VARIABLE LIBGCC_FILE_NAME
  OUTPUT_STRIP_TRAILING_WHITESPACE
  )

This needs to have the optimization flag added or the value will be wrong. I can't see how to do that with generator expressions as the resulting directory name is used to construct the correct path for crtbegin.o and crtend.o; the full paths for those are then passed to a linker command later on in the build.

tejlmand · 2025-05-15T13:21:35Z

This needs to have the optimization flag added or the value will be wrong. I can't see how to do that with generator expressions as the resulting directory name is used to construct the correct path for crtbegin.o and crtend.o; the full paths for those are then passed to a linker command later on in the build.

@keith-packard Took a tour down the rabbit whole.
I've wanted for a long time to collect handling of target toolchain so that FindTargetTools.cmake is the highlevel entry and not pieces scattered throughout the build system. (some in FindTargetTools, some in kernel.cmake, and so on)

Also I improved the property handling so that linker and compiler set the known flags in compiler_flags / linker_flags, and calculated properties can be set in the respective <compiler|linker>/target.cmake files.

There is a little more cleaning to do in future, but this improves the flow, and makes in much cleaner to support optimization flags on the linker.

I took 1bb65eb and reworked it a little bit, namely aligning the two path computation functions so they both returns the results and then it's for the caller to decide how to use the result.
That will also make it more visible and clear which CMake code is responsible for setting properties.

This commit cleanup and improve the structure of target toolchain handling in Zephyr. With this commit FindTargetTools.cmake is now the entry point for loading toolchain CMake file for configuring the toolchain and its properties. The main target toolchain handling structure is now: - Load compiler, linker, bintools template properties. - Load actual target compiler, linker, bintools properties. - Load target compiler, linker, bintools target CMake files. Those are responsible for defining the actual CMake compiler settings, setting up linker invocation, etc. For highlevel toolchain verification, a new `zephyr_verify_toolchain` function has been introduced which invokes the compiler to check its working state. Some compiler and linker properties requires testing the flags by invoking the actual compiler or linker. This is done using `check_set_compiler_property` and `check_set_linker_property`. However such tests cannot be executed until CMake's `project()` function has been called. Therefore calls to those functions are defered until the basic toolchain has been verified using `zephyr_verify_toolchain`. This allows us to introduce the above structure and thereby cleaning up toolchain handling and improving the design. It also allows compiler and linker target CMake files to use compiler properties as they are now available at an earlier stage. Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>

Align the compiler optimization property names with their corresponding Kconfig settings, as example so the property name `size_optimizations` matches `CONFIG_SIZE_OPTIMIZATIONS`. Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>

Support choice handling for compiler property. Some compiler configuration in Kconfig is handled using choices, such as compiler optimization. The compiler infrastructure uses property values for handling compiler flag variations between compilers. This works well when describing the compiler but when having to apply flags based on a Kconfig choice selection it results in constructs like: > if(CONFIG_ENTRY_A) > zephyr_compile_options($<TARGET_PROPERTY:compiler,entry_a>) > elseif(CONFIG_ENTRY_B) > zephyr_compile_options($<TARGET_PROPERTY:compiler,entry_b>) > elseif(CONFIG_ENTRY_C) > zephyr_compile_options($<TARGET_PROPERTY:compiler,entry_c>) > ... which may even have to be repeated multiple times. Introduce a higher property name which can match the choice name and which support taking the currently selected property value, while still keep current infrastructure of the compiler defining property values for all flags. Function signature: > set_compiler_property([APPEND] PROPERTY <name> CHOICE <name>...) This allows the if construct above to be simplified to: > zephyr_compile_options($<TARGET_PROPERTY:compiler,<choice_name>) Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>

Support linker optimization properties. Allow the linker property to inherit the property values of the compiler when those are identical. Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>

The path to a library may be depending on optimization level. Therefor include compiler optimization level for gcc / clang. For gcc and clang, then the same optimization flags are used during linking and therefor the property from the compiler flags is used. Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>

With inclusion of the optimization flag into the multilib selection process, we cannot compute the compiler library path when the compiler's target.cmake is processed as OPTIMIZATION_FLAG is not computed until much later. Instead, add a function (compiler_file_path) which can be used to locate the appropriate crtbegin.o and crtend.o files. Delay computation of lib_include_dir and rt_library until after all compiler flags have been computed by adding compiler_set_linker_properties and calling that just before toolchain_linker_finalize is invoked. Place default implementations of both of these functions in a new file, cmake/compiler/target_template.cmake, where we assume the compiler works like gcc or clang and handlers the --print-file-name and --print-libgcc-file-name options. Compilers needing alternate implementations can override these functions in their target.cmake files. These implementations require that no generator expressions are necessary for the compiler to compute the right library paths. This mechanism is also used to take any additional compiler options and pass them to the linker using toolchain_linker_add_compiler_options. Signed-off-by: Keith Packard <keithp@keithp.com>

sonarqubecloud · 2025-05-16T09:53:45Z

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

keith-packard

I've tested this using Zephyr SDK 0.18 pre-release bits (which enable multilib-space). It's selecting the right libraries for libc, libstdc++, libgcc as well as the correct object files for crtbegin and crtend. If the remaining minor issues can be resolved, this looks good to me

pdgendt · 2025-05-22T17:17:29Z

cmake/compiler/arcmwdt/compiler_flags.cmake

+set_compiler_property(PROPERTY optimization
+                      CHOICE no_optimizations
+		             debug_optimizations
+			     speed_optimizations
+			     size_optimizations
+)


Weird indentation here (mixing leading tabs and spaces)

pdgendt · 2025-05-22T17:18:12Z

cmake/compiler/gcc/compiler_flags.cmake

+set_compiler_property(PROPERTY optimization
+                      CHOICE no_optimizations
+		             debug_optimizations
+			     speed_optimizations
+			     size_optimizations
+			     size_optimizations_aggressive
+)


pdgendt · 2025-05-22T17:18:42Z

cmake/compiler/iar/compiler_flags.cmake

+set_compiler_property(PROPERTY optimization
+                      CHOICE no_optimizations
+		             debug_optimizations
+			     speed_optimizations
+			     size_optimizations
+			     size_optimizations_aggressive
+)


keith-packard · 2025-05-22T17:35:20Z

I can't reproduce the failure shown in CI, but I think it's because compile_rt_library doesn't use the -m32 option. I've created a small patch which just disables use of that function and lets the linker sort out the runtime library itself. The alternative would be to create a function that appends options to TOOLCHAIN_C_FLAGS and have arch/posix/CMakeLists.txt use that.

keith-packard@6c1e250

keith-packard · 2025-05-23T02:35:40Z

I added a bunch of minor fixes on top of this over in #90353. Feel free to mix those in if you like and get this ready for merging.

github-actions · 2025-07-23T00:41:58Z

This pull request has been marked as stale because it has been open (more than) 60 days with no activity. Remove the stale label or add a comment saying that you would like to have the label removed otherwise this pull request will automatically be closed in 14 days. Note, that you can always re-open a closed pull request at any time.

keith-packard · 2025-07-23T00:46:35Z

We still need something like this for SDK 0.18

tejlmand requested a review from keith-packard April 25, 2025 07:33

tejlmand force-pushed the linker_optimization branch from 7a4e6d5 to d8b7285 Compare April 25, 2025 07:34

tejlmand mentioned this pull request Apr 25, 2025

cmake: Pass OPTIMIZATION_FLAG to linker too #87657

Open

tejlmand force-pushed the linker_optimization branch from d8b7285 to 2614027 Compare April 25, 2025 10:25

tejlmand force-pushed the linker_optimization branch from 2614027 to 8744ec0 Compare May 15, 2025 13:08

tejlmand changed the title ~~cmake: support linker optimization levels~~ cmake: compiler / linker infrastructure improvement May 15, 2025

tejlmand marked this pull request as ready for review May 15, 2025 13:21

github-actions bot added area: Bluetooth area: Bluetooth Controller area: Build System area: Toolchains Toolchains labels May 15, 2025

github-actions bot requested review from 57300, abrodkin, carlescufi, cvinayak, erbr-ot, evgeniy-paltsev, jeremybettis, nashif, nordicjm, stephanosio, thoh-ot, Tronil and wopu-ot May 15, 2025 13:22

github-actions bot assigned evgeniy-paltsev and abrodkin May 15, 2025

tejlmand mentioned this pull request May 15, 2025

Simpler link optimization flag #89509

Open

tejlmand force-pushed the linker_optimization branch from 8744ec0 to cc14a48 Compare May 15, 2025 13:41

tejlmand assigned nordicjm and unassigned abrodkin and evgeniy-paltsev May 15, 2025

tejlmand force-pushed the linker_optimization branch from cc14a48 to 370151c Compare May 15, 2025 20:29

tejlmand and others added 6 commits May 16, 2025 11:25

cmake: support linker optimization levels

d3d5569

Support linker optimization properties. Allow the linker property to inherit the property values of the compiler when those are identical. Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>

tejlmand force-pushed the linker_optimization branch from 370151c to 4fc9a17 Compare May 16, 2025 09:25

keith-packard approved these changes May 22, 2025

View reviewed changes

pdgendt reviewed May 22, 2025

View reviewed changes

keith-packard mentioned this pull request May 22, 2025

Fix up native_sim for #89073 #90353

Open

github-actions bot added the Stale label Jul 23, 2025

keith-packard removed the Stale label Jul 23, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

cmake: compiler / linker infrastructure improvement #89073

cmake: compiler / linker infrastructure improvement #89073

Uh oh!

tejlmand commented Apr 25, 2025 •

edited

Loading

Uh oh!

keith-packard commented Apr 25, 2025

Uh oh!

tejlmand commented Apr 28, 2025

Uh oh!

keith-packard commented Apr 28, 2025

Uh oh!

tejlmand commented May 15, 2025

Uh oh!

sonarqubecloud bot commented May 16, 2025

Uh oh!

keith-packard left a comment

Uh oh!

pdgendt May 22, 2025

Uh oh!

pdgendt May 22, 2025

Uh oh!

pdgendt May 22, 2025

Uh oh!

keith-packard commented May 22, 2025

Uh oh!

keith-packard commented May 23, 2025

Uh oh!

github-actions bot commented Jul 23, 2025

Uh oh!

keith-packard commented Jul 23, 2025

Uh oh!

Uh oh!

cmake: compiler / linker infrastructure improvement #89073

Are you sure you want to change the base?

cmake: compiler / linker infrastructure improvement #89073

Uh oh!

Conversation

tejlmand commented Apr 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

keith-packard commented Apr 25, 2025

Uh oh!

tejlmand commented Apr 28, 2025

Uh oh!

keith-packard commented Apr 28, 2025

Uh oh!

tejlmand commented May 15, 2025

Uh oh!

sonarqubecloud bot commented May 16, 2025

Quality Gate passed

Uh oh!

keith-packard left a comment

Choose a reason for hiding this comment

Uh oh!

pdgendt May 22, 2025

Choose a reason for hiding this comment

Uh oh!

pdgendt May 22, 2025

Choose a reason for hiding this comment

Uh oh!

pdgendt May 22, 2025

Choose a reason for hiding this comment

Uh oh!

keith-packard commented May 22, 2025

Uh oh!

keith-packard commented May 23, 2025

Uh oh!

github-actions bot commented Jul 23, 2025

Uh oh!

keith-packard commented Jul 23, 2025

Uh oh!

Uh oh!

tejlmand commented Apr 25, 2025 •

edited

Loading