Skip to content

[DTLTO][Clang] Add support for Integrated Distributed ThinLTO #147265

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 16 commits into from
Jul 15, 2025
Merged

[DTLTO][Clang] Add support for Integrated Distributed ThinLTO #147265

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
16 commits
Select commit Hold shift + click to select a range
737bb8c
[DTLTO][Clang] Add support for Integrated Distributed ThinLTO
bd1976bris Jun 5, 2025
af66910
Avoid committing to any work on Sony’s behalf that’s beyond my remit.
bd1976bris Jul 10, 2025
fd51379
shorten argument strings for readability
bd1976bris Jul 10, 2025
f2c6b8d
Use meaningful prefixes for all check lines.
bd1976bris Jul 10, 2025
cdce7ff
NOMORE -> --implicit-check-not
bd1976bris Jul 10, 2025
75289e7
Use shared response files for all FileCheck invocations.
bd1976bris Jul 10, 2025
bb2f7d3
Simplify test by including more arguments in linker shared response file
bd1976bris Jul 10, 2025
00321cf
Improve match for clang executable
bd1976bris Jul 10, 2025
fb25cda
Removed unneeded -Werror as we can rely on the implicit check not check
bd1976bris Jul 10, 2025
44abe76
Introduce a second linker response file which will simplify the test …
bd1976bris Jul 10, 2025
820a50d
Move option handling into addLTOOptions for consistency.
bd1976bris Jul 10, 2025
60e87c9
On linux the clang exectuable might be clang-20. Fix match.
bd1976bris Jul 10, 2025
c961cc0
Use %/s consistently in response files.
bd1976bris Jul 11, 2025
93dda6f
Use -Werror and remove FileCheck response file
bd1976bris Jul 14, 2025
c8e8305
Remove other response file usage
bd1976bris Jul 14, 2025
1b0257c
Replace accidentally dropped implicit check not lines
bd1976bris Jul 14, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
32 changes: 32 additions & 0 deletions clang/docs/ThinLTO.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -240,6 +240,38 @@ The ``BOOTSTRAP_LLVM_ENABLE_LTO=Thin`` will enable ThinLTO for stage 2 and
stage 3 in case the compiler used for stage 1 does not support the ThinLTO
option.

Integrated Distributed ThinLTO (DTLTO)
--------------------------------------

Integrated Distributed ThinLTO (DTLTO) enables the distribution of backend
ThinLTO compilations via external distribution systems, such as Incredibuild,
during the traditional link step.

The implementation is documented here: https://llvm.org/docs/DTLTO.html.

DTLTO requires the LLD linker (``-fuse-ld=lld``).

``-fthinlto-distributor=<path>``
- Specifies the ``<path>`` to the distributor process executable for DTLTO.
- If specified, ThinLTO backend compilations will be distributed by LLD.

``-Xthinlto-distributor=<arg>``
- Passes ``<arg>`` to the distributor process (see ``-fthinlto-distributor=``).
- Can be specified multiple times to pass multiple options.
- Multiple options can also be specified by separating them with commas.

Examples:
- ``clang -flto=thin -fthinlto-distributor=incredibuild.exe -Xthinlto-distributor=--verbose,--j10 -fuse-ld=lld``
- ``clang -flto=thin -fthinlto-distributor=$(which python) -Xthinlto-distributor=incredibuild.py -fuse-ld=lld``

If ``-fthinlto-distributor=`` is specified, Clang supplies the path to a
compiler to be executed remotely to perform the ThinLTO backend
compilations. Currently, this is Clang itself.

Note that currently, DTLTO is only supported in some LLD flavors. Support will
be added to other LLD flavours in the future.
See `DTLTO <https://lld.llvm.org/dtlto.html>`_ for more information.

More Information
================

Expand Down
14 changes: 13 additions & 1 deletion clang/include/clang/Driver/Options.td
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -990,6 +990,13 @@ def Xlinker : Separate<["-"], "Xlinker">, Flags<[LinkerInput, RenderAsInput]>,
Visibility<[ClangOption, CLOption, FlangOption]>,
HelpText<"Pass <arg> to the linker">, MetaVarName<"<arg>">,
Group<Link_Group>;
def Xthinlto_distributor_EQ : CommaJoined<["-"], "Xthinlto-distributor=">,
Flags<[LinkOption]>,
Visibility<[ClangOption, CLOption]>,
HelpText<"Pass <arg> to the ThinLTO distributor process. Can be specified "
"multiple times or with comma-separated values.">,
MetaVarName<"<arg>">,
Group<Link_Group>;
def Xoffload_linker : JoinedAndSeparate<["-"], "Xoffload-linker">,
Visibility<[ClangOption, FlangOption]>,
HelpText<"Pass <arg> to the offload linkers or the ones identified by -<triple>">,
Expand Down Expand Up @@ -4233,7 +4240,12 @@ def ffinite_loops: Flag<["-"], "ffinite-loops">, Group<f_Group>,
def fno_finite_loops: Flag<["-"], "fno-finite-loops">, Group<f_Group>,
HelpText<"Do not assume that any loop is finite.">,
Visibility<[ClangOption, CC1Option]>;

def fthinlto_distributor_EQ : Joined<["-"], "fthinlto-distributor=">,
Group<f_Group>,
HelpText<"Path to the ThinLTO distributor process. If specified, "
"ThinLTO backend compilations will be distributed by LLD">,
MetaVarName<"<path>">,
Visibility<[ClangOption, CLOption]>;
def ftrigraphs : Flag<["-"], "ftrigraphs">, Group<f_Group>,
HelpText<"Process trigraph sequences">, Visibility<[ClangOption, CC1Option]>;
def fno_trigraphs : Flag<["-"], "fno-trigraphs">, Group<f_Group>,
Expand Down
15 changes: 15 additions & 0 deletions clang/lib/Driver/ToolChains/Gnu.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -455,6 +455,21 @@ void tools::gnutools::Linker::ConstructJob(Compilation &C, const JobAction &JA,
addLTOOptions(ToolChain, Args, CmdArgs, Output, Inputs,
D.getLTOMode() == LTOK_Thin);

// Forward the DTLTO options to the linker. We add these unconditionally,
// rather than in addLTOOptions() as it is the linker that decides whether to
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not be consistent with the other LTO options though, which are added in addLTOOptions? The same argument could be made for those (and maybe they should be sent unconditionally, but that seems like a separate discussion).

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Apologies - we’ve recently started setting LTO options unconditionally in the PS5 driver, and I carried that pattern over here without fully reconsidering it. As you pointed out, the issue is broader than just the DTLTO options and deserves separate handling.

It’s also unclear whether setting options unconditionally is the right long-term approach. An alternative could be to pass a flag to the linker, allowing it to emit a warning if LTO is being performed but the Clang driver wasn't explicitly invoked in LTO mode.

In any case, I’ve moved the DTLTO option handling into addLTOOptions as you suggested. Thanks for the guidance.

// do LTO or not dependent upon whether there are any bitcode input files in
// the link.
if (Arg *A = Args.getLastArg(options::OPT_fthinlto_distributor_EQ)) {
CmdArgs.push_back(
Args.MakeArgString("--thinlto-distributor=" + Twine(A->getValue())));
CmdArgs.push_back(
Args.MakeArgString("--thinlto-remote-compiler=" +
Twine(ToolChain.getDriver().getClangProgramPath())));

for (auto A : Args.getAllArgValues(options::OPT_Xthinlto_distributor_EQ))
CmdArgs.push_back(Args.MakeArgString("--thinlto-distributor-arg=" + A));
}

if (Args.hasArg(options::OPT_Z_Xlinker__no_demangle))
CmdArgs.push_back("--no-demangle");

Expand Down
43 changes: 43 additions & 0 deletions clang/test/Driver/DTLTO/dtlto.c
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
// REQUIRES: lld

/// Check DTLTO options are forwarded to the linker.

// RUN: echo "--target=x86_64-linux-gnu \
// RUN: -Xthinlto-distributor=distarg1 \
// RUN: -Xthinlto-distributor=distarg2,distarg3 \
// RUN: -fuse-ld=lld" > %t.rsp

/// Check that options are forwarded as expected with --thinlto-distributor=.
// RUN: %clang -### @%t.rsp -fthinlto-distributor=dist.exe %s 2>&1 | \
// RUN: FileCheck %s --implicit-check-not=warning

// CHECK: ld.lld
// CHECK-SAME: "--thinlto-distributor=dist.exe"
// CHECK-SAME: "--thinlto-remote-compiler={{.*}}clang
// CHECK-SAME: "--thinlto-distributor-arg=distarg1"
// CHECK-SAME: "--thinlto-distributor-arg=distarg2"
// CHECK-SAME: "--thinlto-distributor-arg=distarg3"


/// Check that options are not added without --thinlto-distributor= and
/// that there is an unused option warning issued for -Xthinlto-distributor=
/// options. We specify -flto here as these options should be unaffected by it.
// RUN: %clang -### @%t.rsp -flto=thin %s 2>&1 | \
// RUN: FileCheck %s --check-prefixes=NONE,NOMORE --implicit-check-not=warning
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Confused about the --implicit-check-not=warning since there are some warnings below. Although I guess this is checking that there are none other than those being explicitly checked below - but is that intended?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes that was the intent. I have now used a common response file for all FileCheck invocations in this test and added a comment to explain the intention behind the use of implicit-check-not arguments.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For clang driver, prefer -Werror to implicit-check-not=warning: .
A warning would make clang -### -Werror exit with code 1.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@MaskRay done. Thanks.


// NONE: warning: argument unused during compilation: '-Xthinlto-distributor=distarg1'
// NONE: warning: argument unused during compilation: '-Xthinlto-distributor=distarg2,distarg3'
// NONE: ld.lld
// NOMORE-NOT: distributor
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe better to put these in --implicit-check-not= since this would not catch any before the above warnings.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done. I have also used the same set of implicit-check-not arguments in each of the FileCheck invocations (via a response file) to reduce the size of the test and improve readability.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@teresajohnson I have now removed the use of response files and implemented (judiciously) @MaskRay's suggestion of using -Werror.

// NOMORE-NOT: remote-compiler


/// Check the expected arguments are forwarded by default with only
/// --thinlto-distributor=.
// RUN: %clang --target=x86_64-linux-gnu -fthinlto-distributor=dist.exe \
// RUN: -fuse-ld=lld -Werror -### %s 2>&1 | \
// RUN: FileCheck %s --check-prefixes=DEFAULT,NOMORE

// DEFAULT: ld.lld
// DEFAULT-SAME: "--thinlto-distributor=dist.exe"
// DEFAULT-SAME: "--thinlto-remote-compiler={{.*}}clang
31 changes: 13 additions & 18 deletions cross-project-tests/dtlto/ld-dtlto.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,13 +5,11 @@

// RUN: rm -rf %t && mkdir %t && cd %t

// RUN: %clang --target=x86_64-linux-gnu -c -flto=thin %s -o dtlto.o

// RUN: ld.lld dtlto.o \
// RUN: --thinlto-distributor=%python \
// RUN: --thinlto-distributor-arg=%llvm_src_root/utils/dtlto/local.py \
// RUN: --thinlto-remote-compiler=%clang \
// RUN: --thinlto-remote-compiler-arg=--save-temps
// RUN: %clang --target=x86_64-linux-gnu %s -flto=thin -fuse-ld=lld \
// RUN: -fthinlto-distributor=%python \
// RUN: -Xthinlto-distributor=%llvm_src_root/utils/dtlto/local.py \
// RUN: -Wl,--thinlto-remote-compiler-arg=--save-temps \
// RUN: -nostdlib -Werror

/// Check that the required output files have been created.
// RUN: ls | sort | FileCheck %s
Expand All @@ -22,18 +20,15 @@
/// Linked ELF.
// CHECK: {{^}}a.out{{$}}

/// Produced by the bitcode compilation.
// CHECK-NEXT: {{^}}dtlto.o{{$}}

/// --save-temps output for the backend compilation.
// CHECK-NEXT: {{^}}dtlto.s{{$}}
// CHECK-NEXT: {{^}}dtlto.s.0.preopt.bc{{$}}
// CHECK-NEXT: {{^}}dtlto.s.1.promote.bc{{$}}
// CHECK-NEXT: {{^}}dtlto.s.2.internalize.bc{{$}}
// CHECK-NEXT: {{^}}dtlto.s.3.import.bc{{$}}
// CHECK-NEXT: {{^}}dtlto.s.4.opt.bc{{$}}
// CHECK-NEXT: {{^}}dtlto.s.5.precodegen.bc{{$}}
// CHECK-NEXT: {{^}}dtlto.s.resolution.txt{{$}}
// CHECK-NEXT: {{^}}ld-dtlto-[[TMP:[a-zA-Z0-9_]+]].s{{$}}
// CHECK-NEXT: {{^}}ld-dtlto-[[TMP]].s.0.preopt.bc{{$}}
// CHECK-NEXT: {{^}}ld-dtlto-[[TMP]].s.1.promote.bc{{$}}
// CHECK-NEXT: {{^}}ld-dtlto-[[TMP]].s.2.internalize.bc{{$}}
// CHECK-NEXT: {{^}}ld-dtlto-[[TMP]].s.3.import.bc{{$}}
// CHECK-NEXT: {{^}}ld-dtlto-[[TMP]].s.4.opt.bc{{$}}
// CHECK-NEXT: {{^}}ld-dtlto-[[TMP]].s.5.precodegen.bc{{$}}
// CHECK-NEXT: {{^}}ld-dtlto-[[TMP]].s.resolution.txt{{$}}

/// No files are expected after.
// CHECK-NOT: {{.}}
Expand Down
Loading