Using exe in custom_target without re-invoking meson

[mitchgrout]

Consider the following setup:

tool    = executable(...)
sources = files(...)
foreach source: sources
  custom_target(input: source, output: '@BASENAME@.result', command: [tool, '@INPUT@', '@OUTPUT@'])
endforeach

Supposing the name of the executable is tool.exe, and one of the sources is foo.c, then the resulting build.ninja (on Windows at least) will contain a build rule to the following effect:

build foo.result: CUSTOM_COMMAND ../foo.c | tool.exe
  COMMAND = "meson" "--internal" "exe" "--unpickle" "[uuid].dat"

My expectation would be that the COMMAND would directly use tool.exe but instead I see that it calls back in via meson. I have a significant number of source files that I would like to run this tool on, and the overhead of invoking meson for each is nearly equal to the actual runtime of the tool itself.

A quick-and-dirty way I could get around this would be to compile tool.exe prior to setting up the build directory, and then accessing it via find_program; in this instance, the COMMAND is exactly what I want (a direct invocation of tool.exe). I was hoping that there would be a solution within meson so that I don't have to worry about compiler detection or formatting of arguments.

Any other approaches to this issue? I would also be interested in knowing why a call-back into meson is necessary here.

[eli-schwartz]

Calling back into meson is required under certain circumstances:

• the custom_target sets environment variables
• the custom_target uses the capture or feed kwargs
• the command line to run contains newlines somewhere
• you are cross compiling and need to set up an exe_wrapper (I am not sure why we enforce this instead of tweaking the command itself)
• on Windows, the program being run has some DLLs that meson believes it needs to load, thus necessitating setting PATH
• on Windows, when using Visual Studio solutions, most of the time (running compilers is a notable exception)
• for meson functions where you can set a workdir kwarg, if that kwarg is set

In all these cases we need to minimize the challenges of running commands with a lot of non-portable context. So we have a helper program that can do all these things at once, and uses a pickled data file to load the context so that we don't need to deal with escaping such fiddly data.

In the specific case of environment variables, if the env command is available (Unix or cygwin/msys2) and the only point on the above list was needing to set environment variables, we will call the env command instead of meson --internal exe.

Additionally, if the capture/feed kwargs were the only point on the above list, we will use --capture and --feed parameters to the internal command, rather than --unpickle, and the resulting command line will be a lot easier to read (but still involve calling back into meson).

...

In your case, the likely reason is solely down to your executable tool having some dependencies that caused meson to think it needs to set a PATH. Does it link to any dependencies?

[mitchgrout]

I see; that explains the diagnostic of (wrapped by meson to set PATH). However, there's nothing in the setup that would innately require PATH to be set; the executable is effectively just:

tool = executable('tool', '../../path/to/tool.c', native: true)

I duplicated the setup under WSL2 (Ubuntu) and observe my desired setup there (e.g. no wrap from meson). I'm running all of this within a mingw shell, so perhaps it's getting a bit crossed up there? It's good to know meson is going to pick the more efficient route where it can, but now I just need to see why it's not

[mitchgrout]

So I made a clone of printenv for the tool, and observed that the only difference between $PATH when running it locally vs running it wrapped by meson is the inclusion of the build directory, nothing else. There could be other adjustments to the environment that I'm not immediately seeing, but the suspicion is that $PATH has something extra, but nothing immediately apparent.

[xclaesse]

That's something I noticed a while ago, determine_windows_extra_paths() has some weird behaviours. Your specific case is that it thinks that the path to the exe itself must be in PATH which is not true, we want only to add DLL paths to PATH. That's a Meson bug AFAIK.

[xclaesse]

Made an issue: #12042. Closing this discussion, let's continue there.