unable to ELT-profile (with ICorProfilerCallback) execution of dotnet test and dotnet vstest

[ltcmelo]

I've built an ELT (Enter/Leave/Tailcall) profile that is a simplified version of dotnet runtime's eltprofiler. In reality, my profiler only implements the Enter callback; and its implementation just prints the name of the function being entered and returns:

{
    std::wcout << GetFunctionIDName(functionIdOrClientID.functionID) << endl;  
    return S_OK;
}

(GetFunctionIDName comes from dotnet runtime's base profiler.)

This profiler works well when profiling a framework-dependent .NET app, with any of the invocations app.exe or dotnet app.dll. In both cases, the functions names are output to the console.

With the invocation dotnet run (assuming at the project's directory), the profiler often crashes dotnet; with dotnet run --no-restore --no-build --no-self-contained, and varying combinations of these flags, crashes become intermittent; yet, when there's no crash, the profiler's Enter callback works as expected and the functions names are output to the console — in such case, I can observe a second instance of the profiler being initialized/shutdown that's associated with the "other" runtime process; I've enabled the CORECLR_PROFILER_... environment variables at system level.

I also have a testapp project for app. But I'm unable to profile the test with neither dotnet test nor dotnet vstest (including different combinations of flags like --no-build and --no-restore). I've written test runners in xunit, nunit, and mstest; all of them exhibit the same behavior:

• Most of the times, the profiler crashes dotnet and the tests don't even start or they start but aren't run to completion.
• Some times the tests run all the way through. However, even when that happens, function names of the app aren't output to the console; i.e. I only see function names of the runtime itself and from the test runner. It seems that, even though I can observe the (two instances of the) profiler being initialized/shutdown, the profiler doesn't appear to be monitoring functions from the app.

So...

1. Is there anything peculiar in regards to how dotnet test and dotnet vstest work?
2. What would be the proper/alternative way to profile app when run with dotnet test testapp and dotnet vstest testapp.dll?

If you have any insights and/or guidance on this topic, I'd appreciate your feedback a lot 🙂 .
Thanks!

Note: Both app and testapp are built/target both .NET 7 and 8, and I've built my profiler with the code base of dotnet's runtime tagged at v8.0.0, with ICorProfilerCallback11 (on Windows with arm64), so in theory things should be binary compatible. Also, in Window's event viewer I see information about the crash but no dump is generated. Any ideas for the reason of the crashes?

[ltcmelo]

I believe that I understand (at least part of) what's going on:

• Standard output produced by (second instance of the) profiler in the "other" runtime isn't forwarded by dotnet test and dotnet vstest, even for failed test runs.

I find the behavior of capturing standard output upon successful tests fine. But upon a failing test I'd expect that every standard output is forwarded, not only that produced by the app's assembly.

Now, I can now see, by outputting to a file, function names of the app when run with dotnet test and dotnet vstest.

I couldn't yet find an explanation for the intermittent crashes, though (or for why dumps aren't generated, despite complete dumps configured on Windows).