[ci/BuildAndTest]: upload test coredump to artifacts on crash

[xmkg]

The BuıldAndTest(Debug) step crashes intermittently without any significant clue for us to pursue. The crash does not happen in the local environment so the best next thing is to grab the core dump & crashed executable and upload them to artifacts so we can investigate them.

MULTI-1845

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 89.12%. Comparing base (1954ca0) to head (26eb607).
Report is 3479 commits behind head on main.

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #3958   +/-   ##
=======================================
  Coverage   89.11%   89.12%           
=======================================
  Files         255      255           
  Lines       14603    14603           
=======================================
+ Hits        13014    13015    +1     
+ Misses       1589     1588    -1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[xmkg]

I've tested the crash dump capture and upload steps with a mock step (which has been removed now), and it seems to work properly. An example set of artifacts uploaded from the mock crash can be found here

I've tried to retry the pipeline an unhealthy number of times but could not trigger the real crash, but I think the code will work for the real thing as the mock is pretty close to the real thing (observable behavior-wise).

[xmkg]

We discussed this out-of-band with @ricab, and he said that he's OK with the concept, so this PR needs some eyes to confirm that I haven't missed anything obvious.

[Sploder12]

Great work! Looks like there is a minor issue where an if isn't being evaluated as it should, but nothing major!

Somewhat unrelated but for separate future task/PR(s): It'd be cool to have something similar for Windows and MacOS as well. I know Apple Clang tends to crash so retrieving that would be nice to have too.

[xmkg]

Somewhat unrelated but for separate future task/PR(s): It'd be cool to have something similar for Windows and MacOS as well. I know Apple Clang tends to crash so retrieving that would be nice to have too.

That's a good idea. Let's use this patch as a pilot program, land it, and see how well it works. Then, we can convert it to a dedicated action that can be reused in different workflows. Maybe we can even register a coredump handler just like apport to have minimal intrusion on the jobs and still be able to capture the stuff we need to troubleshoot.