[LEP-2083] fix(session): async "triggerComponent" to prevent blocking checks

[gyuho]

A manual triggerComponent/triggerComponentCheck request against the disk component blocked GPUd's control-plane session loop. While the disk health probe sat in a long retry cycle, no responses were written back to the control plane, so GPUd appeared healthy locally but the control plane stayed stuck on stale health data.

Issue

• Control-plane machine object continued to show nfs health at 2025-09-18T05:48:16Z, even though GPUd's local disk state had fresh timestamps (for example 2025-09-18T12:07:16Z).
• GPUd logs repeated:
  • session reader: error decoding response: EOF
  • session writer: error making request ... context canceled
  • drained stale messages from reader channel during keep-alive resets.

Root Cause

1. Session.serve processes every control-plane message in a single goroutine. Before the fix it handled triggerComponent synchronously, so it waited for comp.Check() to finish before writing any response.
2. components/disk/component.go retries disk.GetPartitions up to five times per tracked mount, each wrapped in a one-minute timeout. A flaky /mnt/nfs-share can therefore block Check() for roughly five minutes per mount.
3. While Check() blocks, the writer goroutine cannot drain the 20-item s.writer channel. The control plane eventually cancels the streaming HTTP request, triggering the session reader/session writer error messages and the drained stale messages warning when the keep-alive loop tears everything down.
4. The disk component's background ticker keeps updating lastCheckResult, so GPUd's local state reflects the latest degraded result. However, the control plane never receives that update because the synchronous response never flushes.

One manual triggerComponent aimed at the disk component is sufficient to deadlock the session loop. The request is enqueued, Session.serve synchronously calls the long-running disk.Check(), and the writer never sends a response. When the control plane cancels the stuck request, GPUd restarts the session and the control plane still holds the last good health snapshot.

Fix

• triggerComponent / triggerComponentCheck now capture the target component names, enqueue an immediate acknowledgement (with empty States placeholders) back to the control plane, and launch the expensive comp.Check() work in a separate goroutine. This keeps the session writer responsive even when disk checks spend minutes retrying disk.GetPartitions.

[codecov]

Codecov Report

❌ Patch coverage is 69.44444% with 11 lines in your changes missing coverage. Please review.
✅ Project coverage is 67.29%. Comparing base (70316cf) to head (84e2f4b).
⚠️ Report is 3 commits behind head on main.

Files with missing lines	Patch %	Lines
pkg/session/serve.go	69.44%	8 Missing and 3 partials ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1078      +/-   ##
==========================================
+ Coverage   67.27%   67.29%   +0.02%     
==========================================
  Files         316      316              
  Lines       26312    26342      +30     
==========================================
+ Hits        17702    17728      +26     
- Misses       7726     7729       +3     
- Partials      884      885       +1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.