Anyone doing SLO (service-level objectives) with ARC?

[MPV]

I'm looking for others also trying to apply the reliability stack (Error budgets, SLOs and SLIs) to ARC, for sharing knowledge (and using good use cases for driving which/how metrics are available in ARC).

1. What service-level objectives to do have?
2. What service-level indicators do you use?

[MPV]

Hope you don't mind me asking you, as you've also been working in the metrics space, here's me hoping you're also dipping your toes into the SLO field (or at least better alerting and metrics, on top of ARC). CC @zetaab @nikola-jokic @1MaxKoval 🙏

[chlunde]

@MPV have you found any good SLI/SLO?

Based on experience, I'd like to at least detect hanging on "Waiting for a runner to pick up this job..."

And I'd like to measure the time it takes for jobs to start running.