Extract global timeout as CLI arg

[MichaelThamm]

In slow runners, concierge fails to provision the machine within 270s.

Currently, the command string is hardcoded to "status", "--wait-ready", "--timeout", "270s" which should accept a configurable option for --timeout. This exists in the source code here:

• concierge/internal/providers/microk8s.go

  Line 152 in 723a397

  cmd := system.NewCommand("microk8s", []string{"status", "--wait-ready", "--timeout", "270"})

• concierge/internal/providers/lxd.go

  Line 137 in 723a397

  system.NewCommand("lxd", []string{"waitready", "--timeout", "270"}),

• concierge/internal/providers/k8s.go

  Line 179 in 723a397

  cmd := system.NewCommand("k8s", []string{"status", "--wait-ready", "--timeout", "270s"})

And in tests here:

• concierge/internal/providers/lxd_test.go

  Line 54 in 723a397

  "lxd waitready --timeout 270",

• concierge/internal/providers/microk8s_test.go

  Line 96 in 723a397

  "microk8s status --wait-ready --timeout 270",

• concierge/internal/providers/k8s_test.go

  Line 124 in 723a397

  "snap install kubectl --channel stable",

[benhoyt]

Thanks for the report. Do you know how long these machines would take? If there's some way to test that but using your PR branch, that would be great. It might be simpler just to increase the default timeout.

[MichaelThamm]

I have no idea of knowing how long they should take, on ubuntu-latest it is rare (approx. 1/50 times we get a timeout). I switched to a self-hosted runner with 4x the resources, and now the concierge provisioning times out every time.

I KNOW this is counterintuitive, and I am speaking with the runners team to sort this out. This feature is a tandem initiative, and I would assume something like 600s would be sufficient.

I am no expert in Go and created this PR adhoc to get the ball rolling. Not sure if I will have the time to test this from dev.

[MichaelThamm]

I am no longer blocked when this is fixed:

• Need support for k8s / microk8s local image registry / mirror #75

Not sure if you would still welcome a configurable timeout? Otherwise, feel free to close this issue.