Express Benchmarking Machine

[bensternthal]

Express is requesting a benchmarking machine.

First step is figuring out detailed requirements.

[bensternthal]

@UlisesGascon can you outline your requirements for this machine?

[UlisesGascon]

Not sure if we are yet clear on the specs yet (we are starting the wg soon) (cc: @wesleytodd )

[mcollina]

If this is somewhat automated to use, Fastify could use help with that as well. Currently, we are running benchmarks at https://github.com/fastify/benchmarks, and the results are collected on a GitHub action machine.

[bensternthal]

@UlisesGascon do you have an update on this one?

[bensternthal]

@UlisesGascon can you help me to understand your requirements, or is there someone else I should contact?

[wesleytodd]

This is blocked for now. We prioritized the release over the Perf WG and this need is going to be owned by that group (which is also part of our STF work). I will follow up again here when we have more details on the plan, but my goal is to make a reasonable tradeoff between cost, operational complexity, and reliable results. This may mean we are looking for something somewhere between GHA benchmarks (low cost and complexity but poor reliability) and a dedicated machine (high cost and complexity but the best reliability).

@mcollina I would be interested in hearing how you have found the reliability of GHA runners. Maybe this thread is not the best place to sidetrack on that?

[bensternthal]

@wesleytodd understood, and thanks for the update. I'll put this on the back-burner for now. Let us know when you are ready to move forward.