Update to the Elasticsearch benchmark

[xeraa]

Hey everyone,

Thanks a lot for including Elasticsearch in the benchmark!

While skimming through the results and setup, I noticed a few things that looked a bit off:

1. 8.6 was released 2.5 years ago. Would it be possible to use the current version 9.0 (or very soon 9.1)?
2. For logs we've added a specific indexing mode called logsdb. Only that will give you a good representation here; and with 9.0+ it's enabled for the logs-*-* index pattern by default when using "_op_type": "create", in your yield (so you'd need to add this setting and adjust the index name). Full details in https://www.elastic.co/docs/manage-data/data-store/data-streams/logs-data-stream
3. Elasticsearch might behave a bit differently than the other systems in terms of resource consumption: Whatever you allocate to it will get used. Why give it resources to let them sit idle after all? If you want to keep the benchmark comparable across systems for performance vs resource use, it would make sense to limit what's available to the container. This doesn't even require Elasticsearch specific settings, just a reasonable limit on the container configuration.

Thanks a lot,
Philipp

[kirkrodrigues]

Hi @xeraa,

Thanks for the feedback and suggestions for improvement!

1. Yes, we'll run with the latest available and update the results. Apologies for the oversight. We've been developing this benchmark sporadically over a long time, so some versions went out of sync.
2. Cool! We'll try it out and perhaps add it as another config of Elasticsearch in the benchmark.
3. Yes, this is something we want to extend the benchmark to cover. Our rationale for not limiting memory usage was to let tools that need the memory to avoid getting throttled by swapping. However, as you point out, some tools will take advantage of the memory. In the future, we'll use more configs to hopefully make this clearer.

Let us know if you have any other feedback.