model benchmarks for various sampling strategies?

[okuvshynov]

Let's say you pick a model (llama3-8b) and run evaluation of benchmarks typically reported by model creators (MMLU, HumanEval, etc.) with different sampling strategies (for example, do beam search with widths w1, w2, w3, ..) - how different would evaluation look like?

Are there any existing results?
Thank you.

[ggerganov]

AFAIK benchmarks do not use sampling. Not sure what is HumanEval, but MMLU does not involve sampling

[okuvshynov]

Thank you! Sorry i missed your reply