llama_synchronize(): when one should use it?

[raulod]

I noticed llama_synchronize() used in a few samples and not all.

Can someone explain what does it do and when one should use it?

[slaren]

Some functions like llama_decode may return before the computation is finished, llama_synchronize waits until all pending operations are completed. Generally, you should only need to call llama_synchronize directly when you want to measure the time of an operation, otherwise it is called automatically when fetching the results of the operation with functions such as llama_get_logits.