gpt_sampler_sample or llama_sampler_sample?

[raulod]

I have noticed both gpt_sampler_sample and llama_sampler_sample are used in examples/*. Some samples use gpt_sampler_sample while other samples use llama_sampler_sample.

Does anyone know why different APIs are used in different samples for the same purpose? Which one is the recommended API?

Thanks

[ngxson]

gpt_sampler_sample is a cpp wrapper for llama_sampler_apply. It was there to maintain compatibility for examples after this refactoring. Please note that, all gpt_* functions are provided by common library, which is not part of core library, so undocumented breaking changed are expected in the future.

The recommended way is always using llama_*. This can prevent you from breaking changes in the future (or even if there're breaking changes, we will document it more carefully than anything in common.)

[raulod]

Thank you for answering. Since the gpt_* functions have more functionalities, what would you suggest? Should I copy the code from the common to the private codebase, or by any chance do you plan to integrate the gpt_* wrappers into the core library?

Also, is a PR required to migrate examples/* from gpt_* to llama_* functions?

[ngxson]

Should I copy the code from the common to the private codebase

Yes you can, the code in common is an example of how you can use llama_ in your final code

by any chance do you plan to integrate the gpt_* wrappers into the core library?

For now, we don't have any plan to do so

[raulod]

I looked into the common code and found many llama_* functions too, for example, llama_batch_add. what about them?

[ngxson]

These are wrappers too. To correct what I said earlier: you should use API exposed by llama.h

It's possible to use APIs exposed by common.h, but not recommended because breaking changes will not be documented.

[raulod]

Got it, thanks! However, for the sake of clarity, I'd recommend renaming the llama_* functions in common library to to gpt_*, utils_*, or something that llama team feel appropriate. This would help maintain a clear distinction between core library functions and wrappers.

Do you think a PR is needed for this?

[ngxson]

I have no strong preference, but I don't think it's necessary to change. It's not a big problem for now and doing naming change will break many other PRs