How to connect Mixtral 8x7b instructor in windows application using C++?

[aitechguy0105]

I want to use Mixtral8x7b instrutor in windows application.
Is it possible to connect using C++?

[jetro30087]

Yes, llama.cpp can do that.

[aitechguy0105]

Thanks. I have implemented that. Then is it possible to connect with T5 model?
I would really appreciate it if you could provide example.

[jetro30087]

I've never tried connecting two models. But you probably could just load two into memory if you have enough ram. If that doesn't work, then loading/unloading the appropriate model in your program as needed would work just fine. I don't really have an example to provide, but the repo provides example programs for every feature they have in the examples folder.

[aitechguy0105]

Thanks for your kind help.
I will study examples.
Thanks.