Swarm inference UI #23
Description
A buyer should be able to send an inference to any number of connected sellers.
Right now the only buy interface is a standard chatbot with one inference going to one seller.
Conceivably that same chat interface could be extended to also support swarm inference. For now we don't quite know what the appropriate UI for this should be, so we'll do this on a separate page that can be added to the Buy menu when ready.
More on why this is relevant, from recent interview:
I have fun thinking through what basically unlimited swarm AI compute can enable in terms of things that you build on top of that.
One simple thought exercise is, with OpenAI or any of the other larger providers, how do you get 1,000 inferences done in five seconds with different prompts for all of them? If you've got your one API key, you can't do that, but you've got a job that you want to have 1,000 different things said, maybe in different languages or different prompts, different things or different aspects of a scientific calculation that that swarm compute now enables.
That's just one thought. What can people do with that? I don't know, but I think that there's going to be new use cases that get unlocked from this.
Enabling buyers to easily send jobs to all or many sellers could substantially increase the number of jobs received by GPU sellers.