Ideas To Contribute

[Jonneal3]

Hi Guys - Im going to make some contributions here shortly. A number of things come to mind to get it (to what I want), but I want to also know what other contributions other people want to make the best use of my time in contributing. Here are some that I see most pertinent and intend to focus on:

1 - Native stripe integration - Allowing for you to track credits, tokens etc. to then charge users through stripe
2 - API Routes/Docs - Lots of users (my users at least lol) like the ability to use their current workflow builders (think make, manychat) to allow for conversations from external sources with their bot. This is only possible in having api routing/ingress etc that is usable
3 - Integrations - A tab where you can utilize oauth to connect accounts, to then use those with greater specificity in the tools section (this ties in with agents below)
4 - Agents - This is a bit more complicated, but I'd like to build an agents tab in which the user can utilize their oauth credentials in the setup of an assistant, as to provide those details to then also give this platform the ability to run API requests as the "tool choice" JSON output from the OpenAI request to actually run the request (So you can message right with your software :)
5 - Actions - User controlled steps in a "run" request - Similar to running something like running a Zapier zap
6 - Templates - The ability to add a bunch of pre-built templates to aid in building agents/assistants
7 - Fallbacks - Simply add fallback prompts to assistants and dynamic prompt switching if there is "no match" or low score match with a knowledge vector (to avoid hallucination) allowing a user to instruct an assistant in a more concise manner ( similar to google dialogflow fallbacks)

Also, @mckaywrigley if you are already intending on contributing some of these, let me know as well and maybe we can collaborate on the functionality to not build over each-other :)

Thanks! - Jon

[secondtruth]

A good addition would be support for ChatGPT Plugins, especially third-party plugins that could be added just via the provider URL. This is also a much wanted feature for the legacy version (#207, #448, #123). It could also be extended to a whole plugin store.

[Jonneal3]

@secondtruth I second that one second truth! So ideally both the plugins AND tools within chats?

[secondtruth]

That would be ideal, yes.

I envision that plugins could essentially be used just like the other tools, and ideally multiple tools (be it classic tool or plugin) could be used at the same time.

[secondtruth]

More ideas:

• Upload / Sync files from Nextcloud (etc.)
• Upload multiple files at once
• Generation of file embeddings in the background (with spinner icon for files that are currently being processed)

[Jonneal3]

Yes! Have you heard of nango at all? They handle syncing and oauth as well. I was also thinking that for tool calls, there are many cases where the OpenAPI spec needs to be updated with the bearer token, and nango provides that as well. Im wondering if that would be an interesting implementation. But they also provide file "syncing" as well for software that may require oauth so a user can regularly sync a google sheet or whatnot to the files.

[secondtruth]

Nango looks cool and very useful for this, and I like that it's open source too. Thanks for mentioning it. But sadly there seems to be no Nextcloud integration yet.

[Jonneal3]

@secondtruth You can contact them and request integrations and they have them done in like 1 day. I did that for an integration and it was done in hours. I think they also have a way where you can add your own..

[secondtruth]

@Jonneal3 As you suggested, I requested a Nextcloud integration in Nango's Slack. They replied indeed really fast and the integration is in the works. Soon we're one step further.

[Jonneal3]

@secondtruth Thats amazing man glad to hear!