OpenAI API key shared among all users

[LeoLee429]

What Stanford Spezi module is your challenge related to?

SpeziLLM

Description

By my understanding, the current usage is mainly asking users for OpenAI API key, then store them in the keychain for future retrieval.

However, our app would like to provide LLM conversation with only 1 API key so every users can use without any technical knowledge. (e.g. how to generate an API key... Most of our clients would be elderlies) Hardcoding API key directly in the APP is a bad practice.

The only way I can think of is to run LLM module with a proxy, which is host at our backend. I cannot find any guidelines to use the LLM package with proxy. Probably it requires an endpoint injection, and I will try to figure out how to make a proxy in Firebase cloud function.

Please help with this idea, or point me out if there is better way to achieve such functionality with current codes. Thank you so much!

Reproduction

N/A

Expected behavior

N/A

Additional context

No response

Code of Conduct

• I agree to follow this projects's Code of Conduct and Contributing Guidelines

[PSchmiedmayer]

@LeoLee429 Thank you for asking the question; @philippzagar can provide more context about this. But in a few sentences: You are right; creating a proxy is the way to go. You can manually inject a custom token in the model parameters (LLMOpenAIParameters ) and pass in a custom API Key that is relevant for your proxy server and might be tied to your account. Our SpeziLLMFog target provides a mechanism to easily connect to a proxy server or elements in the local network to use for these use cases: https://swiftpackageindex.com/StanfordSpezi/SpeziLLM/documentation/spezillmfog.

@philippzagar I think we can clarify this better and ideally highlight that the usage of a locally stored token should only be considered for testing & small-scale use cases. Maybe we should even aim to add documentation warnings about this and add the custom token injection right in the OpenAI target? I think we might also want to consider how we can more closely align fog and OpenAI.

[lukaskollmer]

to add to this, the App Store review team seems to be moving towards being more strict w.r.t. not allowing apps to let users enter their own API keys, so adding some kind of (external?) mechanism for dealing with (working around) this would be required anyway for some apps

[LeoLee429]

Thanks @PSchmiedmayer @lukaskollmer for the answer.

However, we would not host a Fog node in the local network since we do not have a server. Also, our users will not only connect to one LAN. I had a look at the documentation and I only found parameter to define local node to be found by mDNS(LLMFogPlatformConfiguration.host). I guess it cannot be pointed to an external URL. Even if it is possible, I will still have to figure out how to authenticate with Firebase to call such cloud function.

Since all of our functions will be host on Firebase (and thank you for the integration with Firebase!), I think it is not possible now.

[PSchmiedmayer]

@philippzagar Can add some context about this; I think we would be able to make some simple modifications to make this work. Ideally we would try to align the fog and OpenAI version in the near future.

[philippzagar]

Hey @LeoLee429,

Thank you for your interest in integrating SpeziLLM into your application! 🚀

To expand on @PSchmiedmayer’s earlier response, the recommended approach for your use case is to utilize a proxy service to manage your OpenAI keys securely. Your mobile application would then authenticate with this proxy using a mechanism such as Firebase Authentication tokens.

Our SpeziLLMFog module provides exactly this functionality, authorizing with a Firebase token on a locally deployed proxy service that then forwards the request to the actual inference layer in the case of a valid token. It additionally supports automatic discovery of fog nodes on the local network, which can host and serve inference requests. Detailed documentation is available here.

Currently, SpeziLLMOpenAI does not support this proxy-based approach out of the box. However, it could be extended with small changes by reusing patterns already used in SpeziLLMFog:

• Custom Endpoint Support:
  Allow configuration of a custom OpenAI-compatible endpoint, such as your proxy server.
• Token Injection via Closure:
  Provide an option to define a closure for fetching a dynamic or rotating token. This token would be sent to the proxy using the standard OpenAI bearer token field. For reference, similar functionality exists in our Fog layer:
  • Auth Middleware Example
  • Fog Parameters Example

Optionally, we could also support passing arbitrary headers for greater flexibility.

• Clarified Documentation:
  Update SpeziLLMOpenAI documentation to emphasize that proxy-based setups are recommended for production, particularly when avoiding direct API key exposure.

This enhancement aligns well with ongoing discussions and related feature requests:

• Discussion on aligning Fog and OpenAI layers
• Issue: Code sharing between Fog and OpenAI layers

These improvements would involve little effort and would not introduce breaking changes. I plan to work on implementing this functionality in the coming days. Thank you for identifying this gap and helping us improve the framework!