Can I change the OpenAI API key at runtime?

[cvansteenburg]

Context

I'm working with LCEL in Python. I'm using the ChatOpenAI class (source) within FastAPI. The code initializes and then I want to pass different api keys at runtime, depending on the task I'm performing. ConfigurableField (docs) allows me to change any config except openai_api_key.

>> Is there something in the codebase that forbids this? How can I change the openai_api_key at runtime?

This works

model = ChatOpenAI(temperature=0).configurable_fields(
    temperature=ConfigurableField(
        id="llm_temperature",
        name="LLM Temperature",
        description="The temperature of the LLM",
    )
)
model.with_config(configurable={"llm_temperature": 0.9}).invoke("foo_prompt")

But this doesn't

(It uses "placeholderkey" instead of the desired "somevalidkey")

model = ChatOpenAI(openai_api_key="placeholderkey").configurable_fields(
    openai_api_key=ConfigurableField(
        id="openai_api_key",
        name="API Key",
        description="The API key used when making a call to OpenAI",
    )
)
model.with_config(configurable={"openai_api_key": somevalidkey}).invoke("foo_prompt")

Thank you for any support - much appreciated!

[dosubot[bot]]

🤖

Hey there, @cvansteenburg! Back at it with the good questions, I see. Let's dive into it! 😎

Based on the current design of the ChatOpenAI class, it seems that the openai_api_key is not designed to be changed at runtime. The openai_api_key is set when the ChatOpenAI instance is created and is used to authenticate requests to the OpenAI API. It is not included in the configurable_fields method, which is why you are unable to change it at runtime using the with_config method.

The configurable_fields method is designed to allow certain properties of the ChatOpenAI instance to be changed at runtime. These properties are defined in the ConfigurableField instances that are passed to the configurable_fields method. However, the openai_api_key is not included in these properties, which is why you are unable to change it at runtime.

If you need to use different API keys for different tasks, you might need to create separate ChatOpenAI instances for each task, each with its own openai_api_key. Here is an example:

model1 = ChatOpenAI(openai_api_key="key1")
model2 = ChatOpenAI(openai_api_key="key2")

# Use model1 for task1
response1 = model1.invoke("foo_prompt")

# Use model2 for task2
response2 = model2.invoke("bar_prompt")

In this example, model1 and model2 are separate ChatOpenAI instances, each with its own openai_api_key. You can use model1 for one task and model2 for another task, and they will use their respective API keys when making requests to the OpenAI API.

I hope this helps! If you have any further questions, feel free to ask.

Best,
Dosu

Sources

• libs/langchain/tests/integration_tests/chat_models/test_openai.py
• libs/community/langchain_community/chat_models/openai.py

---

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

[cvansteenburg]

Thank you, but I don't think this answers my question.

My goal is fully dynamic configuration of the API key like every other configurable. I'm hoping someone can help me understand whether this is intended behavior, or a bug / side effect. (Certainly it's not ideal behavior to make the exclusion silent?)

Thank you in advance for any guidance from one who knows the codebase better than I!

Additional context

It looks like ConfigurableField allows these fields to be defined by the user, so no property (including openai_api_key is explicitly provided.

A potential culprit for the openai_api_key's silent exclusion would be lc_secrets which makes openai_api_key non-serializable, but I don't know if that actually has an impact.

From the ChatOpenAI class:

    @property
    def lc_secrets(self) -> Dict[str, str]:
        return {"openai_api_key": "OPENAI_API_KEY"}

I believe the relevant code is here:

Source: ChatOpenAI Class

and here:

Source: RunnableSerializable

class RunnableSerializable(Serializable, Runnable[Input, Output]):
    """A Runnable that can be serialized to JSON."""

    def configurable_fields(
        self, **kwargs: AnyConfigurableField
    ) -> RunnableSerializable[Input, Output]:
        from langchain_core.runnables.configurable import RunnableConfigurableFields

        for key in kwargs:
            if key not in self.__fields__:
                raise ValueError(
                    f"Configuration key {key} not found in {self}: \
                    available keys are {self.__fields__.keys()}"
                )

        return RunnableConfigurableFields(default=self, fields=kwargs)
      ```

[cvansteenburg]

Fixed this in my code. Opened issue #16567 to see if there's appetite for a PR. Closing this discussion.

[rarchitgupta]

@cvansteenburg seems like there was no PR for this, could you show me what changes you made to your particular code to get this to work?

[MahmutAtia]

did they fix this ?
this code is from their server examples but still not working
https://github.com/langchain-ai/langserve/blob/main/examples/configurable_chain/server.py

dynamic_auth_model = ChatOpenAI(openai_api_key="placeholder").configurable_fields(
openai_api_key=ConfigurableField(
id="openai_api_key",
name="OpenAI API Key",
description=("API Key for OpenAI interactions"),
),
)

[FlorianOttenwaelder]

Has anyone tried this with the new Langchain V3 version? I'd love to know if there have been any changes to how API key configuration works with the update. Thanks in advance for sharing any experiences or insights!

[payalsaraljain]

Hi, any update how can we solve this issue, I am facing the same issue