[Q] How to connect/ enable LLM Proxy

[ly000]

Hi everbody,
im struggling to initialize the demo project using a llm proxy, which has access to a few options which are supported my ExtractThinker. In my mind the setup should be equal to setting up the "Local Demo" - Setting the API_BASE env variable and the Key and choosing one of the models provided by my proxy.
e.g.

# Load local LLM
load_dotenv()
llm = LLM('azure/gpt-4o') # Or whatever other model my proxy provides.
extractor.load_llm(llm)

What am i doing wrong, or is this not supported?

Thanks for the help

[enoch3712]

Hello @ly000 !

Yes! Its a bit tricky Azure:

https://docs.litellm.ai/docs/sdk_custom_pricing

Remember, ExtractThinker has 2 drivers in the LLM component, LilteLLM by default, and pydantic AI.

LLM component