-
Notifications
You must be signed in to change notification settings - Fork 0
Description
Hi, thanks for creating this LiteLLM provider! I got it to work after some tweaks on the lx.extract invocation part.
- The example you have in readme shows outdated config passing to the model invocation. It should be like this in official lang extract repo:
config = lx.factory.ModelConfig( model_id="gpt-4", provider="OpenAILanguageModel", # Explicit provider provider_kwargs={"api_key": "..."} ) model = lx.factory.create_model(config)
So, the provider_kwargs is what you were missing when passing in kwargs to lang extract.
———
2. General question on Lang extract and how fenced_outputs or structured outputs/response model are working under the hood. From my u understanding , Google’s LangExtract does NOT use any explicit JSON mode for any model provider, such as Google Gemini, OpenAI, Anthropic. Or if they are using JSON mode or structured outputs, it’s not clear. Do you know how this works under the hood? I’m asking mainly because when we use LiteLLM, we want to maintain those provider-specific configs, right? It OpenAI needs fenced_outputs=true (whatever this does, still not entirely clear based on docs!) , then we need LiteLLm to detect when OpenAI models are being used and turn on this config. Or if Google Gemini is being used, I want the EXACT configuration that Langextract uses for their flagship Googlegemini implementation—even if I use it through LiteLLM.
Does this make sense?