Skip to content

Conversation

mariano
Copy link
Collaborator

@mariano mariano commented Aug 4, 2025

Fixes #53
Addresses #80, #67, #54

Adds base_url parameter to OpenAILanguageModel, enabling connection to any OpenAI-compatible endpoint including:

  • Azure OpenAI endpoints
  • Local deployments (LocalAI, LMStudio)
  • Cloud provider APIs (AliCloud Modelscope)
  • Corporate/private endpoints

This change follows the OpenAI client API design and maintains full backward compatibility.

Usage Examples

# Standard OpenAI (no change to existing usage)
model = OpenAILanguageModel(api_key="your-openai-key")

# Azure OpenAI
model = OpenAILanguageModel(
    api_key="your-azure-key",
    base_url="https://your-resource.openai.azure.com/"
)

# Local OpenAI-compatible server (LocalAI, LMStudio, etc.)
model = OpenAILanguageModel(
    api_key="not-needed-for-local",  # Some local servers ignore this
    base_url="http://localhost:8080/v1"
)

# Cloud provider with OpenAI-compatible API
model = OpenAILanguageModel(
    api_key="your-provider-key",
    base_url="https://api.provider.com/v1"
)

All tests succeed with: poetry run pytest tests/

@mariano
Copy link
Collaborator Author

mariano commented Aug 5, 2025

@aksg87 how does this look to you? 🤓

@aksg87
Copy link
Collaborator

aksg87 commented Aug 6, 2025

Hi @mariano,

Thanks for this PR! Really appreciate your patience with the review process - things have been a bit hectic with the initial launch and the library suddenly getting a lot more usage than we expected.

The base_url addition is exactly what we needed and your implementation is clean. Tests look good, everything passes, and it's ready to go.

Looking forward to more contributions from you as you run into other issues or have ideas for improvements. Thanks again!

@aksg87 aksg87 self-assigned this Aug 6, 2025
@aksg87 aksg87 self-requested a review August 6, 2025 13:02
@mariano
Copy link
Collaborator Author

mariano commented Aug 6, 2025

Hi @mariano,

Thanks for this PR! Really appreciate your patience with the review process - things have been a bit hectic with the initial launch and the library suddenly getting a lot more usage than we expected.

The base_url addition is exactly what we needed and your implementation is clean. Tests look good, everything passes, and it's ready to go.

Looking forward to more contributions from you as you run into other issues or have ideas for improvements. Thanks again!

Thanks mate! I've been following your gh workflow commits for automating the review process, including the check for community reactions. Slick!

I was thinking of pushing a PR to cache pip dependencies, on my workflows it shaves off a lot of CI time, particularly when you have a python matrix of more than one version. If you agree, I can create an issue + PR for it, following this implementation

@aksg87 aksg87 added the ready-to-merge Triggers live API tests for PRs from forks label Aug 6, 2025
Copy link

github-actions bot commented Aug 6, 2025

Running live API tests... This will take a few minutes.

Copy link

github-actions bot commented Aug 6, 2025

✅ Live API tests passed! All endpoints are working correctly.

@aksg87 aksg87 merged commit 234081e into google:main Aug 6, 2025
17 checks passed
@aksg87
Copy link
Collaborator

aksg87 commented Aug 6, 2025

Hi @mariano, Definitely create an issue for this and feel free to send that PR - because it involves infra files it might cause some failures but I can review it soon. Prioritizing things but that sounds like a great idea on first pass :)

@umbraclet16
Copy link

I notice that top_p is not defined and assigned in the constructor of class OpenAILanguageModel. It can be passed to infer() through kwargs, but lx.extract() does not take kwargs. This could cause error when using some openai-compatible server(e.g. Xinference validates top_p in the request and fails).

aksg87 pushed a commit that referenced this pull request Aug 21, 2025
* Add base_url to OpenAILanguageModel

* Github action lint is outdated, so adapting

* Adding base_url to parameterized test

* Lint fixes to inference_test.py
sinnaj referenced this pull request in sinnaj/langextract Sep 3, 2025
* Add base_url to OpenAILanguageModel

* Github action lint is outdated, so adapting

* Adding base_url to parameterized test

* Lint fixes to inference_test.py
aksg87 added a commit that referenced this pull request Sep 12, 2025
* Add base_url to OpenAILanguageModel

* Github action lint is outdated, so adapting

* Adding base_url to parameterized test

* Lint fixes to inference_test.py
@songjh513
Copy link

Hi, how to use it ?
image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready-to-merge Triggers live API tests for PRs from forks

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat: Add base_url support to OpenAILanguageModel

4 participants