Skip to content

Conversation

aksg87
Copy link
Collaborator

@aksg87 aksg87 commented Aug 4, 2025

Description

Adds comprehensive live API integration tests for Gemini and OpenAI providers to ensure langextract works correctly with real APIs.

This PR introduces a new test suite that validates extraction functionality against actual LLM providers, with proper retry mechanisms for transient errors, cost controls, and CI integration.

Related to #13 (adds test coverage that exposes the tokenizer issue)

Choose one: Testing

How Has This Been Tested?

Tested locally with both providers:

$ tox -e live-api
live-api: commands[0]> pytest tests/test_live_api.py -v -m live_api --maxfail=1
========================== test session starts ==========================
collected 5 items                                                       

tests/test_live_api.py::TestLiveAPIGemini::test_medication_extraction PASSED              [ 20%]
tests/test_live_api.py::TestLiveAPIGemini::test_medication_relationship_extraction PASSED [ 40%]
tests/test_live_api.py::TestLiveAPIGemini::test_multilingual_medication_extraction XFAIL  [ 60%]
tests/test_live_api.py::TestLiveAPIOpenAI::test_medication_extraction PASSED              [ 80%]
tests/test_live_api.py::TestLiveAPIOpenAI::test_medication_relationship_extraction PASSED [100%]

========================= 4 passed, 1 xfailed in 12.34s =========================

Checklist:

  • I have read and acknowledged Google's Open Source Code of conduct.
  • I have read the Contributing page, and I either signed the Google Individual CLA or am covered by my company's Corporate CLA.
  • I have discussed my proposed solution with code owners in the linked issue(s) and we have agreed upon the general approach.
  • I have made any needed documentation changes, or noted in the linked issue(s) that documentation elsewhere needs updating.
  • I have added tests, or I have ensured existing tests cover the changes
  • I have followed Google's Python Style Guide and ran pylint over the affected code.

@aksg87 aksg87 self-assigned this Aug 4, 2025
@aksg87 aksg87 force-pushed the feature/live-api-tests branch from 608d593 to 240394f Compare August 4, 2025 14:04
- Gemini & OpenAI test suites with retry on transient errors
- CI: Separate job, Python 3.11 only, skips for forks
- Validates char_interval for all extractions
- Multilingual test xfail (issue #13)

TODO: Remove xfail from multilingual test after tokenizer fix
@aksg87 aksg87 force-pushed the feature/live-api-tests branch from 240394f to cbf9e1f Compare August 4, 2025 14:18
@aksg87 aksg87 merged commit 87c511e into main Aug 4, 2025
4 checks passed
aksg87 added a commit that referenced this pull request Aug 21, 2025
- Gemini & OpenAI test suites with retry on transient errors
- CI: Separate job, Python 3.11 only, skips for forks
- Validates char_interval for all extractions
- Multilingual test xfail (issue #13)

TODO: Remove xfail from multilingual test after tokenizer fix
sinnaj referenced this pull request in sinnaj/langextract Sep 3, 2025
- Gemini & OpenAI test suites with retry on transient errors
- CI: Separate job, Python 3.11 only, skips for forks
- Validates char_interval for all extractions
- Multilingual test xfail (issue #13)

TODO: Remove xfail from multilingual test after tokenizer fix
aksg87 added a commit that referenced this pull request Sep 12, 2025
- Gemini & OpenAI test suites with retry on transient errors
- CI: Separate job, Python 3.11 only, skips for forks
- Validates char_interval for all extractions
- Multilingual test xfail (issue #13)

TODO: Remove xfail from multilingual test after tokenizer fix
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant