Skip to content

Conversation

@ipriyaaanshu
Copy link

Implement native JSON mode with schema for Gemini

Closes #472

This pull request enhances the Google Gemini integration within Mirascope to support native, schema-constrained JSON outputs, aligning with the latest capabilities of the Gemini API.

Key Changes & Motivations:

  1. Native Gemini JSON Mode (response_schema):

    • When using @llm.call(provider="google", ...) (or the older @google_call) with json_mode=True and a Pydantic response_model, Mirascope now directly passes the Pydantic model to the generation_config.response_schema parameter of the Google Gemini API.
    • This leverages Gemini's built-in mechanism for enforcing that the LLM's output strictly adheres to the provided schema, offering a more direct and potentially more reliable method for structured JSON generation compared to relying solely on prompting or tool calling as a fallback for this specific mode.
    • If json_mode=True is used without a response_model (and no tools are specified), a generic prompt instructing the model to output JSON is still appended to the messages, maintaining existing behavior for that scenario.
  2. Updated Warning for Strict Outputs:

    • The warning message in BaseTool (triggered when model_config={"strict": True} is used on a response_model) has been updated to include "google" in its list of providers that support such strict structured outputs when json_mode=True. This reflects the new capability.
  3. Unit Tests:

    • New unit tests have been added to tests/core/google/_utils/test_setup_call.py to specifically verify the new logic:
      • Ensures response_schema is correctly set with the Pydantic model.
      • Ensures response_mime_type is set to "application/json".
      • Confirms that the generic JSON prompting mechanism (via _utils.json_mode_content) is not invoked when response_schema is active.
    • The new tests align with the existing testing structure and conventions in the file.
  4. Minor Docstring Fix:

    • Corrected a minor typo from json_modem to json_mode in the docstring of google_call in mirascope/core/google/_call.py.

How to Test:

Reviewers can test this by:

  1. Checking out this branch.
  2. Ensuring Mirascope is installed in editable mode with the google extra (uv pip install -e ".[google]").
  3. Setting the GOOGLE_API_KEY environment variable.
  4. Running a test code that uses @llm.call(provider="google", response_model=MyModel, json_mode=True).
  5. Observing that the Gemini API call successfully returns structured data conforming to MyModel.
  6. Running pytest to ensure the new and existing unit tests pass, particularly tests/core/google/_utils/test_setup_call.py.

This change aims to provide a more robust and idiomatic integration with Gemini's evolving features for structured data extraction.

Enables direct use of Pydantic models for response_schema in Gemini API calls when json_mode=True. Includes updates to warnings for strict mode and relevant unit tests.
@ipriyaaanshu ipriyaaanshu requested a review from willbakst as a code owner May 30, 2025 17:24
@ipriyaaanshu
Copy link
Author

@willbakst can you please confirm if my recent commits look good?

@codecov
Copy link

codecov bot commented Jun 2, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 100.00%. Comparing base (9be4325) to head (5d90862).

Additional details and impacted files
@@            Coverage Diff             @@
##              main      #983    +/-   ##
==========================================
  Coverage   100.00%   100.00%            
==========================================
  Files          511       511            
  Lines        21072     21172   +100     
==========================================
+ Hits         21072     21172   +100     
Flag Coverage Δ
tests 100.00% <100.00%> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

]

if json_mode:
if json_mode or (response_model and getattr(response_model, "model_config", {}).get("strict", False)):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the null check on response_model should guarantee the presence of model_config, so you can do something like this:

if json_mode or (response_model and response_model.model_config.get("strict", False)): ...

messages[-1]["parts"].append( # pyright: ignore [reportTypedDictNotRequiredAccess, reportOptionalMemberAccess, reportArgumentType]
PartDict(text=_utils.json_mode_content(response_model))
) # pyright: ignore [reportTypedDictNotRequiredAccess, reportOptionalMemberAccess, reportArgumentType]
config.response_mime_type = "application/json"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this was previously inside of the if not tools: ... check because it's possible to set both response_model and tools. We need to make sure that if both are set we don't force the response into application/json since the model may want to respond with a tool call.

config.response_mime_type = "application/json"
if response_model:
config.response_schema = response_model
elif not tools:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why move this inside of this check for not tools? Previously we've always included this json mode content message.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add support for Gemini Structured Outputs

2 participants