Bug Report: JSON Parsing Error with Gemini 2.5 Flash Model

[ariel-frischer]

Describe the bug

The backend crashes with a JSON parsing error when using Google's Gemini 2.5 Flash model via OpenRouter. The planner agent fails to parse the model's response when creating tasks, resulting in a JSONDecodeError at line 79 of planner_agent.py. The error indicates malformed JSON output from the LLM that's missing a comma delimiter.

backend   | ERROR:    Exception in ASGI application
backend   | Traceback (most recent call last):
backend   |   File "/app/api.py", line 241, in process_query
backend   |     success = await think_wrapper(interaction, request.query)
backend   |               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
backend   |   File "/app/api.py", line 219, in think_wrapper
backend   |     raise e
backend   |   File "/app/api.py", line 204, in think_wrapper
backend   |     success = await interaction.think()
backend   |               ^^^^^^^^^^^^^^^^^^^^^^^^^
backend   |   File "/app/sources/interaction.py", line 162, in think
backend   |     self.last_answer, self.last_reasoning = await agent.process(self.last_query, self.speech)
backend   |                                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
backend   |   File "/app/sources/agents/planner_agent.py", line 264, in process
backend   |     agents_tasks = await self.make_plan(goal)
backend   |                    ^^^^^^^^^^^^^^^^^^^^^^^^^^
backend   |   File "/app/sources/agents/planner_agent.py", line 161, in make_plan
backend   |     agents_tasks = self.parse_agent_tasks(answer)
backend   |                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
backend   |   File "/app/sources/agents/planner_agent.py", line 79, in parse_agent_tasks
backend   |     line_json = json.loads(block)
backend   |                 ^^^^^^^^^^^^^^^^^
backend   |   File "/usr/local/lib/python3.11/json/__init__.py", line 346, in loads
backend   |     return _default_decoder.decode(s)
backend   |            ^^^^^^^^^^^^^^^^^^^^^^^^^^
backend   |   File "/usr/local/lib/python3.11/json/decoder.py", line 337, in decode
backend   |     obj, end = self.raw_decode(s, idx=_w(s, 0).end())
backend   |                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
backend   |   File "/usr/local/lib/python3.11/json/decoder.py", line 353, in raw_decode
backend   |     obj, end = self.scan_once(s, idx)
backend   |                ^^^^^^^^^^^^^^^^^^^^^^
backend   | json.decoder.JSONDecodeError: Expecting ',' delimiter: line 48 column 292 (char 3139)
backend   |
backend   | During handling of the above exception, another exception occurred:
backend   |
backend   | Traceback (most recent call last):
backend   |   File "/usr/local/lib/python3.11/site-packages/uvicorn/protocols/http/h11_impl.py", line 403, in run_asgi
backend   |     result = await app(  # type: ignore[func-returns-value]
backend   |              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
backend   |   File "/usr/local/lib/python3.11/site-packages/uvicorn/middleware/proxy_headers.py", line 60, in __call__
backend   |     return await self.app(scope, receive, send)
backend   |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
backend   |   File "/usr/local/lib/python3.11/site-packages/fastapi/applications.py", line 1082, in __call__
backend   |     await super().__call__(scope, receive, send)
backend   |   File "/usr/local/lib/python3.11/site-packages/starlette/applications.py", line 113, in __call__
backend   |     await self.middleware_stack(scope, receive, send)
backend   |   File "/usr/local/lib/python3.11/site-packages/starlette/middleware/errors.py", line 164, in __call__
backend   |     await self.app(scope, receive, _send)
backend   |   File "/usr/local/lib/python3.11/site-packages/starlette/middleware/cors.py", line 93, in __call__
backend   |     await self.simple_response(scope, receive, send, request_headers=headers)
backend   |   File "/usr/local/lib/python3.11/site-packages/starlette/middleware/cors.py", line 144, in simple_response
backend   |     await self.app(scope, receive, send)
backend   |   File "/usr/local/lib/python3.11/site-packages/starlette/middleware/exceptions.py", line 63, in __call__
backend   |     await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
backend   |   File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app
backend   |     await app(scope, receive, sender)
backend   |   File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 716, in __call__
backend   |     await self.middleware_stack(scope, receive, send)
backend   |   File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 736, in app
backend   |     await route.handle(scope, receive, send)
backend   |   File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 290, in handle
backend   |     await self.app(scope, receive, send)
backend   |   File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 78, in app
backend   |     await wrap_app_handling_exceptions(app, request)(scope, receive, send)
backend   |   File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app
backend   |     await app(scope, receive, sender)
backend   |   File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 75, in app
backend   |     response = await f(request)
backend   |                ^^^^^^^^^^^^^^^^
backend   |   File "/usr/local/lib/python3.11/site-packages/fastapi/routing.py", line 308, in app
backend   |     raw_response = await run_endpoint_function(
backend   |                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
backend   |   File "/usr/local/lib/python3.11/site-packages/fastapi/routing.py", line 219, in run_endpoint_function
backend   |     return await dependant.call(**values)
backend   |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
backend   |   File "/app/api.py", line 281, in process_query
backend   |     sys.exit(1)
backend   | SystemExit: 1

To Reproduce

Steps to reproduce the behavior:

1. Configure config.ini with:
   • is_local = False
   • provider_name = openrouter
   • provider_model = google/gemini-2.5-flash
2. Start services with ./start_services.sh full
3. Send any query that requires the planner agent to create tasks
4. Backend crashes with JSON parsing error

Expected behavior

The planner agent should successfully parse the LLM response and create tasks without crashing, regardless of the model used. The JSON parser should handle variations in model output format or have better error handling for malformed JSON.

LLM Model used

google/gemini-2.5-flash (via OpenRouter)

Desktop (please complete the following information):

• OS: Linux (Arch Linux 6.16.5-arch1-1)
• Browser: N/A (affects backend)
• Version: Latest main branch (commit abc7658)

Additional context

• The issue appears to be model-specific as other models like deepseek-r1:14b work without issues
• Gemini 2.5 Flash may be formatting JSON responses differently than expected by the planner_agent.py parser
• The error occurs specifically when parsing agent tasks in the planning phase
• Possible solutions:
  1. Add better JSON validation/sanitization before parsing
  2. Use a more robust JSON parsing method that handles edge cases
  3. Add model-specific response formatting instructions
  4. Implement retry logic with clearer JSON formatting prompts for problematic models

Root Cause Analysis

The Gemini 2.5 Flash model appears to generate JSON with formatting issues (missing comma delimiter at line 48, column 292). This suggests the model might be:

• Truncating responses mid-JSON
• Including special characters that break JSON syntax
• Using a different JSON formatting style than expected
• Exceeding token limits and cutting off mid-object

Suggested Fix

The parse_agent_tasks function in planner_agent.py should include:

1. Try-catch with JSON repair attempts
2. Response validation before parsing
3. Model-specific prompting to ensure valid JSON output
4. Fallback to a simpler task format if JSON parsing fails