feat: tools structured response #3165
Closed
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
kausmeows
changed the title
kausmeows
changed the title
dirkbrnd
reviewed
May 21, 2025
libs/agno/agno/agent/agent.py
Outdated
| self.memory.add_run(session_id=session_id, run=run_response) | ||
|
|
||
| await self._amake_memories_and_summaries(run_messages, session_id, user_id, messages) # type: ignore | ||
| # type: ignore |
dirkbrnd
reviewed
May 21, 2025
libs/agno/agno/agent/agent.py
Outdated
| content=introduction, | ||
| messages=[ | ||
| Message(role=self.model.assistant_message_role, content=introduction) # type: ignore | ||
| # type: ignore |
dirkbrnd
reviewed
May 21, 2025
| # Return the message | ||
| return message | ||
|
|
||
| # This gives the error- agno.exceptions.ModelProviderError: Invalid 'messages[4]'. Image URLs are only allowed for messages with role 'user', but this message with role 'tool' contains an image URL. |
Contributor
There was a problem hiding this comment.
We'll have to create a separate message for "user" with the image, as if the user sent an image after the tool result. Still not sure whether it would work, but worth a try.
dirkbrnd
reviewed
May 21, 2025
| Returns: | ||
| str: str: A message indicating if the image has been generated successfully or an error message. | ||
| """ | ||
| def create_image(self, agent: Union[Agent, Team], prompt: str) -> FunctionCallResult: |
Contributor
There was a problem hiding this comment.
You still need the docstring, for the model
dirkbrnd
reviewed
May 21, 2025
| def create_image(self, agent: Union[Agent, Team], prompt: str) -> FunctionCallResult: | ||
| if not self.api_key: | ||
| return "Please set the OPENAI_API_KEY" | ||
| return FunctionCallResult(content="Please set the OPENAI_API_KEY") |
Contributor
There was a problem hiding this comment.
We don't have to always use the FunctionCallResult, strings and other types should still be allowed.
dirkbrnd
reviewed
May 21, 2025
| image_artifact = ImageArtifact( | ||
| id=str(uuid4()), url=img.url, original_prompt=prompt, revised_prompt=img.revised_prompt | ||
| ) | ||
| agent.add_image(image_artifact) |
Contributor
There was a problem hiding this comment.
Ideally we won't do this anymore but automatically handle it in the model/agent. But yeh for simplicity lets keep it here for now. We can clean up in Agno 2.0
dirkbrnd
reviewed
May 21, 2025
| # Create a more descriptive response that includes details about the image | ||
| content = f"The image shows exactly what was requested - {prompt}. " | ||
|
|
||
| return FunctionCallResult(content=content, images=generated_images if generated_images else None) |
Contributor
Author
|
closing in favour of- #4286 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
passing the user uploaded images in the toolkit
(If applicable, issue number: #____)
Type of change
Checklist
./scripts/format.shand./scripts/validate.sh)Additional Notes
Add any important context (deployment instructions, screenshots, security considerations, etc.)