Skip to content

Mode BEDROCK_TOOL doesn't work with async-boto3 client #1496

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
3 tasks
ebamberg-business opened this issue Apr 21, 2025 · 0 comments
Open
3 tasks

Mode BEDROCK_TOOL doesn't work with async-boto3 client #1496

ebamberg-business opened this issue Apr 21, 2025 · 0 comments
Labels
bug Something isn't working

Comments

@ebamberg-business
Copy link

  • [ X] This is actually a bug report.
  • [ -] I am not getting good LLM Results
  • [ -] I have tried asking for help in the community on discord or discussions and have not received a response.
  • [ X] I have tried searching the documentation and have not found an answer.

What Model are you using?

  • gpt-3.5-turbo
  • gpt-4-turbo
  • gpt-4
  • [ X] Other Bedrock with Claude

Describe the bug
AsyncInstructor with bedrock-async-boto3 client throws exception when used in BEDROCK_TOOL Mode.

StructuredOutput.py", line 95, in invoke_with_schema
response = await client.chat.completions.create(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
...<6 lines>...
)
^
File "/Users/ebamberg/dev/src/analytics-service/venv/lib/python3.13/site-packages/instructor/client.py", line 397, in create
return await self.create_fn(
^^^^^^^^^^^^^^^^^^^^^
...<8 lines>...
)
^
File "/Users/ebamberg/dev/src/analytics-service/venv/lib/python3.13/site-packages/instructor/patch.py", line 161, in new_create_async
response = await retry_async(
^^^^^^^^^^^^^^^^^^
...<9 lines>...
)
^
File "/Users/ebamberg/dev/src/analytics-service/venv/lib/python3.13/site-packages/instructor/retry.py", line 282, in retry_async
raise InstructorRetryException(
...<9 lines>...
) from e
instructor.exceptions.InstructorRetryException: 'dict' object has no attribute 'choices'

To Reproduce
patch a async bedrock client with the following code:
def from_bedrock_async(
client: BaseClient,
mode: instructor.Mode = instructor.Mode.BEDROCK_TOOL,
**kwargs: Any)-> instructor.AsyncInstructor:
async def async_wrapper(*args: Any, **kwargs: Any): # type:ignore
return await client.converse(*args, **kwargs) # type:ignore

return instructor.AsyncInstructor(
        client=client,
        create=instructor.patch(create=async_wrapper, mode=mode),
        provider=instructor.Provider.BEDROCK,
        mode=mode,
        **kwargs,
)

Set the mode to BEDROCK_TOOL and try to get any structured output

Expected behavior
return output without exception

Screenshots

@github-actions github-actions bot added the bug Something isn't working label Apr 21, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant