Claude Sonnet 3.7 Model Not Supported in aws_bedrock_claude_pipeline.py Due to Inference Profile Requirement

[samuelpetermoshi]

hello @g453030291,

Issue Summary

OpenWebUI is currently unable to use the Claude Sonnet 3.7 model via aws_bedrock_claude_pipeline.py. The model does not appear in the available model list due to a filtering condition and fails invocation due to a missing inference profile.

• https://github.com/open-webui/pipelines/blob/main/examples/pipelines/providers/aws_bedrock_claude_pipeline.py

---

Root Cause Analysis (RCA)

1. Model Filtering Excludes Claude Sonnet 3.7

The pipeline script retrieves available models using the following line:

https://github.com/propertyguru/ds-openwebui/blob/bbd5eba50ea32a3e7e4e2e526a5794679d401b2b/pg_openwebui_pipelines/pipelines/aws_bedrock_claude_pipeline.py#L150

response = self.bedrock.list_foundation_models(byProvider='Anthropic', byInferenceType='ON_DEMAND')

However, Claude Sonnet 3.7 is not an on-demand model. As a result, it is excluded from the model list due to this filtering.

---

2. Invocation Requires an Inference Profile

Even when the model is manually added or the filtering condition is removed, invocation fails with the following error:

Invocation of model ID anthropic.claude-3-7-sonnet-20250219-v1:0 with on-demand throughput isn’t supported. Retry your request with the ID or ARN of an inference profile that contains this model.

This confirms that Claude Sonnet 3.7 must be invoked using an inference profile ARN, rather than via the default on-demand method.

---

Suggested Fix

• Update the model listing logic to include models that are not on-demand (e.g., by removing or making byInferenceType optional).
• Support invocation using an inference profile ARN for models that require it, like Claude Sonnet 3.7.

Reference video explaining the inference profile flow:
📹 https://youtu.be/7-BIpBn9ao4?si=HZksyMVVR__eKfhv&t=343

---

Impact

• Claude Sonnet 3.7 is currently inaccessible from OpenWebUI.
• Users cannot leverage the latest capabilities from the Claude 3.x series unless the above is addressed.

cc: @tjbck

[samuelpetermoshi]

Hello @g453030291 and @tjbck
Do you happen to have any updates on this ? :)