Fix content length mismatch

[RohitPanda]

Title

Fix Content-Length mismatch in pass-through endpoints for AWS services

Relevant issues

Fixes issue with "Too little data for declared Content-Length" errors in AWS SigV4 authenticated services
Similar to the fix implemented for SageMaker in PR #9335

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• [x] I have Added testing in the tests/litellm/ directory,
Adding at least 1 test is a hard requirement - see details
• [x] I have added a screenshot of my new test passing locally
• [x] My PR passes all unit tests on (make test-unit)[https://docs.litellm.ai/docs/extras/contributing_code]
• [x] My PR's scope is as isolated as possible, it only solves 1 specific problem

Type

🐛 Bug Fix

Changes

Problem

When making requests to AWS services through the pass-through endpoints, users encounter errors like:

Too little data for declared Content-Length

Here are specific examples of these errors from actual user reports:

Bedrock Error Example:

DEBUG:LiteLLM:

POST Request Sent from LiteLLM:
curl -X POST
https://bedrock-runtime.us-east-1.amazonaws.com/model/anthropic.claude-v2/invoke
-H 'Content-Type: application/json' -H 'X-Amz-Date: 20250214T123456Z' -H 'Authorization: AWS4-HMAC-SHA256 Credential=AKIAXXXXXXXXXXXXXXXX/20250214/us-east-1/bedrock/aws4_request, SignedHeaders=content-type;host;x-amz-date, Signature=abcdef1234567890abcdef1234567890abcdef1234567890' -H 'Content-Length: 114'
-d '{"prompt":"\n\nHuman: Tell me a short joke\n\nAssistant:","max_tokens_to_sample":50,"temperature":0.7,"top_p":0.9}'

12:34:56 - LiteLLM:DEBUG: utils.py:304 - RAW RESPONSE:
Too little data for declared Content-Length

This occurs because:

1. During AWS SigV4 authentication, the Content-Length header is calculated based on the JSON-serialized request body
2. Later, when the actual HTTP request is made, httpx re-serializes the JSON data using the json parameter
3. The re-serialization might produce a slightly different byte length (due to whitespace, ordering, etc.)
4. This mismatch between the Content-Length header and the actual request body size causes the error

Solution

This PR implements a generic solution that:

1. Pre-serializes the JSON data using json.dumps(_parsed_body)
2. Uses the data parameter instead of the json parameter in httpx requests
3. Applies this approach to both regular and streaming requests
4. Works for all services, not just specific AWS services

Code Changes

For Regular Requests:

python

Before

response = await async_client.request(
method=request.method,
url=url,
headers=headers,
params=requested_query_params,
json=_parsed_body, # httpx re-serializes this, potentially causing mismatch
)

After

json_str = json.dumps(_parsed_body) # Pre-serialize JSON to avoid Content-Length mismatch
response = await async_client.request(
method=request.method,
url=url,
headers=headers,
params=requested_query_params,
data=json_str, # Use data with pre-serialized JSON instead of json parameter
)

For Streaming Requests:

python

Before

req = async_client.build_request(
"POST",
url,
json=_parsed_body, # httpx re-serializes this, potentially causing mismatch
params=requested_query_params,
headers=headers,
)

After

json_str = json.dumps(_parsed_body) # Pre-serialize JSON to avoid Content-Length mismatch
req = async_client.build_request(
"POST",
url,
data=json_str, # Use data with pre-serialized JSON instead of json parameter
params=requested_query_params,
headers=headers,
)

Testing

I've added unit tests that verify:

1. Content-Length consistency when using pre-serialized JSON with the data parameter
2. The difference between using json parameter vs. pre-serialized JSON with data parameter
3. How this approach fixes the AWS SigV4 authentication issue

All tests pass successfully, confirming that the fix works as expected.

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
litellm	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Apr 24, 2025 9:31am

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.

---

Rohit Panda seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[krrishdholakia]

move tests inside mock tests folder