Ollama Bridge

A lightweight proxy service that normalizes and optimizes OpenAI-compatible API requests, enabling clients to work seamlessly with Ollama and other OpenAI-compatible APIs.

Problem Statement

Many AI clients send requests with:

• Excessive token requests (often 200,000+ tokens)
• Inconsistent parameter formats
• Unnecessary or incompatible fields

This causes 400 errors or poor performance when connecting to OpenAI-compatible servers like Ollama.

Solution

This bridge acts as a middleware that:

1. Intercepts requests from AI clients
2. Normalizes request parameters
3. Caps unreasonable values (token limits)
4. Forwards clean OpenAI-format requests
5. Returns responses clients understand

Architecture

AI Client → Bridge (port 8443) → Ollama/OpenAI (your server)
         ↓
    [Raw request]
         ↓
    [Normalization]
         ↓
    [OpenAI format]

Quick Start

1. Clone and setup:

git clone <repository>
cd ollama-bridge
cp .env.example .env

2. Configure your target:

# Edit .env file
BRIDGE_TARGET_URL=http://host.docker.internal:11434/v1  # For local Ollama
# or
BRIDGE_TARGET_URL=https://api.openai.com/v1  # For OpenAI

3. Start the bridge:

docker-compose up -d

4. Use the bridge: Your bridge is now running on http://localhost:8443

Point your AI client to http://localhost:8443/v1 instead of the direct API.

Configuration

Environment Variables

Variable	Description	Default	Example
BRIDGE_TARGET_URL	Where to forward requests	http://host.docker.internal:11434/v1	https://api.openai.com/v1
BRIDGE_DEBUG	Enable debug logging	false	true
TARGET_API_KEY	API key for target service	-	sk-your-key-here

Example Configurations

Local Ollama:

BRIDGE_TARGET_URL=http://host.docker.internal:11434/v1
BRIDGE_DEBUG=false

Remote Ollama:

BRIDGE_TARGET_URL=http://your-server:11434/v1
BRIDGE_DEBUG=false

OpenAI:

BRIDGE_TARGET_URL=https://api.openai.com/v1
TARGET_API_KEY=sk-your-openai-key-here
BRIDGE_DEBUG=false

API Compatibility

Supported Transformations

Field	Action
temperature	Preserved
max_tokens	Capped at reasonable limit (4096 if > 100,000)
top_p, frequency_penalty, presence_penalty	Preserved
stream, stop, n	Preserved
Unsupported fields	Filtered out

Request Example

Input (from client):

{
  "model": "llama3.1:latest",
  "messages": [{"role": "user", "content": "Hello"}],
  "max_tokens": 229018,
  "temperature": 0.7
}

Output (to Ollama):

{
  "model": "llama3.1:latest",
  "messages": [{"role": "user", "content": "Hello"}],
  "max_tokens": 4096,
  "temperature": 0.7
}

Health Check

Check if the bridge is running:

curl http://localhost:8443/health

Expected response:

{
  "status": "healthy",
  "bridge": "ollama-bridge",
  "target": "http://host.docker.internal:11434/v1",
  "uptime": 123.45
}

Testing

Test the bridge with a simple request:

curl -X POST http://localhost:8443/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer your-api-key" \
  -d '{
    "model": "llama3.1:latest",
    "messages": [{"role": "user", "content": "Hello"}],
    "temperature": 0.7
  }'

Debugging

Enable Debug Mode

# In .env file
BRIDGE_DEBUG=true

# Restart container
docker-compose restart

View Logs

# Follow logs
docker-compose logs -f

# View recent logs
docker-compose logs --tail=50

Performance

• Latency: +5-10ms per request
• Memory: ~50MB RAM usage
• Throughput: Handles concurrent requests
• Streaming: Supports Server-Sent Events (SSE)

Use Cases

• AI client optimization for better Ollama compatibility
• API normalization layer for consistent requests
• Development testing with different backends
• Token limiting to prevent excessive resource usage

Docker Commands

# Build and start
docker-compose up -d

# View logs
docker-compose logs -f

# Stop
docker-compose down

# Rebuild
docker-compose build --no-cache

# Shell access
docker-compose exec ollama-bridge sh

Troubleshooting

Common Issues

Bridge not responding:

• Check if port 8443 is available: netstat -tulpn | grep 8443
• Verify Docker container is running: docker-compose ps

Connection to target fails:

• Test target URL directly: curl $BRIDGE_TARGET_URL/models
• Check network connectivity from container
• Verify API key if required

400 errors from target:

• Enable debug mode to see request transformation
• Check target API documentation for supported fields

License

MIT License - See LICENSE file for details