API Reference | Swagger

🔥 Why OpenGateLLM?

• 🌐 OpenAI standards: based on OpenAI API conventions. Easy to use with OpenAI SDKs, LangChain, LlamaIndex, etc.
• 🚦 Robust API Gateway: Load balancing, authentication, and seamless integration with OpenAI, vLLM, HuggingFace TEI.
• 📖 Open Source: developed by the French Government, fully open-source forever.
• ⚙️ Production-ready: ready to serve your models in production.
• 📚 Full stack genAI API: Built-in Retrieval-Augmented Generation (RAG), OCR, audio transcription, and more.
• ✍️ High code standards

🎯 Key Features

API Gateway: supported providers

Provider	Supported endpoints
Albert API (French Public Administration)	Language, Embeddings, Reranking, Transcription
OpenAI	Language, Embeddings, Reranking, Transcription
vLLM	Language
HuggingFace TEI	Embeddings, Reranking

Advanced AI Capabilities

• RAG Integration: Efficiently query vector databases using Elasticsearch or Qdrant.
• Audio & Vision: Transcribe audio (Whisper) and perform OCR on PDF documents.
• Enhanced Security: Built-in API key authentication.

📊 Comparison

Feature	OpenGateLLM ✅	LiteLLM	OpenRouter	OpenAI API
Fully Open Source	✔️	❌	❌	❌
Data Sovereignty	✔️	✔️	❌	❌
Multiple AI Backends	✔️	✔️	✔️	❌
Built-in RAG	✔️	❌	❌	❌
Built-in OCR	✔️	❌	❌	❌
Audio Transcription	✔️	❌	❌	❌
Flexible Deployment	✔️	✔️	❌	❌
OpenAI Compatibility	✔️	✔️	✔️	❌

📘 Tutorials & Guides

Explore practical use cases:

• Chat Completions
• Multi-Model Access
• Retrieval-Augmented Generation (RAG)
• Knowledge Database Import
• Audio Transcriptions
• PDF OCR

🚀 Quickstart

Deploy OpenGateLLM quickly with Docker connected to our own free model and start using it:

make quickstart

Note

It will copy the config.example.yml and .env.example files into config.yml and .env files if they don't already exist.

Tip

Use make help to see all available commands.

Test the API:

curl -X POST "http://localhost:8080/v1/chat/completions" \
-H "Content-Type: application/json" \
-H "Authorization: Bearer changeme" \
-d '{"model": "albert-testbed", "messages": [{"role": "user", "content": "Hello, how are you?"}]}'

The default master API key is changeme.

User interface

A user interface is available at: http://localhost:8081/playground

User: master Password: changeme

Create a first user

make create-user

Configure your models and add features

With configuration file, you can connect to your own models and add addtionnal services to OpenGateLLM. Start by creating a configuration file and a .env dedicated:

cp config.example.yml config.yml
cp .env.example .env

Check the configuration documentation to configure your configuration file.

Vou can then set your environment variables in .env according to your needs.

You can run the services you need by running:

docker compose --env-file .env up <services_you_need> --detach 

For instance:

docker compose --env-file .env up api playground postgres redis elasticsearch secretiveshell --detach 

Alternative: use kubernetes

You can check our helmchart and instructions here: https://github.com/etalab-ia/albert-api-helm

🤝 Contribute

This project exists thanks to all the people who contribute. OpenGateLLM thrives on open-source contributions. Join our community!

• Contribution Guide

Sponsors