Easy LLM API

Easy LLM API allows you to quickly deploy any open-source Large Language Model (LLM) with optimized performance using platforms like Kaggle or Google Colab. This is perfect for projects like hackathons, where you need an instant and efficient LLM deployment.

Features

• Run any open-source LLM easily.
• Supports platforms like Kaggle and Google Colab for seamless execution.
• Provides an API endpoint to interact with the model.
• Uses Ngrok for instant public access.
• Built with FastAPI for high performance.
• CORS enabled for flexible integrations.

Quick Start

Run on Kaggle

Click the button below to open and run the notebook on Kaggle:

Run on Google Colab

Click the button below to run the notebook on Google Colab:

Installation

1. Clone this repository:

   git clone https://github.com/{YOUR_GITHUB_USERNAME}/easy-llm-api.git
   cd easy-llm-api

2. Install dependencies:

   pip install -r requirements.txt

3. Run the API locally:

   python main.py

API Usage

Chat Endpoint

POST /chat

Request Body:

{
    "prompt": "Hello, how are you?",
    "context": ""
}

Response:

{
    "response": "I am fine, how can I assist you?"
}

TODO List (Enhancements & Features)

• Support for additional LLM models.
• Implement model selection via API request.
• Add authentication for API access.
• Optimize inference speed with quantization.
• Provide WebSocket support for real-time chat.
• Add a front-end UI for easier interaction.
• Deploy on cloud platforms (AWS, GCP, Azure).
• Enable multi-user handling for production use.
• Enhance logging and monitoring.
• Improve documentation with example use cases.

License

This project is licensed under the MIT License.