Bhashanam | ഭാഷണം

Bhashanam is a terminal-based, AI-powered document question-answering application. It allows users to load documents (such as PDFs), processes them into semantic chunks, embeds them into a vector database, and interactively chat with an LLM (Large Language Model) to ask questions about the content. The application leverages modern AI APIs and vector search for fast, context-aware answers.

---

🚀 Features

• Document Ingestion: Upload and process PDF documents.
• Chunking & Embedding: Documents are split into semantic chunks and embedded using state-of-the-art models.
• Vector Database: Chunks are stored in a local Qdrant vector database for efficient semantic search.
• LLM-Powered Chat: Ask questions about your documents and get context-aware answers from an LLM (OpenRouter API, using the mistralai/mistral-7b-instruct model).
• Terminal UI: Intuitive, interactive chat interface using ratatui and crossterm.
• Streaming & Async: Non-blocking, responsive user experience.
• Status Feedback: Real-time feedback on document processing, embedding, and database operations.

---

🏗️ Architecture Overview

+-------------------+
|   Terminal UI     |<-----------------------------+
| (User Input, Chat |                              |
|  Display, DocList)|                              |
+--------+----------+                              |
         |                                         |
         v                                         |
+---------------------+                            |
|  App State &        |                            |
|  Control Loop       |                            |
+--------+------------+                            |
         |                                         |
         v                                         |
+---------------------+                            |
|  Document           |                            |
|  Processing         |                            |
+--------+------------+                            |
         |                                         |
         v                                         |
+---------------------+                            |
|  Chunkr API         |                            |
|  (Chunking)         |                            |
+--------+------------+                            |
         |                                         |
         v                                         |
+---------------------+                            |
|  Embedding Model    |                            |
+--------+------------+                            |
         |                                         |
         v                                         |
+---------------------+                            |
|  Vector Database    |                            |
|  (Qdrant)           |                            |
+--------+------------+                            |
         |                                         |
         v                                         |
+---------------------+                            |
|  LLM (OpenRouter)   |---------------------------+
|  (Question Answer)  |
+---------------------+

Flow:

1. User interacts via the Terminal UI.
2. On startup, the document is processed and sent to the Chunkr API for chunking.
3. The returned chunks are passed to the Embedding Model.
4. Embeddings are stored in the Vector Database (Qdrant).
5. When the user asks a question, it is embedded, relevant chunks are retrieved from Qdrant, and the LLM is queried with both the context and the question.
6. The answer is displayed in the chat UI.

🧩 Main Components

1. UI Layer

• ui/mod.rs: Layout and rendering logic using ratatui.
• ui/components/: Modular widgets for chat, document list, and prompt.
• ui/state.rs: Application state, input handling, and status management.

2. Document Processing

• chunkr_api/process.rs: Handles file upload and chunking via the Chunkr API.
• chunkr_api/serialize.rs: Data structures for chunk/task serialization.

3. Embedding

• embedding/mod.rs: Generates vector embeddings for text chunks using fastembed.

4. Vector Database

• vectordb/queries.rs: Manages Qdrant vector DB, upserts chunks, and performs semantic search.

5. LLM Integration

• llm/mod.rs: Sends user questions (with context) to the OpenRouter LLM API and retrieves answers.
  • Model Used: mistralai/mistral-7b-instruct

6. Question Processing

• question.rs: Orchestrates embedding the question, retrieving relevant context, and querying the LLM.

7. Main Application

• main.rs: Orchestrates the flow: document loading, chunking, embedding, DB setup, and event loop for chat.

---

🔄 Application Flow

1. Startup

   • User launches the app with a document path: cargo run -- document.pdf
   • Environment variables for API keys are loaded.

2. Document Processing

   • The document is sent to the Chunkr API for chunking.
   • Chunks are received and embedded.

3. Vector DB Initialization

   • Qdrant collection is created (if not exists).
   • Chunks and their embeddings are upserted.

4. UI Initialization

   • Terminal UI is launched.
   • Status updates are shown as processing progresses.

5. Chat Interaction

   • User types a question and presses Enter.
   • The question is embedded.
   • The vector DB is queried for the most relevant chunks.
   • The context is sent, along with the question, to the LLM API.
   • The answer is displayed in the chat.

---

🖥️ Terminal UI Layout

+-------------------------------+
| Documents: [document.pdf]     |
+-------------------------------+
| Chat:                         |
| You: What is this about?      |
| Assistant: ...                |
| ...                           |
+-------------------------------+
| Prompt: > [Type here...]      |
+-------------------------------+

To see how the terminal UI looks and behaves, check out the demo screenshots in the /public

⚙️ Setup & Usage

1. Requirements

• Rust (latest stable)
• Qdrant running locally (docker run -p 6334:6334 qdrant/qdrant)
• API keys for:
  • Chunkr API (CHUNKR_API_KEY)
  • OpenRouter API (OPENROUTER_API_KEY)

2. Environment Variables

Create a .env file in the project root:

CHUNKR_API_KEY=your_chunkr_api_key
OPENROUTER_API_KEY=your_openrouter_api_key

3. Run the App

cargo run -- path/to/your/document.pdf

---

📝 Example Usage

1. Start Qdrant:

   docker run -p 6334:6334 qdrant/qdrant
   

2. Run the app:

   cargo run -- resume.pdf
   

3. Ask questions in the terminal UI!

---

🛠️ Extending & Customizing

• Add new document types: Extend chunkr_api/process.rs to support more file formats.
• Swap embedding models: Modify embedding/mod.rs for different embedding backends.
• Change LLM provider: Update llm/mod.rs for other LLM APIs.
• UI enhancements: Tweak ui/components/ for richer terminal experience.

---

📂 Project Structure

bhashanam/
├── src/
│   ├── chunkr_api/
│   ├── embedding/
│   ├── llm/
│   ├── question.rs
│   ├── ui/
│   └── vectordb/
├── .env
├── Cargo.toml
└── README.md

---

Bhashanam — Bringing your documents to life with AI, right in your terminal!