A Toolkit for Building Modular LLM Applications

This repository contains a set of modular, reusable Python libraries and example applications for building sophisticated AI systems. It provides foundational tools for document processing that can be integrated with various LLM backends.

Project Philosophy

This project is not a single application, but a toolkit designed for developers. The core philosophy is to provide robust, reusable components that can be composed to build a wide variety of LLM-powered applications.

• Modularity: Each major piece of functionality is a self-contained library in its own directory, designed to be used independently or together.
• Backend Agnostic: Core components like the preprocessing library are designed to be generic and can be used with any LLM provider.
• State-of-the-Art Techniques: The libraries incorporate modern best practices, including spaCy for accurate sentence parsing and tiktoken for precise token-aware chunking.

---

Core Toolkit Libraries

solution_00_preprocessing: A Generic Document Processing Library

This module is a standalone tool for converting raw documents (PDF, TXT, HTML) into clean, structured data suitable for any language model. It has no dependency on any specific LLM provider.

• Features:
  • Extracts text from various file formats, including OCR support.
  • Performs intelligent, token-aware semantic chunking using spaCy and tiktoken.

solution_01_ollama_setup: An Integration Engine for Ollama

This library provides the specific components required to connect to and interact with a local Ollama instance. It serves as an example of how a specific LLM backend can be integrated into the system.

• Features:
  • A client for all core Ollama API interactions (generation, embeddings).
  • Enforces structured, Pydantic-validated outputs from models.
  • Includes a simple in-memory vector store for fast prototyping.

solution_03_rag_classification: Large-Scale Classification with RAG

A sophisticated system for multiclass classification using Retrieval-Augmented Generation, designed to handle thousands of categories with semantic richness.

• Features:
  • Dynamic knowledge base system that scales to thousands of classes
  • TF-IDF + embedding fusion for optimal retrieval
  • LLM reranking for precise final classification
  • Empirically tuned weights for maximum accuracy

solution_04_embedding_finetuning: Domain-Specific Embedding Adaptation

A framework for fine-tuning sentence-transformer models for specialized domains using contrastive learning.

• Features:
  • Contrastive learning with positive/negative pairs
  • Domain-specific adaptations (scientific, legal, medical texts)
  • Comprehensive evaluation with top-k accuracy metrics
  • Production-ready model versioning and deployment

---

Example Application: solution_02_chat_with_docs

To demonstrate how the core libraries can be combined, this directory contains a sophisticated Retrieval-Augmented Generation (RAG) chatbot that can answer questions about document collections.

• Demonstrated Architecture:
  • Uses the generic solution_00_preprocessing library to extract and chunk documents.
  • Integrates the solution_01_ollama_setup library for local LLM interactions.
  • Implements advanced RAG techniques like query expansion, semantic search, and source citation.
  • Processes 21 comprehensive works from the XX_test_docs collection as a working example.

Setup

This project uses uv for package and environment management.

1. Install uv: pip install uv
2. Create Environment & Install Dependencies: uv sync
3. Activate Environment: source .venv/bin/activate
4. Download spaCy Model: uv run python -m spacy download en_core_web_sm

Running the Example Application

The chatbot in solution_02_chat_with_docs is one example of what can be built with this toolkit.

First-Time Run (Building the Vector Store)

To run the example, you must first build its vector store from the source documents in XX_test_docs/ using the --reprocess flag. This is a one-time process.

uv run python -m solution_02_chat_with_docs.chatbot --reprocess

Subsequent Runs

After the initial processing, you can start the chatbot example instantly.

uv run python -m solution_02_chat_with_docs.chatbot

The application will automatically download any required Ollama models if they are not already present locally.