Welcome to TalkLLM, a lightweight and modern chat interface powered by MLC AI WebLLM. This app allows you to run a Large Language Model (LLM) directly in your browser with no backend or server required.
Built using React, this app uses the Llama 3 model locally for a private and blazing-fast AI assistant experience.
- ✅ Local-first LLM (no internet API calls)
- 💬 Seamless chat experience with context
- 📦 Runs directly in the browser using
@mlc-ai/web-llm - 🛡️ No data sent to servers
- ⚡ Clean, modern UI with loading indicators and error handling
Before running this project, make sure your system meets the following requirements:
-
✅ A browser that supports WebGPU
- Recommended: Latest Chrome, Edge, or Safari
- Check if your browser supports WebGPU
-
✅ A device with WebAssembly (Wasm) support (most modern devices have this by default)
- Node.js (v16 or above recommended)
- npm or yarn
https://github.com/mahmud-r-farhan/TalkLLM.git
cd TalkLLM
npm install
npm run dev
The app will open at http://localhost:5173.
This app uses:
Llama-3.1-8B-Instruct-q4f32_1-MLC
The model is initialized locally using:
webllm.CreateMLCEngine(modelName, { initProgressCallback });No server, no data sharing — everything runs in the browser.
src/
├── App.jsx # Main chat component
├── ui.scss # Styling
You can easily switch to another supported model by updating this line:
const selectedModel = "Llama-3.1-8B-Instruct-q4f32_1-MLC";Refer to MLC model catalog for available models.
-
React
-
@mlc-ai/web-llm
-
Sass (
.scsssupport for UI styling)
Install Sass if needed:
npm install sass
or,
npm i sass-embedded
Pull requests are welcome! Feel free to open issues or submit improvements.
- MLC AI for the awesome WebLLM engine
🧪 Tip: You can use this as a local AI assistant app without relying on OpenAI or any third-party API.