Built-in AI provider for Vercel AI SDK

TypeScript libraries that provide access to in-browser AI models with seamless fallback to using server-side models using the Vercel AI SDK.

Note

Only works with the new v5 of Vercel AI SDK.

• @built-in-ai/core package is the AI SDK model provider for your Chrome and Edge browser's built-in AI models.
• @built-in-ai/web-llm package is the AI SDK model provider for open-source models (using WebLLM) running directly in the browser.

We are also currently working on creating a model provider for transformers.js

Quick start

# For Chrome/Edge built-in AI models
npm i @built-in-ai/core

# For open-source models via WebLLM
npm i @built-in-ai/web-llm

Basic Usage with Chrome/Edge AI

import { streamText } from "ai";
import { builtInAI } from "@built-in-ai/core";

const result = streamText({
  model: builtInAI(),
  messages: [{ role: "user", content: "Hello, how are you?" }],
});

for await (const chunk of result.textStream) {
  console.log(chunk);
}

Basic Usage with WebLLM

import { streamText } from "ai";
import { webLLM } from "@built-in-ai/web-llm";

const result = streamText({
  model: webLLM("Llama-3.2-3B-Instruct-q4f16_1-MLC"),
  messages: [{ role: "user", content: "Hello, how are you?" }],
});

for await (const chunk of result.textStream) {
  console.log(chunk);
}

Documentation

For detailed documentation, browser requirements and advanced usage:

• @built-in-ai/core documentation
• @built-in-ai/web-llm documentation

Contributing

Contributions are more than welcome! However, please make sure to check out the contribution guidelines before contributing.

Why?

If you've ever built apps with local language models, you're likely familiar with the challenges: creating custom hooks and UI components, while also building complex integration layers to fall back to server-side models when compatibility is an issue.

This library bridges this gap by providing a unified solution that lets you:

• Experiment with in-browser AI models using familiar patterns
• Seamlessly fall back to server-side models when needed
• Use the same Vercel AI SDK eco system you already know
• Avoid building complex integration layers from scratch

While there was an existing package that attempted to provide similar functionality for Chrome, it has been inactive for the past year with no maintenance or updates. We're grateful for their initial exploration and the foundation they provided for the community.