This is a sample Next.js ai chat application that uses Deep Infra models for inference and Vercel AI SDK.

Table of Contents

• Deploy
• Getting Started: Use the Deep Infra Vercel integration to quickly setup and run this sample app
• Manual Setup: Detailed instruction for local development
• Experiment: Try diffenet models and inference options

Deploy

Getting Started

This section assumes you have set up a Deep Infra account and project using the Vercel Integration (press deploy button above).

Step 1. Pull environment variables

You'll need a Deep Infra API key in your environment variables to connect to the model. Run the following command to pull them from Vercel:

vercel env pull

Step 2. Run the app

Run npm run dev. You can start chatting with the ai model immediately.

Manual Setup

Step 1. Deep Infra account

Create a Deep Infra account either through the Vercel marketplace integration or by directly registering at Deep Infra

Step 2. Clone the sample app

git clone git@github.com:deepinfra/deepinfra-chat.git

cd deepinfra-chat

npm install

Step 3. Set up environment variables

Copy the .env.local.example file in this directory to .env.local (which will be ignored by Git):

cp .env.local.example .env.local

From the api keys page in your Deep Infra dashboard, create a new token or use an existing one. Use that token to set the DEEPINFRA_API_KEY variable in .env.local

Experiment

By default the sample app uses model meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo specified in app/page.tsx:

const DI_MODEL = "meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo";

The actual inference calls to the model happen in app/api/chat/route.ts:

export async function POST(req: Request) {
  const { messages, model } = await req.json();

  const result = streamText({
    model: deepinfra(model),
    system: "Be a helpful assistant.",
    messages,
  });

  return result.toDataStreamResponse();
}

You can experiment with different Deep Infra models, prompts and options. See the Deep Infra docs, Vercel AI SDK docs and Deep Infra AI SDK Provider docs.