🎙️ Vapi Personal Assistant

A Flutter-based voice assistant application built using the Vapi SDK for seamless voice interactions and raw websockets implementation for custom solutions. This project demonstrates the implementation of a voice-enabled personal assistant with real-time communication capabilities.

Preview

Todos

Simulator.-.iPhone.13.-.16.June.2025.mp4

Reminders

Simulator.-.iPhone.13.-.16.June.2025.1.mp4

Events

Simulator.-.iPhone.13.-.16.June.2025.2.mp4

Raw Websocket connection

Simulator.-.iPhone.13.-.16.June.2025.3.mp4

✨ Features

• 🗣️ Voice Interaction: Natural voice communication with AI assistant
• 🔊 Real-time Audio Processing:
  • High-quality audio streaming
  • Voice activity detection
  • Background noise handling
• 🛠️ Core Capabilities:
  • Speech-to-Text (STT) conversion
  • Text-to-Speech (TTS) synthesis
  • Real-time websocket communication
• 📱 Mobile Integration:
  • Native Flutter implementation
  • iOS and Android support
  • Permission handling
  • Audio device management

🔧 Prerequisites

• Flutter SDK ≥ 3.0.0
• Android:
  • compileSdkVersion ≥ 33
  • minSdkVersion ≥ 24
  • NDK ≥ 25.1.8937393
• Vapi API credentials

🚀 Quick Start

1. Project Setup

Clone and set up the project:

git clone 
cd vapi-personal-assistant

# Install dependencies
flutter pub get

2. Configure Credentials

Create a credentials.dart file in the lib directory:

const VAPI_API_KEY = 'Your VAPI API key';
const VAPI_ASSISTANT_ID = 'Your VAPI assistant ID';
const VAPI_PUBLIC_KEY = 'Your VAPI public key';

3. Platform Configuration

iOS Setup

Add to your Info.plist:

<key>NSMicrophoneUsageDescription</key>
<string>This app requires access to the microphone for voice communication.</string>

Update your Podfile:

post_install do |installer|
  installer.pods_project.targets.each do |target|
    flutter_additional_ios_build_settings(target)
    target.build_configurations.each do |config|
      config.build_settings['GCC_PREPROCESSOR_DEFINITIONS'] ||= [
        '$(inherited)',
        'PERMISSION_MICROPHONE=1',
      ]
    end
  end
end

Android Setup

Add to your AndroidManifest.xml:

<uses-permission android:name="android.permission.INTERNET" />
<uses-permission android:name="android.permission.RECORD_AUDIO" />
<uses-permission android:name="android.permission.MODIFY_AUDIO_SETTINGS" />

4. Run the Application

flutter run

🏗️ Project Structure

lib/
├── main.dart           # Application entry point
├── credentials.dart    # API credentials
├── vapi/              # Vapi implementation
│   ├── Vapi.dart      # Main Vapi client
│   └── vapi_event.dart # Event handling
└── screens/           # UI components
    ├── vapi_screen.dart
    └── vapi_custom_screen.dart

backend/
├── index.js           # Express server setup
├── routes/            # API endpoints
├── models/            # Database models
│   ├── Todo.js
│   ├── reminder.js
│   └── calendar.js
├── config/            # Configuration files
└── utils/            # Helper functions

🔍 Key Components

• Vapi Client: Handles communication with Vapi services
• Audio Management:
  • Recording with AudioRecorder
  • Playback with FlutterSoundPlayer
• WebSocket Communication: Real-time bidirectional data transfer
• Permission Handling: Microphone and audio settings management

🔧 Backend Implementation

The project includes a Node.js backend with Express that provides additional functionality through tool calling:

Database

• SQLite database using Sequelize ORM
• Models for todos, reminders, and calendar events
• Automatic table creation and synchronization

Tool Calling API

The backend implements several tools that can be called by the voice assistant:

Todo Management

• createTodo: Create new todo items
• getTodos: Retrieve all todos
• deleteTodo: Remove specific todos

Reminders

• addReminder: Set new reminders with importance levels
• getReminders: List all reminders
• deleteReminders: Remove specific reminders

Calendar Events

• createEvent: Schedule new calendar events
• getEvents: Retrieve all calendar events
• deleteEvent: Remove specific events

API Structure

• RESTful endpoints with VAPI request validation
• Structured response format for tool calls
• Error handling and logging
• Request validation middleware

To run the backend:

cd backend
npm install
node index.js

The server will start on http://localhost:3000 and automatically set up the database. NOTE: If you are running backend on localhost, you probably would need to setup ngrock to make Vapi tool calling work properly

📱 User Interface

The application provides two main interfaces:

• Web Call Screen: Standard voice interaction interface
• Support Call Screen: Customized voice interaction experience

📝 License

This project is licensed under the MIT License - see the LICENSE.txt file for details.

---

Created by Oleksandr Samoilenko
Extrawest.com, 2025