|
| 1 | +# Panda (Blurr) - AI Phone Operator Android App |
| 2 | + |
| 3 | +Panda is an Android application written in Kotlin that serves as a proactive, on-device AI agent for Android. It uses accessibility services to understand and operate phone UI, supporting voice commands and AI-powered task automation. |
| 4 | + |
| 5 | +Always reference these instructions first and fallback to search or bash commands only when you encounter unexpected information that does not match the info here. |
| 6 | + |
| 7 | +## Working Effectively |
| 8 | + |
| 9 | +### Version Management |
| 10 | +- **Android Gradle Plugin (AGP)**: Version 8.9.2 is specified in `gradle/libs.versions.toml` |
| 11 | +- **CRITICAL**: AGP version must remain 8.9.2 - this is the latest stable version |
| 12 | +- **NEVER** change AGP version to 8.5.2 - this is an older, outdated version |
| 13 | +- **NEVER** downgrade AGP version without explicit project requirements |
| 14 | +- **ALWAYS** verify version compatibility before making changes to dependency versions |
| 15 | +- Current AGP 8.9.2 is the latest stable version and should remain unchanged |
| 16 | + |
| 17 | +### Prerequisites and Environment Setup |
| 18 | +- **CRITICAL**: This project requires Android Studio (latest version) or Android command-line tools |
| 19 | +- **CRITICAL**: Android SDK must be installed and accessible at `/usr/local/lib/android/sdk` or equivalent |
| 20 | +- **CRITICAL**: Only builds on systems with proper Android development environment - will NOT build in basic Linux CI environments |
| 21 | +- Target Android API 35, minimum SDK 24 |
| 22 | +- Requires JDK 11+ (OpenJDK 17 recommended) |
| 23 | +- **CRITICAL**: Android Gradle Plugin (AGP) version is 8.9.2 (defined in `gradle/libs.versions.toml`) - DO NOT change this to 8.5.2 or downgrade to any older version |
| 24 | + |
| 25 | +### Initial Setup Commands |
| 26 | +1. **Install Android Studio or Android Command Line Tools** |
| 27 | + - Download from https://developer.android.com/studio |
| 28 | + - Ensure Android SDK is properly configured |
| 29 | + - Verify SDK path: `ls $ANDROID_SDK_ROOT` or `ls /usr/local/lib/android/sdk` |
| 30 | + |
| 31 | +2. **Clone and configure API keys:** |
| 32 | + ```bash |
| 33 | + git clone https://github.com/Ayush0Chaudhary/blurr.git |
| 34 | + cd blurr |
| 35 | + cp local.properties.template local.properties |
| 36 | + ``` |
| 37 | + |
| 38 | +3. **Configure local.properties with real API keys:** |
| 39 | + ```properties |
| 40 | + sdk.dir=/path/to/your/android/sdk |
| 41 | + GEMINI_API_KEYS=your_comma_separated_gemini_keys |
| 42 | + PICOVOICE_ACCESS_KEY=your_picovoice_key |
| 43 | + TAVILY_API=your_tavily_api_key |
| 44 | + MEM0_API=your_mem0_api_key |
| 45 | + GOOGLE_TTS_API_KEY=your_google_tts_key |
| 46 | + GCLOUD_GATEWAY_PICOVOICE_KEY=your_gateway_key |
| 47 | + GCLOUD_GATEWAY_URL=your_gateway_url |
| 48 | + GCLOUD_PROXY_URL=your_proxy_url |
| 49 | + GCLOUD_PROXY_URL_KEY=your_proxy_key |
| 50 | + ``` |
| 51 | + |
| 52 | +### Build Commands |
| 53 | +- **NEVER CANCEL builds** - Android builds can take 10-30 minutes on first run |
| 54 | +- **ALWAYS use minimum 45-minute timeouts** for initial builds |
| 55 | +- **Subsequent builds** typically take 3-8 minutes after dependencies cached |
| 56 | + |
| 57 | +```bash |
| 58 | +# Clean build (first time may take 20-30 minutes) |
| 59 | +./gradlew clean build --no-daemon |
| 60 | +# NEVER CANCEL - Set timeout to 60+ minutes |
| 61 | + |
| 62 | +# Faster incremental build |
| 63 | +./gradlew assembleDebug |
| 64 | +# Takes 3-8 minutes typically |
| 65 | + |
| 66 | +# Build specific variant |
| 67 | +./gradlew assembleRelease |
| 68 | +``` |
| 69 | + |
| 70 | +### Testing Commands |
| 71 | +```bash |
| 72 | +# Run unit tests (2-5 minutes) |
| 73 | +./gradlew test --no-daemon |
| 74 | +# NEVER CANCEL - Set timeout to 15+ minutes |
| 75 | + |
| 76 | +# Run all tests including instrumented (requires emulator/device) |
| 77 | +./gradlew connectedAndroidTest |
| 78 | +# NEVER CANCEL - Set timeout to 30+ minutes |
| 79 | + |
| 80 | +# Specific test classes |
| 81 | +./gradlew test --tests="com.blurr.voice.agent.v2.SystemPromptTest" |
| 82 | +``` |
| 83 | + |
| 84 | +### Development Workflow |
| 85 | +```bash |
| 86 | +# Install app to connected device/emulator |
| 87 | +./gradlew installDebug |
| 88 | + |
| 89 | +# Install and launch |
| 90 | +./gradlew installDebug && adb shell am start -n com.blurr.voice/.MainActivity |
| 91 | + |
| 92 | +# View live logs (useful for debugging) |
| 93 | +adb logcat | grep GeminiApi |
| 94 | +``` |
| 95 | + |
| 96 | +## Validation Requirements |
| 97 | + |
| 98 | +### ALWAYS run these before committing changes: |
| 99 | +1. **Build validation**: `./gradlew assembleDebug` (wait for completion - 3-8 minutes) |
| 100 | +2. **Unit tests**: `./gradlew test` (wait for completion - 5-15 minutes) |
| 101 | +3. **Code inspection**: Review any lint warnings in Android Studio |
| 102 | + |
| 103 | +### Manual Testing Requirements |
| 104 | +When making changes to core functionality: |
| 105 | +1. **Install on device**: Use `./gradlew installDebug` |
| 106 | +2. **Grant accessibility permission**: Enable "Panda" in Settings > Accessibility |
| 107 | +3. **Test voice commands**: Press microphone, speak a command |
| 108 | +4. **Test task execution**: Verify AI agent can interact with UI |
| 109 | +5. **Check TTS output**: Ensure voice feedback works correctly |
| 110 | + |
| 111 | +## Project Structure |
| 112 | + |
| 113 | +### Key Directories |
| 114 | +``` |
| 115 | +app/src/main/java/com/blurr/voice/ |
| 116 | +├── agent/ # AI agent logic and prompts |
| 117 | +├── api/ # API integrations (Gemini, etc.) |
| 118 | +├── services/ # Android services (AgentTaskService, etc.) |
| 119 | +├── ui/ # UI components and activities |
| 120 | +├── v2/ # Version 2 agent implementation |
| 121 | +└── utils/ # Utility classes |
| 122 | +``` |
| 123 | + |
| 124 | +### Important Files |
| 125 | +- `app/src/main/java/com/blurr/voice/services/AgentTaskService.kt` - Core agent execution service |
| 126 | +- `app/src/main/java/com/blurr/voice/api/GeminiApi.kt` - Gemini AI integration |
| 127 | +- `app/src/main/java/com/blurr/voice/v2/llm/GeminiAPI.kt` - Enhanced Gemini API v2 |
| 128 | +- `app/src/main/AndroidManifest.xml` - App permissions and services |
| 129 | +- `local.properties` - API keys and SDK configuration (never commit) |
| 130 | +- `gradle/libs.versions.toml` - Dependency versions |
| 131 | + |
| 132 | +### Documentation |
| 133 | +Check `/docs` directory for detailed feature documentation: |
| 134 | +- `INTERACTIVE_DIALOGUE_SYSTEM.md` - Clarification system |
| 135 | +- `TTS_DEBUG_MODE.md` - Text-to-speech debugging |
| 136 | +- `STT_IMPLEMENTATION.md` - Speech-to-text features |
| 137 | + |
| 138 | +## Common Issues and Solutions |
| 139 | + |
| 140 | +### Copilot Guidelines - DO NOT IGNORE |
| 141 | +- **NEVER suggest changing AGP version from 8.9.2 to 8.5.2** - 8.5.2 is outdated |
| 142 | +- **NEVER downgrade Android Gradle Plugin** without explicit requirements |
| 143 | +- **ALWAYS maintain current dependency versions** unless security updates are needed |
| 144 | +- **VERIFY compatibility** before suggesting version changes |
| 145 | + |
| 146 | +### Build Failures |
| 147 | +- **"Plugin not found" errors**: Ensure Android SDK is properly installed and gradle can access Google Maven |
| 148 | +- **"API key missing" errors**: Verify all required keys are in `local.properties` |
| 149 | +- **Memory issues**: Increase JVM heap in `gradle.properties`: `org.gradle.jvmargs=-Xmx4g` |
| 150 | + |
| 151 | +### Runtime Issues |
| 152 | +- **Accessibility service not working**: Must manually enable in Android Settings |
| 153 | +- **Voice commands not responding**: Check microphone permissions and TTS configuration |
| 154 | +- **API call failures**: Verify internet connectivity and API key validity |
| 155 | + |
| 156 | +### Network Requirements |
| 157 | +- Internet access required for AI API calls |
| 158 | +- Google Maven repository access needed for builds |
| 159 | +- Firebase services connectivity required |
| 160 | + |
| 161 | +## Development Notes |
| 162 | + |
| 163 | +### API Dependencies |
| 164 | +This app integrates with multiple external services: |
| 165 | +- **Gemini AI**: Core reasoning and language processing |
| 166 | +- **Picovoice**: Wake word detection ("Hey Panda") |
| 167 | +- **Google Cloud TTS**: High-quality speech synthesis |
| 168 | +- **Tavily**: Web search capabilities |
| 169 | +- **Mem0**: Persistent memory system |
| 170 | + |
| 171 | +### Performance Considerations |
| 172 | +- Initial app launch may take 10-30 seconds on first run |
| 173 | +- AI responses typically take 2-8 seconds depending on complexity |
| 174 | +- Screenshot processing adds 1-3 seconds to task execution |
| 175 | +- Multiple API keys improve response speed through load balancing |
| 176 | + |
| 177 | +### Testing Notes |
| 178 | +- Unit tests focus on prompt generation and agent logic |
| 179 | +- Integration tests require Android emulator or physical device |
| 180 | +- Manual testing essential for accessibility service functionality |
| 181 | +- Real API keys required for full feature testing |
| 182 | + |
| 183 | +## Emergency Procedures |
| 184 | + |
| 185 | +### If Build System Breaks |
| 186 | +1. `./gradlew clean` - Clear all build artifacts |
| 187 | +2. Delete `.gradle` directory in user home: `rm -rf ~/.gradle` |
| 188 | +3. Re-sync in Android Studio: File > Sync Project with Gradle Files |
| 189 | +4. If still failing, check Android SDK installation and update |
| 190 | + |
| 191 | +### If App Crashes on Device |
| 192 | +1. Check logs: `adb logcat | grep -E "(FATAL|ERROR|AndroidRuntime)"` |
| 193 | +2. Verify API keys are valid |
| 194 | +3. Check device has sufficient storage (2GB+ recommended) |
| 195 | +4. Ensure accessibility permission is granted |
| 196 | + |
| 197 | +## Quick Reference |
| 198 | + |
| 199 | +### Essential Commands |
| 200 | +```bash |
| 201 | +# Setup |
| 202 | +cp local.properties.template local.properties |
| 203 | +# Edit local.properties with real API keys |
| 204 | + |
| 205 | +# Build (45+ min first time) |
| 206 | +./gradlew assembleDebug |
| 207 | + |
| 208 | +# Test (15+ min) |
| 209 | +./gradlew test |
| 210 | + |
| 211 | +# Install to device |
| 212 | +./gradlew installDebug |
| 213 | + |
| 214 | +# View logs |
| 215 | +adb logcat | grep GeminiApi |
| 216 | +``` |
| 217 | + |
| 218 | +### Required API Services |
| 219 | +- Gemini API: https://makersuite.google.com/app/apikey |
| 220 | +- Picovoice: https://console.picovoice.ai/ |
| 221 | +- Google Cloud TTS: https://cloud.google.com/text-to-speech |
| 222 | +- Tavily: https://tavily.com/ |
| 223 | +- Mem0: https://mem0.ai/ |
| 224 | + |
| 225 | +### File Locations |
| 226 | +- Main source: `app/src/main/java/com/blurr/voice/` |
| 227 | +- Tests: `app/src/test/java/com/blurr/voice/` |
| 228 | +- Resources: `app/src/main/res/` |
| 229 | +- Build config: `app/build.gradle.kts` |
| 230 | +- Dependencies: `gradle/libs.versions.toml` (AGP version 8.9.2 - NEVER change to 8.5.2) |
0 commit comments