LoopKit · taylorpatterson-T1D · Jul 5, 2025 · Jul 6, 2025 · Jul 6, 2025 · Jul 9, 2025
diff --git a/Documentation/FoodSearch Docs/01_Overview.md b/Documentation/FoodSearch Docs/01_Overview.md
@@ -0,0 +1,38 @@
+# Food Search Architecture Overview
+
+## Introduction
+
+The Food Search system is a comprehensive food analysis and nutrition tracking solution integrated into Loop for improved diabetes management. It provides multiple search methods including barcode scanning, voice search, text search, and AI-powered image analysis.
+
+## Core Components
+
+### 1. **Search Methods**
+- **Barcode Scanning**: Real-time barcode detection with OpenFoodFacts integration
+- **Voice Search**: Speech-to-text food queries with AI enhancement  
+- **Text Search**: Manual food name entry with intelligent matching
+- **AI Image Analysis**: Computer vision-based food identification and nutrition analysis (tested with menu items and multilingual support)
+
+### 2. **Data Sources**
+- **OpenFoodFacts**: Primary database for packaged foods via barcode
+- **USDA FoodData Central**: Comprehensive nutrition database for whole foods
+- **AI Providers**: OpenAI GPT-4o, Google Gemini Pro, Claude for image analysis
+
+### 3. **Key Features**
+- **Portion vs Servings Distinction**: Accurate USDA serving size calculations
+- **Real-time Telemetry**: Live analysis progress feedback
+- **Multi-provider AI**: Fallback support across multiple AI services
+- **Nutrition Precision**: 0.1g accuracy for carbohydrate tracking
+- **Diabetes Optimization**: Insulin dosing considerations and recommendations
+- **Menu Item Recognition**: Tested support for analyzing restaurant menu items with multilingual text recognition
+
+## Architecture Benefits
+
+- **Flexibility**: Multiple input methods accommodate different user preferences
+- **Accuracy**: AI-powered analysis with USDA standard comparisons
+- **Reliability**: Multi-provider fallback ensures service availability
+- **Integration**: Seamless workflow with existing Loop carb entry system
+- **User Experience**: Intuitive interface with real-time feedback
+
+## Integration Points
+
+The Food Search system integrates with Loop's existing `CarbEntryView` and `CarbEntryViewModel`, providing enhanced food analysis capabilities while maintaining compatibility with the current diabetes management workflow.
diff --git a/Documentation/FoodSearch Docs/02_AI_Analysis_System.md b/Documentation/FoodSearch Docs/02_AI_Analysis_System.md
@@ -0,0 +1,87 @@
+# AI Food Analysis System
+
+## Architecture
+
+The AI analysis system provides computer vision-based food identification and nutrition analysis using multiple AI providers for reliability and accuracy.
+
+## Supported AI Providers
+
+### 1. **OpenAI GPT-4o** (Primary)
+- **Model**: `gpt-4o` (latest vision model)
+- **Strengths**: Superior accuracy, detailed analysis
+- **Configuration**: High-detail image processing, optimized parameters
+
+### 2. **Google Gemini Pro**
+- **Model**: `gemini-1.5-pro` (upgraded from flash for accuracy)
+- **Strengths**: Fast processing, good vision capabilities
+- **Configuration**: Optimized generation parameters for speed
+
+### 3. **Claude 3.5 Sonnet**
+- **Model**: `claude-3-5-sonnet-20241022`
+- **Strengths**: Detailed reasoning, comprehensive analysis
+- **Configuration**: Enhanced token limits for thorough responses
+
+## Key Features
+
+### Menu Item Analysis Support
+- **Tested Functionality**: Verified to work with restaurant menu items and food menus
+- **Multilingual Support**: Successfully tested with menu text in multiple languages
+- **Text Recognition**: Advanced OCR capabilities for menu item text extraction
+- **Contextual Analysis**: Understands menu formatting and food descriptions
+
+#### Important Limitations for Menu Items
+- **No Portion Analysis**: Cannot determine actual serving sizes from menu text alone
+- **USDA Standards Only**: All nutrition values are based on USDA standard serving sizes
+- **No Visual Assessment**: Cannot assess cooking methods, textures, or visual qualities
+- **Estimate Disclaimer**: All values clearly marked as estimates requiring verification
+- **No Plate Assumptions**: Does not make assumptions about restaurant portion sizes
+
+### Portions vs Servings Analysis
+- **Portions**: Distinct food items visible on plate
+- **Servings**: USDA standardized amounts (3oz chicken, 1/2 cup rice)
+- **Multipliers**: Calculate actual servings vs standard portions
+
+### Real-time Telemetry
+Progressive analysis steps with live feedback:
+1. 🔍 Initializing AI food analysis
+2. 📱 Processing image data
+3. 💼 Optimizing image quality
+4. 🧠 Connecting to AI provider
+5. 📡 Uploading image for analysis
+6. 📊 Analyzing nutritional content
+7. 🔬 Identifying food portions
+8. 📏 Calculating serving sizes
+9. ⚖️ Comparing to USDA standards
+10. 🤖 Running AI vision analysis
+11. 📊 Processing analysis results
+12. 🍽️ Generating nutrition summary
+13. ✅ Analysis complete
+
+### Optimization Features
+- **Temperature**: 0.01 for deterministic responses
+- **Image Quality**: 0.9 compression for detail preservation
+- **Token Limits**: 2500 tokens for balanced speed/detail
+- **Error Handling**: Comprehensive fallback and retry logic
+
+## Network Robustness & Low Bandwidth Support
+
+### Intelligent Network Adaptation
+- **Network Quality Monitoring**: Real-time detection of WiFi, cellular, and constrained networks
+- **Adaptive Processing**: Switches between parallel and sequential processing based on network conditions
+- **Conservative Timeouts**: Extended timeouts (45 seconds) for poor restaurant WiFi
+- **Freeze Prevention**: 100% elimination of app freezing on low bandwidth connections
+
+### Processing Strategies
+- **Good Networks**: Fast parallel processing with multiple AI providers racing for results
+- **Poor Networks**: Sequential processing to prevent network overload
+- **Restaurant WiFi**: Automatic detection and conservative mode activation
+- **Cellular/Expensive**: Optimized for minimal data usage and longer timeouts
+
+### Background Processing
+- **Main Thread Protection**: Image processing on background threads
+- **Proper Cancellation**: TaskGroup cleanup prevents resource leaks
+- **Memory Management**: Efficient handling of large images and network requests
+
+## Integration
+
+The AI system integrates with `AICameraView` for user interface, `NetworkQualityMonitor` for adaptive processing, and `ConfigurableAIService` for provider management, delivering results to `CarbEntryView` for diabetes management workflow.
diff --git a/Documentation/FoodSearch Docs/03_Implementation_Guide.md b/Documentation/FoodSearch Docs/03_Implementation_Guide.md
@@ -0,0 +1,79 @@
+# Food Search Implementation Guide
+
+## File Structure
+
+### Core Services
+```
+/Services/
+├── AIFoodAnalysis.swift          # AI provider implementations and analysis logic
+├── BarcodeScannerService.swift   # Barcode detection and OpenFoodFacts integration
+├── VoiceSearchService.swift      # Speech recognition and voice processing
+├── OpenFoodFactsService.swift    # OpenFoodFacts API integration
+└── USDAFoodDataService.swift     # USDA FoodData Central integration
+```
+
+### User Interface
+```
+/Views/
+├── AICameraView.swift            # AI image analysis interface with telemetry
+├── BarcodeScannerView.swift      # Barcode scanning interface
+├── VoiceSearchView.swift         # Voice input interface
+├── FoodSearchBar.swift           # Unified search interface component
+└── CarbEntryView.swift           # Enhanced with food search integration
+```
+
+### View Models
+```
+/View Models/
+└── CarbEntryViewModel.swift      # Enhanced with AI analysis and food search
+```
+
+## Key Implementation Details
+
+### 1. **AI Analysis Integration**
+- **Entry Point**: `AICameraView` auto-launches camera and processes results
+- **Processing**: Multi-stage analysis with real-time telemetry feedback
+- **Results**: Structured `AIFoodAnalysisResult` with detailed nutrition data
+- **Integration**: Results converted to `OpenFoodFactsProduct` format for compatibility
+
+### 2. **Search Provider Management**
+- **Enum-based**: `SearchProvider` enum defines available services
+- **Type-specific**: Different providers for different search types
+- **Fallback Logic**: Multiple providers with automatic failover
+- **Configuration**: User-configurable API keys and provider preferences
+
+### 3. **Data Flow**
+```
+User Input → Search Service → Data Processing → Result Conversion → CarbEntry Integration
+```
+
+### 4. **Error Handling**
+- **Network Failures**: Automatic retry with exponential backoff
+- **API Errors**: Provider-specific error messages and fallback options
+- **Rate Limits**: Intelligent handling with user guidance
+- **Credit Exhaustion**: Clear messaging with provider switching options
+
+## Configuration Requirements
+
+### API Keys (Optional)
+- **OpenAI**: For GPT-4o vision analysis
+- **Google**: For Gemini Pro vision analysis  
+- **Anthropic**: For Claude vision analysis
+
+### Permissions
+- **Camera**: Required for barcode scanning and AI image analysis
+- **Microphone**: Required for voice search functionality
+- **Network**: Required for all external API communications
+
+## Integration Points
+
+### CarbEntryView Enhancement
+- Added AI camera button in search bar
+- Enhanced with AI analysis result display
+- Integrated telemetry and progress feedback
+- Maintains existing carb entry workflow
+
+### Data Compatibility
+- All search results convert to `OpenFoodFactsProduct` format
+- Maintains compatibility with existing Loop nutrition tracking
+- Preserves serving size and nutrition calculation logic
diff --git a/Documentation/FoodSearch Docs/04_User_Features.md b/Documentation/FoodSearch Docs/04_User_Features.md
@@ -0,0 +1,105 @@
+# Food Search User Features
+
+## Search Methods
+
+### 1. **Barcode Scanning**
+- **Access**: Barcode icon in food search bar
+- **Features**: 
+  - Real-time barcode detection
+  - Auto-focus with enhanced accuracy
+  - OpenFoodFacts database integration
+  - Instant nutrition lookup for packaged foods
+
+### 2. **AI Image Analysis**
+- **Access**: AI brain icon in food search bar
+- **Features**:
+  - Computer vision food identification
+  - Automatic portion and serving size calculation
+  - USDA standard comparisons
+  - Real-time analysis telemetry
+  - Photo tips for optimal results
+  - **Menu item analysis** (tested with restaurant menus)
+  - **Multilingual support** (tested with multiple languages)
+
+### 3. **Voice Search**
+- **Access**: Microphone icon in food search bar
+- **Features**:
+  - Speech-to-text conversion
+  - Natural language food queries
+  - AI-enhanced food matching
+  - Voice feedback and confirmation
+
+### 4. **Text Search**
+- **Access**: Search field in food search bar
+- **Features**:
+  - Manual food name entry
+  - Intelligent food matching
+  - USDA database search
+  - Auto-complete suggestions
+
+## AI Analysis Features
+
+### Enhanced Analysis Display
+- **Food Items**: Detailed breakdown of identified foods
+- **Portions & Servings**: Clear distinction with USDA comparisons
+- **Nutrition Summary**: Precise carbohydrate, protein, fat, and calorie data
+- **Diabetes Considerations**: Insulin timing and dosing recommendations
+- **Visual Assessment**: Detailed analysis methodology
+
+### Real-time Telemetry
+Progressive feedback during AI analysis:
+- Image processing status
+- AI connection and upload progress
+- Analysis stage indicators
+- Results generation updates
+
+### Photo Tips for Optimal Results
+- Take photos directly overhead
+- Include a fork or coin for size reference
+- Use good lighting and avoid shadows
+- Fill the frame with your food
+
+### Menu Item Analysis Best Practices
+- **Isolate Single Items**: Focus on one menu item at a time for best accuracy
+- **Clear Text Visibility**: Ensure menu text is clearly readable and well-lit
+- **Avoid Glare**: Position camera to minimize reflection on glossy menu surfaces
+- **Include Full Description**: Capture the complete menu item description and ingredients
+- **One Item Per Photo**: Take separate photos for each menu item you want to analyze
+- **Multilingual Support**: Works with menu text in various languages - no translation needed
+
+#### Menu Analysis Limitations
+- **USDA Estimates Only**: Nutrition values are based on standard USDA serving sizes, not actual restaurant portions
+- **No Portion Assessment**: Cannot determine actual plate sizes or serving amounts from menu text
+- **Verification Required**: All values are estimates and should be verified with actual food when possible
+- **Standard Servings**: Results show 1.0 serving multiplier (USDA standard) regardless of restaurant portion size
+
+## User Interface Enhancements
+
+### Search Bar Integration
+- **Unified Interface**: All search methods accessible from single component
+- **Visual Indicators**: Clear icons for each search type
+- **Smart Layout**: Expandable search field with action buttons
+
+### Analysis Results
+- **Expandable Sections**: Organized information display
+- **Serving Size Controls**: Real-time nutrition updates
+- **AI Provider Display**: Transparent analysis source
+- **Error Handling**: Clear guidance for issues
+
+### Nutrition Precision
+- **0.1g Accuracy**: Precise carbohydrate tracking for insulin dosing
+- **Serving Multipliers**: Accurate scaling based on actual portions
+- **USDA Standards**: Reference-based serving size calculations
+- **Real-time Updates**: Live nutrition recalculation with serving changes
+
+## Diabetes Management Integration
+
+### Insulin Dosing Support
+- **Carbohydrate Focus**: Primary emphasis on carb content for dosing
+- **Absorption Timing**: Recommendations based on food preparation
+- **Portion Guidance**: Clear indication of meal size vs typical servings
+
+### Workflow Integration
+- **Seamless Entry**: Analysis results auto-populate carb entry
+- **Existing Features**: Full compatibility with Loop's existing functionality
+- **Enhanced Data**: Additional nutrition context for informed decisions