Flutter Object Detection and Tracking Demo

This is a Flutter application that demonstrates real-time object detection and tracking using a custom TensorFlow Lite model. The application utilizes the device's camera to identify an object, display its label, and track its center with a red dot on the screen.

🚀 Features

• Real-time Object Detection: Detects objects from the live camera feed.
• Custom TFLite Model: Uses a custom TensorFlow Lite model for object detection.
• Object Tracking: Tracks the detected object and visualizes its center with a moving dot.
• Label Display: Shows the label of the detected object.
• State Management with BLoC: Manages the application state efficiently using the BLoC pattern.
• Isolate for Performance: Performs heavy object detection tasks in a separate isolate to prevent UI jank.

🛠️ Technologies & Packages Used

• Framework: Flutter
• Language: Dart
• State Management: flutter_bloc
• Camera: camera
• ML/Vision: google_mlkit_object_detection
• Isolates: integral_isolates

⚙️ Project Structure

The project follows a clean architecture, separating concerns into different layers:

lib/
├── constants/
│   └── constants.dart          # Application-wide constants
├── device/
│   └── mlkit_object_detection_camera_repository.dart # Handles camera and ML Kit logic
├── logic/
│   └── cubit/                  # BLoC state management
│       ├── object_tracker_cubit.dart
│       └── object_tracker_state.dart
├── presentation/
│   └── screens/
│       └── object_tracker_screen.dart # Main UI screen
├── widgets/
│   └── dot_painter_widget.dart # Custom painter for the tracking dot and label
└── main.dart                   # App entry point

🏃 Getting Started

Prerequisites

• Flutter SDK installed.
• An IDE like VS Code or Android Studio.
• A physical device or emulator for testing.

Installation

1. Clone the repository:

   git clone https://github.com/IoT-gamer/flutter_object_detection_tracking_demo.git
   cd flutter_object_detection_tracking_demo

2. Get Flutter packages:

   flutter pub get

3. Place your TFLite model:

   • This project expects a TensorFlow Lite model at assets/ml/.
   • If you have your own model, create an assets/ml directory and place your .tflite file there. Update the modelPath in lib/constants/constants.dart if you use a different name.
   • Here is a list of compatible pre-trained models you can use:
     • Object Detection TFLite Models
   • Requirments for pre-trained models:
     • TensorFlow Lite model compatibility
   • Ensure your pubspec.yaml file has the assets directory declared:

     flutter:
       uses-material-design: true
       assets:
         - assets/ml/

4. Android Additional Setup:

   • Ensure Gradle doesn't compress the TFLite model file. Add the following to your android/app/build.gradle.kts file:

   aaptOptions {
       noCompress("tflite")
   }

5. Run the app:

   flutter run

📖 How It Works

1. Camera Initialization: The ObjectTrackerCubit initializes the front-facing camera.
2. Image Stream: The MLKITObjectDetectionCameraRepository starts an image stream from the camera controller.
3. Isolate Processing: For each frame in the stream, the image data is sent to a separate isolate using the integral_isolates package. This prevents the heavy detection work from blocking the main UI thread.
4. Object Detection: Inside the isolate, google_mlkit_object_detection processes the image using the provided custom TFLite model (2.tflite).
5. State Update: The result, containing the center coordinates and the label of the detected object, is sent back to the main thread. The ObjectTrackerCubit receives this data and emits a new ObjectTrackerSuccess state.
6. UI Rendering: The ObjectTrackerScreen listens to state changes. When it receives a new success state, it rebuilds the UI, passing the object's center coordinates and label to the DotPainterWidget.
7. Visualization: The CustomPaint widget uses DotPainterWidget to draw a red dot and the object's label at the calculated position on top of the CameraPreview.

📄 License

This project is licensed under the MIT License. See the LICENSE file for details