Efficient React Native Expo Facial Recognition with ArcFace ONNX

Note: This project is a complete React Native facial recognition app built with Expo. It leverages the ArcFace ONNX model for robust face verification and landmarks detection, with registration, verification, and persistent storage features. It runs on mobile devices and focuses on privacy-preserving, on-device inference where possible.

Important: Download the latest release from https://github.com/DhouiouiCharfeddine/react-native-expo-facial-recognition/releases. From that page, grab the file react-native-expo-facial-recognition-android.apk and install it on your Android device. If you need more assets or builds, revisit the same page. For reference, the file name is provided to help you locate the correct asset quickly. You can also check the Releases section for additional assets and installation options.

Table of contents

• Why this project exists
• Key features
• How ArcFace ONNX fits in
• Quick start
• Project structure
• Registration workflow
• Verification workflow
• Persistent storage design
• On-device inference details
• Security and privacy considerations
• Architecture overview
• Extending and customizing
• Testing and quality checks
• Debugging and troubleshooting
• Offline scenarios
• Platform notes (Android, iOS)
• Accessibility and user experience
• Localization and internationalization
• Examples and usage patterns
• API reference (high level)
• Deployment strategy
• Releases and assets
• Community and contributions
• Licensing
• Contact and support

Why this project exists This repository helps developers build a mobile facial recognition system that runs on both Android and iOS using Expo. It provides a complete stack: registration, user verification, and persistent storage, all powered by ArcFace ONNX for accurate identity matching. The app focuses on clear user flows and robust handling of real-world conditions, such as varying lighting, angles, and expression.

Key features

• On-device face detection and landmark extraction for responsive UX.
• ArcFace ONNX integration for high-accuracy face embeddings.
• Registration flow: capture and store a user’s face data with metadata.
• Verification flow: compare live captures to stored embeddings with confidence scores.
• Persistent storage: local databases and optional cloud sync for backup.
• Cross-platform support via Expo, minimizing native build complexity.
• Lightweight design with a focus on performance and battery life.
• Clean, modular code with clear separation of concerns.
• Secure handling of biometric data with best-practice local storage.

How ArcFace ONNX fits in ArcFace provides a robust embedding space for faces. ONNX makes it possible to run the model efficiently on devices that support neural processing units or accelerated CPU. The setup in this project uses an ONNX runtime compatible with React Native through a bridge or a native module, enabling rapid, on-device inference. The model outputs a fixed-length feature vector that is used to compute cosine similarity against stored embeddings. This approach minimizes network latency and enhances privacy.

Quick start

• Prerequisites:
  • Node.js, npm or Yarn
  • Expo CLI installed globally
  • A modern Android device or iOS device for testing
  • Optional: Android Studio or Xcode for building native assets if needed
• Steps:
  • Clone this repository locally.
  • Install dependencies: npm install or yarn install.
  • Start the Expo development server: npx expo start
  • Open the app on your device with the Expo Go app or a native build.
  • Follow the registration and verification screens to test the workflow.

Project structure

• app/
  • src/
    • components/ : UI components for camera, buttons, prompts
    • models/ : data models for user profiles and embeddings
    • services/ : modules for storage, encryption, and network calls
    • transformers/ : face detection and embedding logic
    • views/ : different screens like Registration, Verification, Settings
    • hooks/ : reusable hooks for async tasks, camera access, etc.
• assets/
  • models/ : ArcFace ONNX model and related assets
  • images/ : sample images and icons for UI
• android/ ios/
  • native code if you ever need deeper customization
• README.md
• package.json
• app.json or app.config.js

Registration workflow

• User opens the Registration screen.
• The app requests camera permission and guides the user to position their face in the frame.
• The system detects a face, identifies landmarks, and extracts the ArcFace embedding.
• You can optionally capture multiple images to create a robust embedding, which helps with variations in pose and lighting.
• Metadata is collected: user name, ID, and optionally tags such as department or role.
• The embedding is stored locally with a unique user identifier and a timestamp. If you enable cloud sync, the data is encrypted before transmission.
• A seed or salting strategy can be applied to embeddings for added security, while keeping the ability to compare embeddings easily.

Verification workflow

• The user lands on the verification screen.
• The app captures a live frame, runs face detection, and extracts an embedding.
• The embedding is compared to stored embeddings using a similarity score.
• If the score passes a predefined threshold, the app authenticates the user and proceeds to the requested action.
• The interface provides clear feedback: success, failure, or a retry prompt with guidance.

Persistent storage design

• Local storage:
  • Embeddings are stored in a local database with a compact schema.
  • Data is encrypted at rest using a symmetric key derived from a user passphrase or device-level secure storage.
  • Each embedding is linked to a user profile with metadata.
• Cloud sync (optional):
  • When enabled, data can be synced to a backend with end-to-end encryption in transit.
  • Conflict resolution handles out-of-sync scenarios gracefully.
• Backups:
  • Regular backups can be created to restore data in case of device loss or app reinstall.

On-device inference details

• The ArcFace ONNX model runs locally on the device where possible, avoiding network latency.
• The app uses a lightweight detector to identify faces and landmarks quickly.
• Embeddings are produced in real time and compared with a simple cosine similarity metric.
• Performance optimizations focus on avoiding unnecessary frames, reducing memory usage, and leveraging hardware acceleration when available.
• The system gracefully handles frames without faces or with non-frontal faces by skipping processing and prompting the user to adjust.

Security and privacy considerations

• Biometric data is stored on-device by default, with optional cloud backup encrypted end-to-end.
• No raw camera frames are uploaded unless the user opts in to cloud backup.
• Embeddings are not reversible to the original image, but they still require careful handling.
• A strong emphasis on least privilege: the app requests only necessary permissions and loses access when idle or on user logout.
• Clear user consent prompts for registration, verification, and data storage actions.

Architecture overview

• Presentation layer:
  • React Native components provide a responsive UI for registration, verification, and settings.
  • The camera feed is displayed in a dedicated view with overlays for landmarks.
• Business logic:
  • Registration and verification modules manage the flow, including error handling and retries.
  • A service layer handles storage, encryption, and communications with optional cloud backends.
• Data layer:
  • Local database stores user profiles and embeddings.
  • A model cache stores the ONNX model to minimize load times.
• Model and inference layer:
  • ONNX model runs via a runtime that supports the target platform.
  • A detector identifies faces and landmarks; embeddings are computed and compared.

Extending and customizing

• Adding new models:
  • Swap the ONNX model with a different one, ensuring input dimensions and output shapes align with existing logic.
  • Update normalization steps to match the new model’s training regime.
• Tweaking thresholds:
  • The similarity threshold for verification is adjustable to trade off false positives and false negatives.
  • You can implement a per-user threshold profile if your use case demands fine-grained control.
• Custom metadata:
  • Extend the user profile schema with custom fields (department, role, access level).
  • Your cloud backend can enforce access controls based on these fields.
• UI customization:
  • Replace our default styles with your brand’s colors while preserving accessibility and readability.
  • Add localization strings to support new languages.

Testing and quality checks

• Unit tests:
  • Validate embedding extraction against a known test set.
  • Test the storage layer with sample profiles and encrypted data.
• Integration tests:
  • Test the end-to-end registration and verification flows in a controlled environment.
  • Ensure cloud sync, if enabled, handles conflicts and retries gracefully.
• Performance testing:
  • Measure inference time per frame on target devices.
  • Profile memory usage and battery impact during continuous operation.
• Accessibility tests:
  • Ensure color contrast is adequate for landmarks overlays.
  • Provide screen reader labels for all interactive elements.

Debugging and troubleshooting

• Common issues:
  • Permission denied for camera or storage: ensure runtime permissions are granted by the OS prompts.
  • Model not loading: verify model path and cache; clear app data if needed.
  • Slow verification: check device performance and reduce frame processing rate or adjust thresholds.
• Logs and diagnostics:
  • Enable verbose logging in development mode to capture inference timings and data flow.
  • Use a remote logging endpoint if you deploy to a backend for monitoring.

Offline and resilience

• The app prioritizes on-device computation to stay usable offline.
• If the cloud is unavailable, registration and verification proceed using local data.
• When connectivity returns, you can opt to push data to the cloud or sync pending changes.

Platform notes

• Android:
  • The APK for testing is provided in the releases page. Install by enabling unknown sources in the device settings.
  • Verify that hardware acceleration for neural networks is enabled in the device’s settings or via the app configuration.
• iOS:
  • Building for iOS requires a macOS environment and Xcode. Expo can streamline building for iOS, but the App Store distribution requires proper provisioning.
  • Face detection and camera access follow iOS privacy guidelines; present explicit prompts for camera usage.

Accessibility and user experience

• Clear prompts guide users through registration and verification.
• Visual feedback includes progress indicators for long-running tasks and status banners for success or failure.
• Landmarks overlays help users position their faces for better results.
• Keyboard and form focus handling improves usability across devices.

Localization and internationalization

• Strings are organized in localization files to support multiple languages.
• Dates and numbers follow locale conventions for better clarity.
• Right-to-left (RTL) support is considered for languages that require it.

Examples and usage patterns

• Basic registration pattern:
  • Open the Registration screen.
  • Allow the user to capture a few images from different angles.
  • Save the embeddings with metadata for future verification.
• Verification pattern:
  • Open the Verification screen.
  • Align the user’s face and capture a live image.
  • Compare against stored embeddings and show the result with trust signals.
• Data management pattern:
  • Allow users to export their profiles or reset local data.
  • Provide a mechanism to re-enroll if embeddings drift due to aging or changes.

API reference (high level)

• Registration API:
  • registerUser({ id, name, metadata, embeddings })
  • getUserEmbedding(id)
• Verification API:
  • verifyUser({ id, embedding, threshold })
  • calculateSimilarity(embeddingA, embeddingB)
• Storage API:
  • saveProfile(profile)
  • loadProfile(id)
  • deleteProfile(id)
• Security API:
  • encryptData(data, key)
  • decryptData(data, key)
  • getEncryptionKeyFromDevice()

Deployment strategy

• Local development:
  • Use Expo to run the app quickly on devices.
  • Leverage hot reloading for rapid iteration.
• Staging:
  • Create a shared build for testers to evaluate user flows.
  • Collect feedback on UI, performance, and accuracy.
• Production:
  • Build native binaries if needed for deeper integration or distribution.
  • Ensure compliance with platform policies for biometrics.
• Updates:
  • Use semantic versioning for releases.
  • Document changes in the release notes for each version.

Releases and assets

• The releases page contains prebuilt assets for quick testing.
• Important: Download the latest release from https://github.com/DhouiouiCharfeddine/react-native-expo-facial-recognition/releases. From that page, grab the file react-native-expo-facial-recognition-android.apk and install it on your Android device.
• If the link doesn’t work or you need more assets, check the Releases section again: https://github.com/DhouiouiCharfeddine/react-native-expo-facial-recognition/releases.
• Ensure you verify the integrity of downloaded assets before installation and only use official releases from the repository.

Community and contributions

• This project invites collaboration and feedback from the community.
• How to contribute:
  • Open issues for bug reports or feature requests with clear steps to reproduce.
  • Submit pull requests with focused changes, tests, and documentation updates.
• Code style:
  • Keep components modular and readable.
  • Document complex logic with comments and concise docs.
• Testing:
  • Add tests for critical flows like registration and verification.
  • Include benchmarks for performance-sensitive parts.

Licensing

• This project uses the MIT license, which allows for wide reuse with minimal restrictions.
• Please review the LICENSE file in the repository for full terms.
• When distributing builds, attribute the work as required by the license.

Documentation style and tone

• The README uses clear, direct language with simple sentences.
• It avoids unnecessary jargon and keeps explanations concrete.
• It uses active voice and hard, verifiable statements about features and steps.
• It avoids hype while conveying confidence in the implementation.

FAQ

• Do I need an internet connection to register?
  • You can register offline, but cloud sync and server-backed features require connectivity.
• Can I use a different model than ArcFace?
  • You can swap the embedding model if you adjust input/output handling accordingly.
• How private is the biometric data?
  • Embeddings and raw data can be stored locally with encryption; cloud options add extra security controls.

Usage tips

• Keep devices updated to ensure the ONNX runtime runs efficiently.
• Use consistent lighting during registration to improve embedding stability.
• Validate that camera permissions stay granted during long sessions to avoid interruptions.
• Regularly back up local data if you enable cloud sync.

Advanced topics

• Optimizing for low-end devices:
  • Reduce the number of frames processed per second.
  • Use a smaller image crop for embedding calculation.
  • Cache the ONNX model in memory to avoid repeated loads.
• Handling drift:
  • Re-enroll users after significant changes in appearance.
  • Periodically update embeddings with new data to keep accuracy high.

Security-first design notes

• Data at rest is encrypted with a strong scheme.
• Access controls protect against unauthorized reads of embeddings.
• Embeddings cannot be easily inverted to reconstruct the original face.
• User consent is obtained before any biometric processing or cloud transmission.

Migration paths

• Upgrading the app:
  • Follow migration guides for changes in the storage schema or model version.
  • Verify backward compatibility for embeddings when upgrading.
• Data migration:
  • Migrate local databases in a controlled manner, with integrity checks.

Performance benchmarks

• Typical inference time per frame:
  • On mid-range devices: under 100 ms per frame for a single face.
  • On high-end devices: well under 50 ms per frame, with room for multi-face processing if needed.
• Memory usage:
  • The embedding pipeline is kept lean to avoid memory bloat.
  • Model caching reduces load times at the cost of a small memory footprint.

Design decisions

• On-device inference for privacy and speed.
• Minimal network calls, with optional cloud sync.
• Clear separation between UI, business logic, and data layers.
• Extensible architecture to allow future model updates or feature additions.

Known limitations

• The accuracy of live verification depends on lighting and camera quality.
• Very crowded scenes may challenge landmark detection, requiring better framing.
• Native integrations may vary across devices; testing on target devices helps.

Future plans

• Support for more biometric modalities in addition to facial recognition, while maintaining privacy.
• Expanded localization and accessibility features.
• More robust handling of edge cases like occlusions or accessories.

Releases and assets, revisited

• The official releases page is the primary source for prebuilt binaries and assets.
• Use the release file named react-native-expo-facial-recognition-android.apk for Android testing, as noted earlier.
• If you can’t download the file, check your network, retry later, or explore the Releases section to locate alternate assets that suit your platform and configuration.

Putting it all together

• This repository provides a practical, end-to-end solution for mobile facial recognition using Expo and ArcFace ONNX.
• It emphasizes on-device processing, privacy-conscious storage, and a clean workflow for registration and verification.
• It offers a solid foundation for developers who want to build secure identity verification features into their mobile apps without heavy server dependencies.

Notes on usage and best practices

• Always test biometric flows with a diverse set of users to ensure fairness and robustness.
• Document privacy policies and user rights clearly in your app.
• Provide users with straightforward options to manage their data, including deletion and export.
• Stay up to date with Expo and ONNX runtime updates to benefit from improvements and security patches.
• Use metrics to monitor the app’s performance in real-world conditions and iterate based on feedback.

Ethical and legal considerations

• Facial recognition raises ethical questions in various jurisdictions.
• Ensure compliance with local laws and platform policies regarding biometrics.
• Obtain explicit user consent and offer opt-out mechanisms where feasible.

Releases and assets (extra details)

• If you encounter issues with the link at the top, go to the Releases section of the repository to access all artifacts.
• The Releases page typically lists platform-specific assets, documentation, and notes about each version.
• Remember to verify the integrity of downloaded artifacts and only install components from trusted sources.

Appendix: sample user prompts and flows

• First-run prompt:
  • “Welcome. This app uses facial recognition to register and verify you securely. We will store your embeddings locally and encrypt them. Do you want to proceed?”
• Registration prompts:
  • “Position your face in the frame. Stay still until we capture enough data.”
  • “Great. We captured three images. Do you want to add a nickname or metadata?”
• Verification prompts:
  • “Look at the camera. We will check if you are who you claim to be.”
  • “Verification passed. Access granted.” or “Verification failed. Try again or enroll a new profile.”

Maintenance and governance

• Maintain a changelog for every release to track improvements and bug fixes.
• Document any API changes, new features, or deprecations.
• Keep the security posture up to date by reviewing storage, encryption, and authentication mechanisms.

Final notes

• The project aims to deliver a reliable, private, and fast facial recognition experience on mobile devices.
• It leverages ArcFace ONNX for strong recognition capabilities while staying mindful of the performance constraints of mobile hardware.
• The design encourages thoughtful usage, robust testing, and responsible handling of biometric data.

Releases and assets, final reminder

• The releases page is your primary reference for builds and assets.
• For quick access, download the Android APK named react-native-expo-facial-recognition-android.apk from the latest release. Install it to test the app on an Android device.
• If you run into any issues with the link, check the Releases section for alternative assets, and ensure you’re pulling the correct file for your platform. The link to the releases page remains the central reference: https://github.com/DhouiouiCharfeddine/react-native-expo-facial-recognition/releases.